Quantcast

solr replication failing with error: Master at: is not available. Index fetch failed

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

solr replication failing with error: Master at: is not available. Index fetch failed

geeky2
hello all,

enviornment: centOS and solr 3.5

i am attempting to set up replication betweeen two solr boxes (master and slave).

i am getting the following in the logs on the slave box.

2012-04-23 10:54:59,985 SEVERE [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Master at: <a href="http://someip:someport/somepath/somecore/admin/replication/">http://someip:someport/somepath/somecore/admin/replication/ is not available. Index fetch failed. Exception: Invalid version (expected 2, but 10) or the data in not in 'javabin' format

master jvm (jboss host) is being started like this:

-Denable.master=true

slave jvm (jboss host) is being started like this:

-Denable.slave=true

does anyone have any ideas?

i have done the following:

used curl <a href="http://someip:someport/somepath/somecore/admin/replication/">http://someip:someport/somepath/somecore/admin/replication/ from slave to successfully see master

used ping from slave to master

switched out the dns name for master to hard coded ip address

made sure i can see <a href="http://someip:someport/somepath/somecore/admin/replication/">http://someip:someport/somepath/somecore/admin/replication/ in a browser


this is my request handler - i am using the same config file on both the master and slave - but sending in the appropriate switch on start up (per the solr wiki page on replication)

    <lst name="master">

      <str name="enable">${enable.master:false}</str>
      <str name="replicateAfter">startup</str>
      <str name="replicateAfter">commit</str>



      <str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>

      <str name="commitReserveDuration">00:00:10</str>
    </lst>

    <str name="maxNumberOfBackups">1</str>
    <lst name="slave">

      <str name="enable">${enable.slave:false}</str>
      <str name="masterUrl"><a href="http://someip:someport/somecore/admin/replication/">http://someip:someport/somecore/admin/replication/</str>

      <str name="pollInterval">00:00:20</str>


      <str name="compression">internal</str>

      <str name="httpConnTimeout">5000</str>
      <str name="httpReadTimeout">10000</str>

    </lst>
  </requestHandler>


any suggestions would be great

thank you,
mark

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: solr replication failing with error: Master at: is not available. Index fetch failed

Erick Erickson
Hmmm, does your master have an index? In other words have you
added anything to it? I actually doubt that's an issue, but....

An aside, a polling interval of 20 seconds is rather short, beware of
your autowarming time exceeding your index updates....

But my _first_ guess is that somehow you're Solrs aren't the same
version or you have a foo'd index on your master.

Best
Erick

On Mon, Apr 23, 2012 at 12:10 PM, geeky2 <[hidden email]> wrote:

> hello all,
>
> enviornment: centOS and solr 3.5
>
> i am attempting to set up replication betweeen two solr boxes (master and
> slave).
>
> i am getting the following in the logs on the slave box.
>
> 2012-04-23 10:54:59,985 SEVERE [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Master at:
> <a href="http://someip:someport/somepath/somecore/admin/replication/">http://someip:someport/somepath/somecore/admin/replication/ is not
> available. Index fetch failed. Exception: Invalid version (expected 2, but
> 10) or the data in not in 'javabin' format
>
> master jvm (jboss host) is being started like this:
>
> -Denable.master=true
>
> slave jvm (jboss host) is being started like this:
>
> -Denable.slave=true
>
> does anyone have any ideas?
>
> i have done the following:
>
> used curl <a href="http://someip:someport/somepath/somecore/admin/replication/">http://someip:someport/somepath/somecore/admin/replication/ from
> slave to successfully see master
>
> used ping from slave to master
>
> switched out the dns name for master to hard coded ip address
>
> made sure i can see
> <a href="http://someip:someport/somepath/somecore/admin/replication/">http://someip:someport/somepath/somecore/admin/replication/ in a browser
>
>
> this is my request handler - i am using the same config file on both the
> master and slave - but sending in the appropriate switch on start up (per
> the solr wiki page on replication)
>
>    <lst name="master">
>
>      <str name="enable">${enable.master:false}</str>
>      <str name="replicateAfter">startup</str>
>      <str name="replicateAfter">commit</str>
>
>
>
>      <str name="confFiles">schema.xml,stopwords.txt,elevate.xml</str>
>
>      <str name="commitReserveDuration">00:00:10</str>
>    </lst>
>
>    <str name="maxNumberOfBackups">1</str>
>    <lst name="slave">
>
>      <str name="enable">${enable.slave:false}</str>
>      <str
> name="masterUrl"><a href="http://someip:someport/somecore/admin/replication/">http://someip:someport/somecore/admin/replication/</str>
>
>      <str name="pollInterval">00:00:20</str>
>
>
>      <str name="compression">internal</str>
>
>      <str name="httpConnTimeout">5000</str>
>      <str name="httpReadTimeout">10000</str>
>
>    </lst>
>  </requestHandler>
>
>
> any suggestions would be great
>
> thank you,
> mark
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/solr-replication-failing-with-error-Master-at-is-not-available-Index-fetch-failed-tp3932921p3932921.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: solr replication failing with error: Master at: is not available. Index fetch failed

geeky2
hello,

thank you for the reply,

yes - master has been indexed.

ok - makes sense - the polling interval needs to change

i did check the solr war file on both boxes (master and slave).  they are identical.  actually - if they were not indentical - this would point to a different issue altogether - since our deployment infrastructure - rolls the war file to the slaves when you do a deployment on the master.

this has me stumped - not sure what to check next.

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: solr replication failing with error: Master at: is not available. Index fetch failed

Rahul Warawdekar
Hi,

In Solr wiki, for replication, the master url is defined as follows
<str name="masterUrl"><a href="http://master_host:port">http://master_host:port
/solr/corename/replication</str>

This url does not contain "admin" in its path where as in the master url
provided by you, you have an additional "admin" in the url.
Not very sure if this might be an issue but you can just check removing
"admin" and check if replication works.


On Tue, Apr 24, 2012 at 11:49 AM, geeky2 <[hidden email]> wrote:

> hello,
>
> thank you for the reply,
>
> yes - master has been indexed.
>
> ok - makes sense - the polling interval needs to change
>
> i did check the solr war file on both boxes (master and slave).  they are
> identical.  actually - if they were not indentical - this would point to a
> different issue altogether - since our deployment infrastructure - rolls
> the
> war file to the slaves when you do a deployment on the master.
>
> this has me stumped - not sure what to check next.
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/solr-replication-failing-with-error-Master-at-is-not-available-Index-fetch-failed-tp3932921p3935699.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



--
Thanks and Regards
Rahul A. Warawdekar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: solr replication failing with error: Master at: is not available. Index fetch failed

geeky2
that was it!

thank you.

i did notice something else in the logs now ...

what is the meaning or implication of the message, "Connection reset".?



2012-04-24 12:59:19,996 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-24 12:59:39,998 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-24 12:59:59,997 SEVERE [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Master at: <a href="http://bogus:bogusport/somepath/somecore/replication/">http://bogus:bogusport/somepath/somecore/replication/ is not available. Index fetch failed. Exception: Connection reset
2012-04-24 13:00:19,998 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-24 13:00:40,004 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-24 13:00:59,992 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-24 13:01:19,993 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-24 13:01:39,992 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-24 13:01:59,989 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-24 13:02:19,990 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-24 13:02:39,989 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-24 13:02:59,991 INFO  [org.a
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: solr replication failing with error: Master at: is not available. Index fetch failed

Rahul Warawdekar
Hi,

Is the replication still failing or working fine with that change ?

On Tue, Apr 24, 2012 at 2:16 PM, geeky2 <[hidden email]> wrote:

> that was it!
>
> thank you.
>
> i did notice something else in the logs now ...
>
> what is the meaning or implication of the message, "Connection reset".?
>
>
>
> 2012-04-24 12:59:19,996 INFO  [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Slave in sync with master.
> 2012-04-24 12:59:39,998 INFO  [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Slave in sync with master.
> *2012-04-24 12:59:59,997 SEVERE [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Master at:
> <a href="http://bogus:bogusport/somepath/somecore/replication/">http://bogus:bogusport/somepath/somecore/replication/ is not available.
> Index fetch failed. Exception: Connection reset*
> 2012-04-24 13:00:19,998 INFO  [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Slave in sync with master.
> 2012-04-24 13:00:40,004 INFO  [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Slave in sync with master.
> 2012-04-24 13:00:59,992 INFO  [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Slave in sync with master.
> 2012-04-24 13:01:19,993 INFO  [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Slave in sync with master.
> 2012-04-24 13:01:39,992 INFO  [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Slave in sync with master.
> 2012-04-24 13:01:59,989 INFO  [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Slave in sync with master.
> 2012-04-24 13:02:19,990 INFO  [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Slave in sync with master.
> 2012-04-24 13:02:39,989 INFO  [org.apache.solr.handler.SnapPuller]
> (pool-12-thread-1) Slave in sync with master.
> 2012-04-24 13:02:59,991 INFO  [org.a
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/solr-replication-failing-with-error-Master-at-is-not-available-Index-fetch-failed-tp3932921p3936107.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



--
Thanks and Regards
Rahul A. Warawdekar
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: solr replication failing with error: Master at: is not available. Index fetch failed

geeky2
hello,

sorry - i overlooked this message - thanks for checking back and thanks for the info.

yes - replication seems to be working now:

tailed from logs just now:

2012-04-26 09:21:33,284 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-26 09:21:53,279 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-26 09:22:13,279 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.
2012-04-26 09:22:33,279 INFO  [org.apache.solr.handler.SnapPuller] (pool-12-thread-1) Slave in sync with master.



 
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: solr replication failing with error: Master at: is not available. Index fetch failed

Mark Miller-3
In reply to this post by geeky2

On Apr 23, 2012, at 12:10 PM, geeky2 wrote:

> <a href="http://someip:someport/somepath/somecore/admin/replication/">http://someip:someport/somepath/somecore/admin/replication/ is not
> available. Index fetch failed. Exception: Invalid version (expected 2, but
> 10) or the data in not in 'javabin' format

This is kind of a bug. When Solr tries to talk in javabin and gets an http response instead (like a 404 response - what this likely is) it does this. Really it should detect this case and give you the proper error. I almost think someone made this change already in trunk based on what I was seeing yesterday, but I'm not sure.

- Mark Miller
lucidimagination.com











Loading...