Internal Server Error and waitSearcher="false" for commit/optimize

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Internal Server Error and waitSearcher="false" for commit/optimize

Jason Rennie-2
Hello,

We're using solr 1.2 and a nightly build of the solrj client code.  We very
occasionally see things like this:

org.apache.solr.client.solrj.SolrServerException: Error executing query
        at org.apache.solr.client.solrj.request.QueryRequest.process(
QueryRequest.java:86)
        at org.apache.solr.client.solrj.impl.BaseSolrServer.query(
BaseSolrServer.java:99)
...
Caused by: org.apache.solr.common.SolrException: Internal Server Error

I'm guessing that this might be due to solr being in the middle of a commit
or optimize.  Could solr throw an exception like that in this case?

We also occasionally see solr taking too long to respond.  We currently make
our commit/optimize calls without any arguments.  I'm wondering whether
setting waitSearcher="false" might allow search queries to be served while a
commit/optimize is being run.  I found this in an old message from this
list:

> Yes, it looks like there is no difference... the code to make commit
> totally asynchronous was never put in (so you can't really get commit
> to return instantly, it will always wait until the IndexWriter is closed).


This isn't a problem for us as the thread making the commit/optimize call is
separate from thread(s) making queries.  Is waitSearcher="false" designed to
allow queries to be processed while a commit/optimize is being run?  Are
there any negative side effects to this setting (other than a query being
slightly out-of-date :)?

Thanks,

Jason
Reply | Threaded
Open this post in threaded view
|

Re: Internal Server Error and waitSearcher="false" for commit/optimize

Yonik Seeley-2
On 10/10/07, Jason Rennie <[hidden email]> wrote:

> We're using solr 1.2 and a nightly build of the solrj client code.  We very
> occasionally see things like this:
>
> org.apache.solr.client.solrj.SolrServerException: Error executing query
>         at org.apache.solr.client.solrj.request.QueryRequest.process(
> QueryRequest.java:86)
>         at org.apache.solr.client.solrj.impl.BaseSolrServer.query(
> BaseSolrServer.java:99)
> ...
> Caused by: org.apache.solr.common.SolrException: Internal Server Error

Is there a longer stack trace somewhere concerning the internal server error?

> We also occasionally see solr taking too long to respond.  We currently make
> our commit/optimize calls without any arguments.  I'm wondering whether
> setting waitSearcher="false" might allow search queries to be served while a
> commit/optimize is being run.  I found this in an old message from this
> list:

While commit/optimize is being run, requests are served using the old
searcher - there shouldn't be any blocking.

> Is waitSearcher="false" designed to
> allow queries to be processed while a commit/optimize is being run?

No, waitSearcher="true" was designed such that a client could do a
commit, and wait for a new searcher to be registered such that a new
query request is guaranteed to see the changes.
waitSearcher=true/false only affects the thread calling commit... it
has no effect on other query requests which will continue to use the
previous searcher until the new one is registered.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: Internal Server Error and waitSearcher="false" for commit/optimize

Jason Rennie-2
Many thanks for your reply, Yonik.

On 10/11/07, Yonik Seeley <[hidden email]> wrote:
>
> > Caused by: org.apache.solr.common.SolrException: Internal Server Error
>
> Is there a longer stack trace somewhere concerning the internal server
> error?


I shouldn't have bothered you with this.  We've discovered two causes (1)
mal-formed query string, and (2) OOME.  I should have done more homework...

No, waitSearcher="true" was designed such that a client could do a
> commit, and wait for a new searcher to be registered such that a new
> query request is guaranteed to see the changes.
> waitSearcher=true/false only affects the thread calling commit... it
> has no effect on other query requests which will continue to use the
> previous searcher until the new one is registered.
>

Thanks for the explanation.  This is exactly what we needed to know.  Our
query threads are separate from the commit/optimize thread, so this option
would not affect operations.

In case you're curious, we use solr as the search engine for
www.stylefeeder.com.  It has served us very well so far, handling over 3000
queries/day.

Thanks,

Jason

--
Jason Rennie
Head of Machine Learning Technologies, StyleFeeder
http://www.stylefeeder.com/