Solr performance warnings

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr performance warnings

Michael Thessel
Hello,

in my logs I get from time to time this message:

INFO: PERFORMANCE WARNING: Overlapping onDeckSearchers=2

What does this mean? What can I do to avoid this?


Cheers,

Michael
Reply | Threaded
Open this post in threaded view
|

Re: Solr performance warnings

Erik Hatcher

On Apr 19, 2007, at 7:47 PM, Michael Thessel wrote:
> in my logs I get from time to time this message:
>
> INFO: PERFORMANCE WARNING: Overlapping onDeckSearchers=2
>
> What does this mean? What can I do to avoid this?

I think you have issued multiple commits (or optimizes) that hadn't  
fully finished.   Is that the case?

        Erik

Reply | Threaded
Open this post in threaded view
|

Re: Solr performance warnings

Michael Thessel
Hey Erik,

thanks for the fast reply. Yes this could be possible. I currently got
solr running for the indexing of a forum with 100k users. It could
definitely be possible that two commits overlap. But I need to commit
all changes because the new posts must be available in the search as
soon as they are posted.

Do you think there is a way to optimize this?

Cheers,

Michael



Erik Hatcher wrote:

>
> On Apr 19, 2007, at 7:47 PM, Michael Thessel wrote:
>> in my logs I get from time to time this message:
>>
>> INFO: PERFORMANCE WARNING: Overlapping onDeckSearchers=2
>>
>> What does this mean? What can I do to avoid this?
>
> I think you have issued multiple commits (or optimizes) that hadn't
> fully finished.   Is that the case?
>
>     Erik
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr performance warnings

Mike Klaas
On 4/20/07, Michael Thessel <[hidden email]> wrote:
> Hey Erik,
>
> thanks for the fast reply. Yes this could be possible. I currently got
> solr running for the indexing of a forum with 100k users. It could
> definitely be possible that two commits overlap. But I need to commit
> all changes because the new posts must be available in the search as
> soon as they are posted.
>
> Do you think there is a way to optimize this?

"As soon as" is a rather vague requirement.  If you can specify the
minimum acceptible delay, then you can use Solr's autocommit
functionality to trigger commits.

-Mike
Reply | Threaded
Open this post in threaded view
|

Re: Solr performance warnings

Michael Thessel
Mike Klaas wrote:

> On 4/20/07, Michael Thessel <[hidden email]> wrote:
>> Hey Erik,
>>
>> thanks for the fast reply. Yes this could be possible. I currently got
>> solr running for the indexing of a forum with 100k users. It could
>> definitely be possible that two commits overlap. But I need to commit
>> all changes because the new posts must be available in the search as
>> soon as they are posted.
>>
>> Do you think there is a way to optimize this?
>
> "As soon as" is a rather vague requirement.  If you can specify the
> minimum acceptible delay, then you can use Solr's autocommit
> functionality to trigger commits.
>
> -Mike

I didn't know about the timed commits. That's perfect for me.

Thanks,

Michael
Reply | Threaded
Open this post in threaded view
|

Re: Solr performance warnings

Michael Thessel
Michael Thessel wrote:

> Mike Klaas wrote:
>> On 4/20/07, Michael Thessel <[hidden email]> wrote:
>>> Hey Erik,
>>>
>>> thanks for the fast reply. Yes this could be possible. I currently got
>>> solr running for the indexing of a forum with 100k users. It could
>>> definitely be possible that two commits overlap. But I need to commit
>>> all changes because the new posts must be available in the search as
>>> soon as they are posted.
>>>
>>> Do you think there is a way to optimize this?
>>
>> "As soon as" is a rather vague requirement.  If you can specify the
>> minimum acceptible delay, then you can use Solr's autocommit
>> functionality to trigger commits.
>>
>> -Mike
>
> I didn't know about the timed commits. That's perfect for me.
>
> Thanks,
>
> Michael

The timed commits don't work for me. The webinterface says 0 commits
since the server was restarted. And nothing in the logs as well.

I use:
apache-solr-1.1.0-incubating


My updateHandler section from solrconfig.xml:

<updateHandler class="solr.DirectUpdateHandler2">
   <autoCommit>
     <maxTime>10000</maxTime>
   </autoCommit>
</updateHandler>

I also tried <maxTime>10</maxTime> in case its seconds and not ms.

Cheers Michael

Reply | Threaded
Open this post in threaded view
|

Re: Solr performance warnings

Mike Klaas
On 4/20/07, Michael Thessel <[hidden email]> wrote:
>
> The timed commits don't work for me. The webinterface says 0 commits
> since the server was restarted. And nothing in the logs as well.
>
> I use:
> apache-solr-1.1.0-incubating

Sorry about that--I see that that change was added a few weeks after
1.1 was cut.  I suggest using a nightly build from Feb 2 or later, or
waiting until 1.2 is released.

cheers,
-Mike