performance differences between 1.4.3 and 1.9.1

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

performance differences between 1.4.3 and 1.9.1

RONALD MANTAY
Hi chaps ,
   
  I ran the same search code with lucene-1.4.3.jar and then with lucene-core-1.9.1.jar
   
  The good news is there appeared to be a performance improvement with 1.9.1 both with single index searching both exact and fuzzy mode,
   
  However when searching muliple indexes with multiSearcher and with a FuzzyQuery with a prefixLength of 1. The search against 3.7m documents spread over 23 indexes (due to the natural grouping of the data) the time changed from 800ms to 4500 ms.
   
  Has any one else had similar observations ? any Ideas on how to address it ?
   
  Regards
Reply | Threaded
Open this post in threaded view
|

Re: performance differences between 1.4.3 and 1.9.1

Daniel Naber-5
On Mittwoch 26 April 2006 01:22, RONALD MANTAY wrote:

>   However when searching muliple indexes with multiSearcher and with a
> FuzzyQuery with a prefixLength of 1. The search against 3.7m documents
> spread over 23 indexes (due to the natural grouping of the data) the
> time changed from 800ms to 4500 ms.

MultiSearcher in Lucene 1.4 had a broken ranking implementation. This has
been fixed in Lucene 1.9, but this might have bad effects on performance.
23 indexes is quite much, maybe you can speed up things greatly be using a
smaller number of indexes.

Regards
 Daniel

--
http://www.danielnaber.de

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: performance differences between 1.4.3 and 1.9.1

Andy Goodell-2
For my application we have several hundred indexes, different subsets
of which are searched depending on the situation.  Aside from not
upgrading to lucene 1.9, or making a big index for every possible
subset, do you have any ideas for how can we maintain fast
performance?

- andy g

On 4/26/06, Daniel Naber <[hidden email]> wrote:

> MultiSearcher in Lucene 1.4 had a broken ranking implementation. This has
> been fixed in Lucene 1.9, but this might have bad effects on performance.
> 23 indexes is quite much, maybe you can speed up things greatly be using a
> smaller number of indexes.
>
> Regards
>  Daniel
>
> --
> http://www.danielnaber.de

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: performance differences between 1.4.3 and 1.9.1

Karl Wettin-3

27 apr 2006 kl. 02.18 skrev Andy Goodell:

> For my application we have several hundred indexes, different subsets
> of which are searched depending on the situation.  Aside from not
> upgrading to lucene 1.9, or making a big index for every possible
> subset, do you have any ideas for how can we maintain fast
> performance?

You probably need to explain the reason for splitting them up in  
order to get a good answer to that. And how big are they?

Without knowing anything about your application I say: merge them all  
to one and add a field you apply to a boolean clause.
But with a few hundred indices it sounds like you have a design plan  
that don't work with above.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]