indexing in lucene 1.9.1

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

indexing in lucene 1.9.1

Harini-2
Hi All,
We have recently upgraded from lucene 1.4.3 to lucene 1.9.1 version.
After the upgrade, we are facing some issues:
1. Indexing seems to be behaving differently. There were more than 300
segment files(.cfs) in the index and the IndexSearcher is taking forever
to refresh the index. Have there been any changes in 1.9.1 wrt default
values for merging segment files/ indexing?
2. Our application downloads documents and indexes them every min as a
continuous process. So, we have a Quartz job that refreshes the Index
Searcher every 4 hours. Would this have any effect on the indexing
process/ add more no of segments?
Any help would be appreciated.
Thanks,
Harini

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: indexing in lucene 1.9.1

OneWhoMikes
Hello Harini,

When you are finished indexing the documents are you running the
optimize() method on the IndexWriter before closing it?  This should
reduce the number of segments and make searching faster.  Just a
thought.


--Mike



On 5/22/06, Harini Raghavan <[hidden email]> wrote:

> Hi All,
> We have recently upgraded from lucene 1.4.3 to lucene 1.9.1 version.
> After the upgrade, we are facing some issues:
> 1. Indexing seems to be behaving differently. There were more than 300
> segment files(.cfs) in the index and the IndexSearcher is taking forever
> to refresh the index. Have there been any changes in 1.9.1 wrt default
> values for merging segment files/ indexing?
> 2. Our application downloads documents and indexes them every min as a
> continuous process. So, we have a Quartz job that refreshes the Index
> Searcher every 4 hours. Would this have any effect on the indexing
> process/ add more no of segments?
> Any help would be appreciated.
> Thanks,
> Harini
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: indexing in lucene 1.9.1

Harini-2
Hi Mike,

Yes you are right, when we run the optimize(), it creates one large
segment file and makes the searching faster. But  the issue is our index
keeps growing every minute as we download documents add to the index, so
we cannot call optimize so often. The indexing seemed to be fine till we
migrated to lucene 1.9.1.

I just compared the IndexWriter classes in 1.4.3 and 1.9.1 versions and
found that there are some changes wrt to creating new segments. Any idea
if that has impacted indexing? Has anyone else faced a similar issue
with the new version of lucene?

-Harini

Mike Richmond wrote:

> Hello Harini,
>
> When you are finished indexing the documents are you running the
> optimize() method on the IndexWriter before closing it?  This should
> reduce the number of segments and make searching faster.  Just a
> thought.
>
>
> --Mike
>
>
>
> On 5/22/06, Harini Raghavan <[hidden email]> wrote:
>
>> Hi All,
>> We have recently upgraded from lucene 1.4.3 to lucene 1.9.1 version.
>> After the upgrade, we are facing some issues:
>> 1. Indexing seems to be behaving differently. There were more than 300
>> segment files(.cfs) in the index and the IndexSearcher is taking forever
>> to refresh the index. Have there been any changes in 1.9.1 wrt default
>> values for merging segment files/ indexing?
>> 2. Our application downloads documents and indexes them every min as a
>> continuous process. So, we have a Quartz job that refreshes the Index
>> Searcher every 4 hours. Would this have any effect on the indexing
>> process/ add more no of segments?
>> Any help would be appreciated.
>> Thanks,
>> Harini
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]