Lucene Migration query

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Lucene Migration query

Adarsh Sunilkumar
Hi Team,

Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1. Should
I do reindexing in this case ?
Can I make use of backward codec jar without a reindex?


Thanks & Regards,
Adarsh Sunilkumar
Reply | Threaded
Open this post in threaded view
|

RE: Lucene Migration query

Uwe Schindler
Hi,

> Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1. Should
> I do reindexing in this case ?

No, you don't need that.

> Can I make use of backward codec jar without a reindex?

Yes, just add the JAR file to your classpath and it can read the indexes. Updates written to the index will use the new codecs. To force a full upgrade (rewrite all segments), invoke the IndexUpgrader class either from your code or using the command line. But this is not needed, it just makes sure that you can get rid of the backwards-codecs jar.

Uwe


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene Migration query

Michael Sokolov-4
I think running the upgrade tool would also be necessary to set you up for
the next upgrade, when 9.0 comes along.

On Fri, Nov 20, 2020, 4:25 AM Uwe Schindler <[hidden email]> wrote:

> Hi,
>
> > Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1.
> Should
> > I do reindexing in this case ?
>
> No, you don't need that.
>
> > Can I make use of backward codec jar without a reindex?
>
> Yes, just add the JAR file to your classpath and it can read the indexes.
> Updates written to the index will use the new codecs. To force a full
> upgrade (rewrite all segments), invoke the IndexUpgrader class either from
> your code or using the command line. But this is not needed, it just makes
> sure that you can get rid of the backwards-codecs jar.
>
> Uwe
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Lucene Migration query

Erick Erickson
The IndexUpgraderTool does a forceMerge(1). If you have a large index,
that has its own problems, but will work. The threshold for the issues is
5G. See: https://lucidworks.com/post/solr-and-optimizing-your-index-take-ii/
I should emphasize that if you have a very large single segment as a
result, it’ll eventually shrink if it accumulated deleted (or updated) documents,
it’ll just require a bunch of I/O amortized over time.

IndexUpgraderTool will _not_ allow you to take an index originally created with
7x to be used in 9x. (Uwe, I’ve been telling people this for a long time, if I’ve
been lying please let me know!). Starting with Lucene 6, a version is written into
each segment. Upon merge, the lowest version stamp is preserved. Lucene
will refuse to open an index where _any_ segment has a version stamp X-2 or
older.

Best,
Erick

> On Nov 20, 2020, at 7:57 AM, Michael Sokolov <[hidden email]> wrote:
>
> I think running the upgrade tool would also be necessary to set you up for
> the next upgrade, when 9.0 comes along.
>
> On Fri, Nov 20, 2020, 4:25 AM Uwe Schindler <[hidden email]> wrote:
>
>> Hi,
>>
>>> Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1.
>> Should
>>> I do reindexing in this case ?
>>
>> No, you don't need that.
>>
>>> Can I make use of backward codec jar without a reindex?
>>
>> Yes, just add the JAR file to your classpath and it can read the indexes.
>> Updates written to the index will use the new codecs. To force a full
>> upgrade (rewrite all segments), invoke the IndexUpgrader class either from
>> your code or using the command line. But this is not needed, it just makes
>> sure that you can get rid of the backwards-codecs jar.
>>
>> Uwe
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene Migration query

Michael Sokolov-4
Ah, sorry for the misdirection, thanks for the correction, Erick. That
does jibe with what I now remember having heard before. I guess we
reserve the right to create index data structures in the future for
which we did not save sufficient data in the past.

On Fri, Nov 20, 2020 at 9:15 AM Erick Erickson <[hidden email]> wrote:

>
> The IndexUpgraderTool does a forceMerge(1). If you have a large index,
> that has its own problems, but will work. The threshold for the issues is
> 5G. See: https://lucidworks.com/post/solr-and-optimizing-your-index-take-ii/
> I should emphasize that if you have a very large single segment as a
> result, it’ll eventually shrink if it accumulated deleted (or updated) documents,
> it’ll just require a bunch of I/O amortized over time.
>
> IndexUpgraderTool will _not_ allow you to take an index originally created with
> 7x to be used in 9x. (Uwe, I’ve been telling people this for a long time, if I’ve
> been lying please let me know!). Starting with Lucene 6, a version is written into
> each segment. Upon merge, the lowest version stamp is preserved. Lucene
> will refuse to open an index where _any_ segment has a version stamp X-2 or
> older.
>
> Best,
> Erick
>
> > On Nov 20, 2020, at 7:57 AM, Michael Sokolov <[hidden email]> wrote:
> >
> > I think running the upgrade tool would also be necessary to set you up for
> > the next upgrade, when 9.0 comes along.
> >
> > On Fri, Nov 20, 2020, 4:25 AM Uwe Schindler <[hidden email]> wrote:
> >
> >> Hi,
> >>
> >>> Currently I am using Lucene 7.3, I want to upgrade to lucene 8.5.1.
> >> Should
> >>> I do reindexing in this case ?
> >>
> >> No, you don't need that.
> >>
> >>> Can I make use of backward codec jar without a reindex?
> >>
> >> Yes, just add the JAR file to your classpath and it can read the indexes.
> >> Updates written to the index will use the new codecs. To force a full
> >> upgrade (rewrite all segments), invoke the IndexUpgrader class either from
> >> your code or using the command line. But this is not needed, it just makes
> >> sure that you can get rid of the backwards-codecs jar.
> >>
> >> Uwe
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]