solr 8.3 indexing wrong values in some fields

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

solr 8.3 indexing wrong values in some fields

Odysci
Hi,
I have a solr cloud setup using solr 8.3 and zookeeper, which I recently
converted from solr 7.7. I converted the index using the index updater and
it all went fine. My index has about 40 million docs.
I used a separate program to check the values of all fields in the solr
docs, for consistency (e.g., fields which are supposed to have
only numbers, or only alpha chars, etc.). I ran this program immediately
after the index updating and it did not detect any problems.

Then I started regular use of the system, indexing new documents, and I
noticed that some fields were getting the wrong values. For example, a
field which was supposed to be a string with only digits had a string
containing parts of another field name. It looked as if memory was getting
corrupted. There were no error msgs in the solr logs.
In other words, solr 8.3 seems to be indexing wrong values in some fields.
This happens very few times, but it's happening.
Has anyone seen this happening?
Thanks!

Reinaldo
Reply | Threaded
Open this post in threaded view
|

Re: solr 8.3 indexing wrong values in some fields

Colvin Cowie
This sounds like https://issues.apache.org/jira/browse/SOLR-13963
Solr 8.3.1 is likely to be available soon - RC2 is at
https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.3.1-RC2-reva3d456fba2cd1b9892defbcf46a0eb4d4bb4d01f/solr/
Re-index on it, and see if you still have issues.

On Sun, 1 Dec 2019 at 17:35, Odysci <[hidden email]> wrote:

> Hi,
> I have a solr cloud setup using solr 8.3 and zookeeper, which I recently
> converted from solr 7.7. I converted the index using the index updater and
> it all went fine. My index has about 40 million docs.
> I used a separate program to check the values of all fields in the solr
> docs, for consistency (e.g., fields which are supposed to have
> only numbers, or only alpha chars, etc.). I ran this program immediately
> after the index updating and it did not detect any problems.
>
> Then I started regular use of the system, indexing new documents, and I
> noticed that some fields were getting the wrong values. For example, a
> field which was supposed to be a string with only digits had a string
> containing parts of another field name. It looked as if memory was getting
> corrupted. There were no error msgs in the solr logs.
> In other words, solr 8.3 seems to be indexing wrong values in some fields.
> This happens very few times, but it's happening.
> Has anyone seen this happening?
> Thanks!
>
> Reinaldo
>
Reply | Threaded
Open this post in threaded view
|

Re: solr 8.3 indexing wrong values in some fields

Odysci
Hi Colvin,

I updated my setup to 8.3.1-RC2 and so far it seems to work. I've converted
my 7.7 index again and indexed a whole bunch of new doc and haven't
detected any memory corruption.
Thanks a lot!

On Mon, Dec 2, 2019 at 6:40 AM Colvin Cowie <[hidden email]>
wrote:

> This sounds like https://issues.apache.org/jira/browse/SOLR-13963
> Solr 8.3.1 is likely to be available soon - RC2 is at
>
> https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.3.1-RC2-reva3d456fba2cd1b9892defbcf46a0eb4d4bb4d01f/solr/
> Re-index
> <https://dist.apache.org/repos/dist/dev/lucene/lucene-solr-8.3.1-RC2-reva3d456fba2cd1b9892defbcf46a0eb4d4bb4d01f/solr/Re-index>
> on it, and see if you still have issues.
>
> On Sun, 1 Dec 2019 at 17:35, Odysci <[hidden email]> wrote:
>
> > Hi,
> > I have a solr cloud setup using solr 8.3 and zookeeper, which I recently
> > converted from solr 7.7. I converted the index using the index updater
> and
> > it all went fine. My index has about 40 million docs.
> > I used a separate program to check the values of all fields in the solr
> > docs, for consistency (e.g., fields which are supposed to have
> > only numbers, or only alpha chars, etc.). I ran this program immediately
> > after the index updating and it did not detect any problems.
> >
> > Then I started regular use of the system, indexing new documents, and I
> > noticed that some fields were getting the wrong values. For example, a
> > field which was supposed to be a string with only digits had a string
> > containing parts of another field name. It looked as if memory was
> getting
> > corrupted. There were no error msgs in the solr logs.
> > In other words, solr 8.3 seems to be indexing wrong values in some
> fields.
> > This happens very few times, but it's happening.
> > Has anyone seen this happening?
> > Thanks!
> >
> > Reinaldo
> >
>