Adding fields with same field type complains that they have different term vector settings

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Adding fields with same field type complains that they have different term vector settings

Albert MacSweeny
Hi,

I'm upgrading a project to lucene 8.5.2 which had been using 3.0.0.

Some tests are failing with a strange issue. The gist of it is, we create fields that need position and offset information. Inserting one field works ok, but then searching for the document and adding another value for the same field results in the following exception

java.lang.IllegalArgumentException: all instances of a given field name must have the same term vectors settings (storeTermVectorPositions changed for field="f1")
    at org.apache.lucene.index.TermVectorsConsumerPerField.start(TermVectorsConsumerPerField.java:166)
    at org.apache.lucene.index.TermsHashPerField.start(TermsHashPerField.java:294)
    at org.apache.lucene.index.FreqProxTermsWriterPerField.start(FreqProxTermsWriterPerField.java:72)
    at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:810)
    at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:442)
    at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:406)
    at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
    at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:495)
    at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
    at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1213)
    at com.profium.sir.LuceneTest.writeDoc(LuceneTest.java:66)
    at com.profium.sir.LuceneTest.testLucene(LuceneTest.java:58)

This is happening even though the exact same FieldType object is being used in the field each time, and it is frozen.

I've isolated the problem to the following code snippet which reproduces it:


    import java.io.IOException;
    import java.nio.file.Path;
   
    import org.apache.lucene.analysis.en.EnglishAnalyzer;
    import org.apache.lucene.document.Document;
    import org.apache.lucene.document.Field;
    import org.apache.lucene.document.FieldType;
    import org.apache.lucene.index.DirectoryReader;
    import org.apache.lucene.index.IndexOptions;
    import org.apache.lucene.index.IndexWriter;
    import org.apache.lucene.index.IndexWriterConfig;
    import org.apache.lucene.search.IndexSearcher;
    import org.apache.lucene.store.Directory;
    import org.apache.lucene.store.MMapDirectory;
   
    public class LuceneTest {
       
        private static FieldType FIELD_TYPE = new FieldType();

        static {
            FIELD_TYPE.setStored(true);
            FIELD_TYPE.setTokenized(true);
            FIELD_TYPE.setIndexOptions(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS);
            FIELD_TYPE.setStoreTermVectors(true);
            FIELD_TYPE.setStoreTermVectorPayloads(true);
            FIELD_TYPE.setStoreTermVectorPositions(true);
            FIELD_TYPE.setStoreTermVectorOffsets(true);
            FIELD_TYPE.freeze();
        }
       
        public static void main(String[] args) throws IOException {
            testLucene();
        }      
           
        public static void testLucene() throws IOException {
            Document doc = new Document();
            doc.add(new Field("f1", "foo", FIELD_TYPE));
            writeDoc(doc);        
            IndexSearcher searcher = new IndexSearcher(DirectoryReader.open(getDirectory()));
            doc = searcher.doc(0);      
           
            doc.add(new Field("f1", "bar", FIELD_TYPE));
            writeDoc(doc);
        }
   
        private static void writeDoc(Document doc)
                throws IOException {
            Directory directory = getDirectory();
            IndexWriterConfig conf = new IndexWriterConfig(new EnglishAnalyzer());
            IndexWriter writer = new IndexWriter(directory , conf);
            writer.addDocument(doc);
            writer.flush();
            writer.close();
        }
   
        private static Directory getDirectory() throws IOException {
            return new MMapDirectory(Path.of("lucenttest"));
        }
    }

Experimenting shows that if the following three properties are not set on the FieldType, the exception is no longer thrown, but removing them breaks functionality we have that depends on the position and offset info.

 FIELD_TYPE.setStoreTermVectorPayloads(true);
 FIELD_TYPE.setStoreTermVectorPositions(true);
 FIELD_TYPE.setStoreTermVectorOffsets(true);

Perhaps I'm doing something I shouldn't be, thanks in advance for any help!

Regards,
Albert



Albert MacSweeny
Profium, Lars Sonckin kaari 12, 02600 Espoo, Finland
Tel. +358 (0)9 855 98 000  Mob. +353 (0)87 664 2560
Internet: http://www.profium.com

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Adding fields with same field type complains that they have different term vector settings

Michael McCandless-2
Hi Albert,

Unfortunately, you have fallen into a common and sneaky Lucene trap.

The problem happens because you loaded a Document from the index's stored
fields (the one you previously indexed) and then tried to modify that one
and re-index.

Lucene does not guarantee that this will work, because Lucene does not
store all information necessary to precisely reconstruct the original
document you had indexed.

The Document you loaded from the index is subtly different from the one you
had previously indexed.  In particular, your custom FIELD_TYPE details were
lost.

To sidestep this tar pit you must fully reconstruct the document yourself
each time you add it to the index.

Mike McCandless

http://blog.mikemccandless.com


On Mon, Jun 29, 2020 at 9:56 AM Albert MacSweeny <
[hidden email]> wrote:

> Hi,
>
> I'm upgrading a project to lucene 8.5.2 which had been using 3.0.0.
>
> Some tests are failing with a strange issue. The gist of it is, we create
> fields that need position and offset information. Inserting one field works
> ok, but then searching for the document and adding another value for the
> same field results in the following exception
>
> java.lang.IllegalArgumentException: all instances of a given field name
> must have the same term vectors settings (storeTermVectorPositions changed
> for field="f1")
>     at
> org.apache.lucene.index.TermVectorsConsumerPerField.start(TermVectorsConsumerPerField.java:166)
>     at
> org.apache.lucene.index.TermsHashPerField.start(TermsHashPerField.java:294)
>     at
> org.apache.lucene.index.FreqProxTermsWriterPerField.start(FreqProxTermsWriterPerField.java:72)
>     at
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:810)
>     at
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:442)
>     at
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:406)
>     at
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
>     at
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:495)
>     at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
>     at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1213)
>     at com.profium.sir.LuceneTest.writeDoc(LuceneTest.java:66)
>     at com.profium.sir.LuceneTest.testLucene(LuceneTest.java:58)
>
> This is happening even though the exact same FieldType object is being
> used in the field each time, and it is frozen.
>
> I've isolated the problem to the following code snippet which reproduces
> it:
>
>
>     import java.io.IOException;
>     import java.nio.file.Path;
>
>     import org.apache.lucene.analysis.en.EnglishAnalyzer;
>     import org.apache.lucene.document.Document;
>     import org.apache.lucene.document.Field;
>     import org.apache.lucene.document.FieldType;
>     import org.apache.lucene.index.DirectoryReader;
>     import org.apache.lucene.index.IndexOptions;
>     import org.apache.lucene.index.IndexWriter;
>     import org.apache.lucene.index.IndexWriterConfig;
>     import org.apache.lucene.search.IndexSearcher;
>     import org.apache.lucene.store.Directory;
>     import org.apache.lucene.store.MMapDirectory;
>
>     public class LuceneTest {
>
>         private static FieldType FIELD_TYPE = new FieldType();
>
>         static {
>             FIELD_TYPE.setStored(true);
>             FIELD_TYPE.setTokenized(true);
>
> FIELD_TYPE.setIndexOptions(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS);
>             FIELD_TYPE.setStoreTermVectors(true);
>             FIELD_TYPE.setStoreTermVectorPayloads(true);
>             FIELD_TYPE.setStoreTermVectorPositions(true);
>             FIELD_TYPE.setStoreTermVectorOffsets(true);
>             FIELD_TYPE.freeze();
>         }
>
>         public static void main(String[] args) throws IOException {
>             testLucene();
>         }
>
>         public static void testLucene() throws IOException {
>             Document doc = new Document();
>             doc.add(new Field("f1", "foo", FIELD_TYPE));
>             writeDoc(doc);
>             IndexSearcher searcher = new
> IndexSearcher(DirectoryReader.open(getDirectory()));
>             doc = searcher.doc(0);
>
>             doc.add(new Field("f1", "bar", FIELD_TYPE));
>             writeDoc(doc);
>         }
>
>         private static void writeDoc(Document doc)
>                 throws IOException {
>             Directory directory = getDirectory();
>             IndexWriterConfig conf = new IndexWriterConfig(new
> EnglishAnalyzer());
>             IndexWriter writer = new IndexWriter(directory , conf);
>             writer.addDocument(doc);
>             writer.flush();
>             writer.close();
>         }
>
>         private static Directory getDirectory() throws IOException {
>             return new MMapDirectory(Path.of("lucenttest"));
>         }
>     }
>
> Experimenting shows that if the following three properties are not set on
> the FieldType, the exception is no longer thrown, but removing them breaks
> functionality we have that depends on the position and offset info.
>
>  FIELD_TYPE.setStoreTermVectorPayloads(true);
>  FIELD_TYPE.setStoreTermVectorPositions(true);
>  FIELD_TYPE.setStoreTermVectorOffsets(true);
>
> Perhaps I'm doing something I shouldn't be, thanks in advance for any help!
>
> Regards,
> Albert
>
>
>
> Albert MacSweeny
> Profium, Lars Sonckin kaari 12, 02600 Espoo, Finland
> Tel. +358 (0)9 855 98 000  Mob. +353 (0)87 664 2560
> Internet: http://www.profium.com
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Adding fields with same field type complains that they have different term vector settings

Albert MacSweeny
Hi MIke,

Thanks for the quick reply. I guess this approach worked ok in version 3.0.0 since the project I'm working on relied on it? I know it's a long time ago maybe you don't remember :)

I'm worried on my side that reconstructing a full doc in this situation might have a high performance cost so I'd like to avoid it if I can (or I might not have the original value of all fields available). Do you think it would work to just reconstruct the values for the field being modified, or am I likely to just run into more issues by modifying a loaded Document?

Regards,
Albert

> From: "Michael McCandless" <[hidden email]>
> To: "java-user" <[hidden email]>, "albert macsweeny"
> <[hidden email]>
> Sent: Monday, 29 June, 2020 15:23:43
> Subject: Re: Adding fields with same field type complains that they have
> different term vector settings

> Hi Albert,
> Unfortunately, you have fallen into a common and sneaky Lucene trap.
> The problem happens because you loaded a Document from the index's stored fields
> (the one you previously indexed) and then tried to modify that one and
> re-index.

> Lucene does not guarantee that this will work, because Lucene does not store all
> information necessary to precisely reconstruct the original document you had
> indexed.

> The Document you loaded from the index is subtly different from the one you had
> previously indexed. In particular, your custom FIELD_TYPE details were lost.

> To sidestep this tar pit you must fully reconstruct the document yourself each
> time you add it to the index.

> Mike McCandless

> [ http://blog.mikemccandless.com/ | http://blog.mikemccandless.com ]

> On Mon, Jun 29, 2020 at 9:56 AM Albert MacSweeny < [
> mailto:[hidden email] | [hidden email] ] > wrote:

>> Hi,

>> I'm upgrading a project to lucene 8.5.2 which had been using 3.0.0.

>> Some tests are failing with a strange issue. The gist of it is, we create fields
>> that need position and offset information. Inserting one field works ok, but
>> then searching for the document and adding another value for the same field
>> results in the following exception

>> java.lang.IllegalArgumentException: all instances of a given field name must
>> have the same term vectors settings (storeTermVectorPositions changed for
>> field="f1")
>> at
>> org.apache.lucene.index.TermVectorsConsumerPerField.start(TermVectorsConsumerPerField.java:166)
>> at org.apache.lucene.index.TermsHashPerField.start(TermsHashPerField.java:294)
>> at
>> org.apache.lucene.index.FreqProxTermsWriterPerField.start(FreqProxTermsWriterPerField.java:72)
>> at
>> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:810)
>> at
>> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:442)
>> at
>> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:406)
>> at
>> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
>> at
>> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:495)
>> at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
>> at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1213)
>> at com.profium.sir.LuceneTest.writeDoc(LuceneTest.java:66)
>> at com.profium.sir.LuceneTest.testLucene(LuceneTest.java:58)

>> This is happening even though the exact same FieldType object is being used in
>> the field each time, and it is frozen.

>> I've isolated the problem to the following code snippet which reproduces it:

>> import java.io.IOException;
>> import java.nio.file.Path;

>> import org.apache.lucene.analysis.en.EnglishAnalyzer;
>> import org.apache.lucene.document.Document;
>> import org.apache.lucene.document.Field;
>> import org.apache.lucene.document.FieldType;
>> import org.apache.lucene.index.DirectoryReader;
>> import org.apache.lucene.index.IndexOptions;
>> import org.apache.lucene.index.IndexWriter;
>> import org.apache.lucene.index.IndexWriterConfig;
>> import org.apache.lucene.search.IndexSearcher;
>> import org.apache.lucene.store.Directory;
>> import org.apache.lucene.store.MMapDirectory;

>> public class LuceneTest {

>> private static FieldType FIELD_TYPE = new FieldType();

>> static {
>> FIELD_TYPE.setStored(true);
>> FIELD_TYPE.setTokenized(true);
>> FIELD_TYPE.setIndexOptions(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS);
>> FIELD_TYPE.setStoreTermVectors(true);
>> FIELD_TYPE.setStoreTermVectorPayloads(true);
>> FIELD_TYPE.setStoreTermVectorPositions(true);
>> FIELD_TYPE.setStoreTermVectorOffsets(true);
>> FIELD_TYPE.freeze();
>> }

>> public static void main(String[] args) throws IOException {
>> testLucene();
>> }

>> public static void testLucene() throws IOException {
>> Document doc = new Document();
>> doc.add(new Field("f1", "foo", FIELD_TYPE));
>> writeDoc(doc);
>> IndexSearcher searcher = new
>> IndexSearcher(DirectoryReader.open(getDirectory()));
>> doc = searcher.doc(0);

>> doc.add(new Field("f1", "bar", FIELD_TYPE));
>> writeDoc(doc);
>> }

>> private static void writeDoc(Document doc)
>> throws IOException {
>> Directory directory = getDirectory();
>> IndexWriterConfig conf = new IndexWriterConfig(new EnglishAnalyzer());
>> IndexWriter writer = new IndexWriter(directory , conf);
>> writer.addDocument(doc);
>> writer.flush();
>> writer.close();
>> }

>> private static Directory getDirectory() throws IOException {
>> return new MMapDirectory(Path.of("lucenttest"));
>> }
>> }

>> Experimenting shows that if the following three properties are not set on the
>> FieldType, the exception is no longer thrown, but removing them breaks
>> functionality we have that depends on the position and offset info.

>> FIELD_TYPE.setStoreTermVectorPayloads(true);
>> FIELD_TYPE.setStoreTermVectorPositions(true);
>> FIELD_TYPE.setStoreTermVectorOffsets(true);

>> Perhaps I'm doing something I shouldn't be, thanks in advance for any help!

>> Regards,
>> Albert

>> Albert MacSweeny
>> Profium, Lars Sonckin kaari 12, 02600 Espoo, Finland
>> Tel. +358 (0)9 855 98 000 Mob. +353 (0)87 664 2560
>> Internet: [ http://www.profium.com/ | http://www.profium.com ]

>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [ mailto:[hidden email] |
>> [hidden email] ]
>> For additional commands, e-mail: [ mailto:[hidden email] |
>> [hidden email] ]
Reply | Threaded
Open this post in threaded view
|

Re: Adding fields with same field type complains that they have different term vector settings

Michael McCandless-2
Hi Albert,

You're welcome.

Yes, it is likely Lucene changed behavior on this front since 3.0, and I
fear you are correct: I do not remember!

You could get away with loading a Document and making a new Document (for
indexing) from that one, if you use the loaded Document only to create e.g.
string and numeric values from the original document, but not schema level
information like whether offsets/positions are indexed into postings and
term vectors for each field, or not.  That would be safe, if you are trying
to avoid the cost of retrieving the full values for all fields from your
backing source of truth.

But you must make all such fields (that might need re-indexing) stored in
your index, so that their original value is returned when you load the
Document.  This may increase the size of your index beyond what would be
needed purely for searching.

Mike McCandless

http://blog.mikemccandless.com


On Mon, Jun 29, 2020 at 10:39 AM Albert MacSweeny <
[hidden email]> wrote:

> Hi MIke,
>
> Thanks for the quick reply. I guess this approach worked ok in version
> 3.0.0 since the project I'm working on relied on it? I know it's a long
> time ago maybe you don't remember :)
>
> I'm worried on my side that reconstructing a full doc in this situation
> might have a high performance cost so I'd like to avoid it if I can (or I
> might not have the original value of all fields available). Do you think it
> would work to just reconstruct the values for the field being modified, or
> am I likely to just run into more issues by modifying a loaded Document?
>
> Regards,
> Albert
>
> > From: "Michael McCandless" <[hidden email]>
> > To: "java-user" <[hidden email]>, "albert macsweeny"
> > <[hidden email]>
> > Sent: Monday, 29 June, 2020 15:23:43
> > Subject: Re: Adding fields with same field type complains that they have
> > different term vector settings
>
> > Hi Albert,
> > Unfortunately, you have fallen into a common and sneaky Lucene trap.
> > The problem happens because you loaded a Document from the index's
> stored fields
> > (the one you previously indexed) and then tried to modify that one and
> > re-index.
>
> > Lucene does not guarantee that this will work, because Lucene does not
> store all
> > information necessary to precisely reconstruct the original document you
> had
> > indexed.
>
> > The Document you loaded from the index is subtly different from the one
> you had
> > previously indexed. In particular, your custom FIELD_TYPE details were
> lost.
>
> > To sidestep this tar pit you must fully reconstruct the document
> yourself each
> > time you add it to the index.
>
> > Mike McCandless
>
> > [ http://blog.mikemccandless.com/ | http://blog.mikemccandless.com ]
>
> > On Mon, Jun 29, 2020 at 9:56 AM Albert MacSweeny < [
> > mailto:[hidden email] | [hidden email] ] >
> wrote:
>
> >> Hi,
>
> >> I'm upgrading a project to lucene 8.5.2 which had been using 3.0.0.
>
> >> Some tests are failing with a strange issue. The gist of it is, we
> create fields
> >> that need position and offset information. Inserting one field works
> ok, but
> >> then searching for the document and adding another value for the same
> field
> >> results in the following exception
>
> >> java.lang.IllegalArgumentException: all instances of a given field name
> must
> >> have the same term vectors settings (storeTermVectorPositions changed
> for
> >> field="f1")
> >> at
> >>
> org.apache.lucene.index.TermVectorsConsumerPerField.start(TermVectorsConsumerPerField.java:166)
> >> at
> org.apache.lucene.index.TermsHashPerField.start(TermsHashPerField.java:294)
> >> at
> >>
> org.apache.lucene.index.FreqProxTermsWriterPerField.start(FreqProxTermsWriterPerField.java:72)
> >> at
> >>
> org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:810)
> >> at
> >>
> org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:442)
> >> at
> >>
> org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:406)
> >> at
> >>
> org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:250)
> >> at
> >>
> org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:495)
> >> at
> org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1594)
> >> at
> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1213)
> >> at com.profium.sir.LuceneTest.writeDoc(LuceneTest.java:66)
> >> at com.profium.sir.LuceneTest.testLucene(LuceneTest.java:58)
>
> >> This is happening even though the exact same FieldType object is being
> used in
> >> the field each time, and it is frozen.
>
> >> I've isolated the problem to the following code snippet which
> reproduces it:
>
> >> import java.io.IOException;
> >> import java.nio.file.Path;
>
> >> import org.apache.lucene.analysis.en.EnglishAnalyzer;
> >> import org.apache.lucene.document.Document;
> >> import org.apache.lucene.document.Field;
> >> import org.apache.lucene.document.FieldType;
> >> import org.apache.lucene.index.DirectoryReader;
> >> import org.apache.lucene.index.IndexOptions;
> >> import org.apache.lucene.index.IndexWriter;
> >> import org.apache.lucene.index.IndexWriterConfig;
> >> import org.apache.lucene.search.IndexSearcher;
> >> import org.apache.lucene.store.Directory;
> >> import org.apache.lucene.store.MMapDirectory;
>
> >> public class LuceneTest {
>
> >> private static FieldType FIELD_TYPE = new FieldType();
>
> >> static {
> >> FIELD_TYPE.setStored(true);
> >> FIELD_TYPE.setTokenized(true);
> >>
> FIELD_TYPE.setIndexOptions(IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS);
> >> FIELD_TYPE.setStoreTermVectors(true);
> >> FIELD_TYPE.setStoreTermVectorPayloads(true);
> >> FIELD_TYPE.setStoreTermVectorPositions(true);
> >> FIELD_TYPE.setStoreTermVectorOffsets(true);
> >> FIELD_TYPE.freeze();
> >> }
>
> >> public static void main(String[] args) throws IOException {
> >> testLucene();
> >> }
>
> >> public static void testLucene() throws IOException {
> >> Document doc = new Document();
> >> doc.add(new Field("f1", "foo", FIELD_TYPE));
> >> writeDoc(doc);
> >> IndexSearcher searcher = new
> >> IndexSearcher(DirectoryReader.open(getDirectory()));
> >> doc = searcher.doc(0);
>
> >> doc.add(new Field("f1", "bar", FIELD_TYPE));
> >> writeDoc(doc);
> >> }
>
> >> private static void writeDoc(Document doc)
> >> throws IOException {
> >> Directory directory = getDirectory();
> >> IndexWriterConfig conf = new IndexWriterConfig(new EnglishAnalyzer());
> >> IndexWriter writer = new IndexWriter(directory , conf);
> >> writer.addDocument(doc);
> >> writer.flush();
> >> writer.close();
> >> }
>
> >> private static Directory getDirectory() throws IOException {
> >> return new MMapDirectory(Path.of("lucenttest"));
> >> }
> >> }
>
> >> Experimenting shows that if the following three properties are not set
> on the
> >> FieldType, the exception is no longer thrown, but removing them breaks
> >> functionality we have that depends on the position and offset info.
>
> >> FIELD_TYPE.setStoreTermVectorPayloads(true);
> >> FIELD_TYPE.setStoreTermVectorPositions(true);
> >> FIELD_TYPE.setStoreTermVectorOffsets(true);
>
> >> Perhaps I'm doing something I shouldn't be, thanks in advance for any
> help!
>
> >> Regards,
> >> Albert
>
> >> Albert MacSweeny
> >> Profium, Lars Sonckin kaari 12, 02600 Espoo, Finland
> >> Tel. +358 (0)9 855 98 000 Mob. +353 (0)87 664 2560
> >> Internet: [ http://www.profium.com/ | http://www.profium.com ]
>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [ mailto:
> [hidden email] |
> >> [hidden email] ]
> >> For additional commands, e-mail: [ mailto:
> [hidden email] |
> >> [hidden email] ]
>