what is the fdx file exactly mean

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

what is the fdx file exactly mean

wangjing
.fdx file  contains, for each document, a pointer to its field data.

BUT fdx is contains pointer to WHAT? it's a pointer of  field data
offset in the fdt file?

my app is

                        File file = new File(path);
                        FSDirectory directory = FSDirectory.open(file);
                        IndexWriterConfig conf = new
IndexWriterConfig(Version.LUCENE_36,new
StandardAnalyzer(Version.LUCENE_36));
                        conf.setOpenMode(IndexWriterConfig.OpenMode.CREATE);
                        IndexWriter iw = new IndexWriter(directory, conf);
                        iw.setInfoStream(System.out);
                        Document doc = new Document();
                        Field storeYes = new Field("content1", "my storeyes test",
                                        Store.YES, Index.ANALYZED);
                        Field storeNo = new Field("content2", "my storeno test", Store.YES,
                                        Index.ANALYZED);
                        doc.add(storeYes);
                        doc.add(storeNo);
                        Document doc1 = new Document();
                        Field storeYes1 = new Field("doc1content1",
                                        "doc1 my storeyes test", Store.YES, Index.ANALYZED);
                        doc1.add(storeYes1);
                        iw.addDocument(doc1);
                        iw.addDocument(doc);
                        iw.forceMerge(1);
                        iw.close();

the fdx content is blow:
00 00 00 03 00 00 00 00  00 00 00 04 00 00 00 00
00 00 00 1D

the fdt content is
00 00 00 03 01 00 01 15  64 6F 63 31 20 6D 79 20
73 74 6F 72 65 79 65 73  20 74 65 73 74 02 01 01
10 6D 79 20 73 74 6F 72  65 79 65 73 20 74 65 73
74 02 01 0F 6D 79 20 73  74 6F 72 65 6E 6F 20 74
65 73 74


the uint64 value in fdx  is  4 and 29  in decimal ,it's which file's offset?

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what is the fdx file exactly mean

Simon Willnauer
see http://lucene.apache.org/core/3_6_0/fileformats.html#field_index
for file format documentation.

simon

On Mon, Jun 25, 2012 at 5:28 AM, wangjing <[hidden email]> wrote:

> .fdx file  contains, for each document, a pointer to its field data.
>
> BUT fdx is contains pointer to WHAT? it's a pointer of  field data
> offset in the fdt file?
>
> my app is
>
>                        File file = new File(path);
>                        FSDirectory directory = FSDirectory.open(file);
>                        IndexWriterConfig conf = new
> IndexWriterConfig(Version.LUCENE_36,new
> StandardAnalyzer(Version.LUCENE_36));
>                        conf.setOpenMode(IndexWriterConfig.OpenMode.CREATE);
>                        IndexWriter iw = new IndexWriter(directory, conf);
>                        iw.setInfoStream(System.out);
>                        Document doc = new Document();
>                        Field storeYes = new Field("content1", "my storeyes test",
>                                        Store.YES, Index.ANALYZED);
>                        Field storeNo = new Field("content2", "my storeno test", Store.YES,
>                                        Index.ANALYZED);
>                        doc.add(storeYes);
>                        doc.add(storeNo);
>                        Document doc1 = new Document();
>                        Field storeYes1 = new Field("doc1content1",
>                                        "doc1 my storeyes test", Store.YES, Index.ANALYZED);
>                        doc1.add(storeYes1);
>                        iw.addDocument(doc1);
>                        iw.addDocument(doc);
>                        iw.forceMerge(1);
>                        iw.close();
>
> the fdx content is blow:
> 00 00 00 03 00 00 00 00  00 00 00 04 00 00 00 00
> 00 00 00 1D
>
> the fdt content is
> 00 00 00 03 01 00 01 15  64 6F 63 31 20 6D 79 20
> 73 74 6F 72 65 79 65 73  20 74 65 73 74 02 01 01
> 10 6D 79 20 73 74 6F 72  65 79 65 73 20 74 65 73
> 74 02 01 0F 6D 79 20 73  74 6F 72 65 6E 6F 20 74
> 65 73 74
>
>
> the uint64 value in fdx  is  4 and 29  in decimal ,it's which file's offset?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: what is the fdx file exactly mean

wangjing
IN DOCUMENT:

Stored fields are represented by two files:

The field index, or .fdx file.
This contains, for each document, a pointer to its field data, as follows:
FieldIndex (.fdx) --> <FieldValuesPosition> SegSize
FieldValuesPosition --> Uint64
This is used to find the location within the field data file of the
fields of a particular document. Because it contains fixed-length
data, this file may be easily randomly accessed. The position of
document n 's field data is the Uint64 at n*8 in this file.


the fdx content is blow:
 00 00 00 03 00 00 00 00  00 00 00 04 00 00 00 00
 00 00 00 1D

 the fdt content is
00 00 00 03 01 00 01 15  64 6F 63 31 20 6D 79 20
73 74 6F 72 65 79 65 73  20 74 65 73 74 02 01 01
10 6D 79 20 73 74 6F 72  65 79 65 73 20 74 65 73
74 02 01 0F 6D 79 20 73  74 6F 72 65 6E 6F 20 74
65 73 74


HEX  04 and 1D is FieldValuesPosition

which Position? in fdt 04 offset is HEX 03 ?!obviously it's not  right.





On Tue, Jun 26, 2012 at 3:44 AM, Simon Willnauer
<[hidden email]> wrote:

> see http://lucene.apache.org/core/3_6_0/fileformats.html#field_index
> for file format documentation.
>
> simon
>
> On Mon, Jun 25, 2012 at 5:28 AM, wangjing <[hidden email]> wrote:
>> .fdx file  contains, for each document, a pointer to its field data.
>>
>> BUT fdx is contains pointer to WHAT? it's a pointer of  field data
>> offset in the fdt file?
>>
>> my app is
>>
>>                        File file = new File(path);
>>                        FSDirectory directory = FSDirectory.open(file);
>>                        IndexWriterConfig conf = new
>> IndexWriterConfig(Version.LUCENE_36,new
>> StandardAnalyzer(Version.LUCENE_36));
>>                        conf.setOpenMode(IndexWriterConfig.OpenMode.CREATE);
>>                        IndexWriter iw = new IndexWriter(directory, conf);
>>                        iw.setInfoStream(System.out);
>>                        Document doc = new Document();
>>                        Field storeYes = new Field("content1", "my storeyes test",
>>                                        Store.YES, Index.ANALYZED);
>>                        Field storeNo = new Field("content2", "my storeno test", Store.YES,
>>                                        Index.ANALYZED);
>>                        doc.add(storeYes);
>>                        doc.add(storeNo);
>>                        Document doc1 = new Document();
>>                        Field storeYes1 = new Field("doc1content1",
>>                                        "doc1 my storeyes test", Store.YES, Index.ANALYZED);
>>                        doc1.add(storeYes1);
>>                        iw.addDocument(doc1);
>>                        iw.addDocument(doc);
>>                        iw.forceMerge(1);
>>                        iw.close();
>>
>> the fdx content is blow:
>> 00 00 00 03 00 00 00 00  00 00 00 04 00 00 00 00
>> 00 00 00 1D
>>
>> the fdt content is
>> 00 00 00 03 01 00 01 15  64 6F 63 31 20 6D 79 20
>> 73 74 6F 72 65 79 65 73  20 74 65 73 74 02 01 01
>> 10 6D 79 20 73 74 6F 72  65 79 65 73 20 74 65 73
>> 74 02 01 0F 6D 79 20 73  74 6F 72 65 6E 6F 20 74
>> 65 73 74
>>
>>
>> the uint64 value in fdx  is  4 and 29  in decimal ,it's which file's offset?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]