Re-created fields consistently indexed?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re-created fields consistently indexed?

SnowCrash
Hi all,

I understand that it is possible to "re-create" fields which are indexed but
not stored (as is done by Luke), and that this is a lossy process, however I
am wondering whether the indexed version of this remains consistent.

That is, if I re-create a non-stored field, then re-index this field in a
new document, will the indexed (inverted) data be the same as in the
original version?

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: Re-created fields consistently indexed?

Erik Hatcher

On Aug 30, 2006, at 11:07 AM, Jason Polites wrote:

> I understand that it is possible to "re-create" fields which are  
> indexed but
> not stored (as is done by Luke), and that this is a lossy process,  
> however I
> am wondering whether the indexed version of this remains consistent.
>
> That is, if I re-create a non-stored field, then re-index this  
> field in a
> new document, will the indexed (inverted) data be the same as in the
> original version?

That really all depends.  For example, the original analysis process  
could put in position increment gaps between sentences or overlay  
terms in same positions, and don't forget about term offsets.   Just  
pulling out the terms without getting those details would be even  
lossier.  I'm not even sure how you could recreate that even if you  
had all of that position and offset information without some kind of  
special Analyzer that would set things exactly as they were.  I  
suspect there is some low-level tricks that could be played to do  
this bypassing an analyzer.

        Erik


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]