Building a codec with terms of custom Comparator<BytesRef>

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Building a codec with terms of custom Comparator<BytesRef>

John Wang-9
Hi guys:

   In the codec api, it seems you can set term order via an arbitrary Comparator<BytesRef>. 

   I tried to use this to create a term dictionary of an order dictated by my own Comparator<BytesRef>.

   The problem arises when building the FST. Specifically BlockTreeTermsWriter.finishTerm() <- here the ordering decided earlier by the Comparator<BytesRef> (from codec) is lost, and errors due to out of order.

   Any ideas on how to fix this?

Thanks

-John 
Reply | Threaded
Open this post in threaded view
|

Re: Building a codec with terms of custom Comparator<BytesRef>

Robert Muir
You can implement your own term dictionary with a different order: but
BlockTreeTermsReader doesnt support this (its terms must be in binary
order)

On Mon, Sep 9, 2013 at 8:27 PM, John Wang <[hidden email]> wrote:

> Hi guys:
>
>    In the codec api, it seems you can set term order via an arbitrary
> Comparator<BytesRef>.
>
>    I tried to use this to create a term dictionary of an order dictated by
> my own Comparator<BytesRef>.
>
>    The problem arises when building the FST. Specifically
> BlockTreeTermsWriter.finishTerm() <- here the ordering decided earlier by
> the Comparator<BytesRef> (from codec) is lost, and errors due to out of
> order.
>
>    Any ideas on how to fix this?
>
> Thanks
>
> -John

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]