Indexing a Date/DateTime/Time field in Lucene 4

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Indexing a Date/DateTime/Time field in Lucene 4

Frederik Van Hoyweghen
Hey everyone,

I'm seeing some conflicting suggestions concerning the type of field to use
for indexing a Date/DateTime/Time value.

Some suggest conversion using DateTools.timeToString() and using a
StringField,
while others suggest using the long value of getTime() and using a
LongField (this is supposed to perform better using NumericRangeQuery).

What are your opinions on this?

Kind regards,
Frederik
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Indexing a Date/DateTime/Time field in Lucene 4

Adrien Grand
Hi Frederik,

Both options would work but LongField (or LongPoint on Lucene 6.0+) would
indeed provide better performance for range queries. If you need to sort or
aggregate date values, you might also want to add a NumericDocValuesField.

Le mer. 5 avr. 2017 à 14:38, Frederik Van Hoyweghen <
[hidden email]> a écrit :

> Hey everyone,
>
> I'm seeing some conflicting suggestions concerning the type of field to use
> for indexing a Date/DateTime/Time value.
>
> Some suggest conversion using DateTools.timeToString() and using a
> StringField,
> while others suggest using the long value of getTime() and using a
> LongField (this is supposed to perform better using NumericRangeQuery).
>
> What are your opinions on this?
>
> Kind regards,
> Frederik
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Indexing a Date/DateTime/Time field in Lucene 4

Frederik Van Hoyweghen
Let's say I want to search between 2 dates, search for a date that's
before/after another, etc (the usual stuff ^^ ), is this all with either
fieldtype?

Thanks for your reply!
Frederik

On Wed, Apr 5, 2017 at 3:04 PM, Adrien Grand <[hidden email]> wrote:

> Hi Frederik,
>
> Both options would work but LongField (or LongPoint on Lucene 6.0+) would
> indeed provide better performance for range queries. If you need to sort or
> aggregate date values, you might also want to add a NumericDocValuesField.
>
> Le mer. 5 avr. 2017 à 14:38, Frederik Van Hoyweghen <
> [hidden email]> a écrit :
>
> > Hey everyone,
> >
> > I'm seeing some conflicting suggestions concerning the type of field to
> use
> > for indexing a Date/DateTime/Time value.
> >
> > Some suggest conversion using DateTools.timeToString() and using a
> > StringField,
> > while others suggest using the long value of getTime() and using a
> > LongField (this is supposed to perform better using NumericRangeQuery).
> >
> > What are your opinions on this?
> >
> > Kind regards,
> > Frederik
> >
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Indexing a Date/DateTime/Time field in Lucene 4

Uwe Schindler
Hi,

a string field is the worst idea when you want to do range queries (from...to). This will not scale and gets very slow on large indexes with many distinct datetime values! In that case you should use a long field (LongField in Lucene 5, LongPoint in 6+) and search for a long range. For sorting _always_ use long docvalues, strings need much more memory and disk IO and perform slower on sorting.

If you do not need high precision, I'd suggest to take an integer field/point and use some scaled down timestamp (like seconds instead of milliseconds or whole days). If its dates and time is unimportant, just divide by 86.400.000 (to have full day precision) and store as integer (days since 1970-01-01).

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
http://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Frederik Van Hoyweghen
> [mailto:[hidden email]]
> Sent: Wednesday, April 5, 2017 3:17 PM
> To: [hidden email]
> Subject: Re: Indexing a Date/DateTime/Time field in Lucene 4
>
> Let's say I want to search between 2 dates, search for a date that's
> before/after another, etc (the usual stuff ^^ ), is this all with either
> fieldtype?
>
> Thanks for your reply!
> Frederik
>
> On Wed, Apr 5, 2017 at 3:04 PM, Adrien Grand <[hidden email]> wrote:
>
> > Hi Frederik,
> >
> > Both options would work but LongField (or LongPoint on Lucene 6.0+)
> would
> > indeed provide better performance for range queries. If you need to sort or
> > aggregate date values, you might also want to add a
> NumericDocValuesField.
> >
> > Le mer. 5 avr. 2017 à 14:38, Frederik Van Hoyweghen <
> > [hidden email]> a écrit :
> >
> > > Hey everyone,
> > >
> > > I'm seeing some conflicting suggestions concerning the type of field to
> > use
> > > for indexing a Date/DateTime/Time value.
> > >
> > > Some suggest conversion using DateTools.timeToString() and using a
> > > StringField,
> > > while others suggest using the long value of getTime() and using a
> > > LongField (this is supposed to perform better using NumericRangeQuery).
> > >
> > > What are your opinions on this?
> > >
> > > Kind regards,
> > > Frederik
> > >
> >


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Indexing a Date/DateTime/Time field in Lucene 4

nskarthik
In reply to this post by Frederik Van Hoyweghen
I did use the Date into millisec and stored the long into index, this
helped me to convert the searched index into any date format later on the
o/p.

On Wed, Apr 5, 2017 at 6:08 PM, Frederik Van Hoyweghen <
[hidden email]> wrote:

> Hey everyone,
>
> I'm seeing some conflicting suggestions concerning the type of field to use
> for indexing a Date/DateTime/Time value.
>
> Some suggest conversion using DateTools.timeToString() and using a
> StringField,
> while others suggest using the long value of getTime() and using a
> LongField (this is supposed to perform better using NumericRangeQuery).
>
> What are your opinions on this?
>
> Kind regards,
> Frederik
>



--





*N.S.KARTHIKR.M.S.COLONYBEHIND BANK OF INDIAR.M.V 2ND STAGEBANGALORE560094*
Loading...