Rounding dates on sort and filter

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Rounding dates on sort and filter

Charlie Jackson
I've got a legacy date field that I'd like to round for sorting and
filtering. Right now, the index is large enough that sorting or
filtering on a date field takes 10-20 seconds (unless it's cached). I
know this is because the date field's precision is down to the
millisecond, and I don't really need that level of precision for most of
my searches. So, is it possible to round my field at query time without
having to reindex the field or add a second one?

 

I already tried the function sorting in 1.5-dev, but my field isn't a
TrieDate field so I can't use the ms() function (which seems to allow
date math unlike the other functions).

 

Thanks,

Charlie

Reply | Threaded
Open this post in threaded view
|

Re: Rounding dates on sort and filter

Otis Gospodnetic-2
Charlie,

Query-time terms/tokens need to match what's in your index, and my guess is that if you just altered query-time date field analysis, you'd get a mismatch.  Easy enough to check through Solr Admin Analysis page.

Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch



----- Original Message ----

> From: Charlie Jackson <[hidden email]>
> To: [hidden email]
> Sent: Tue, January 19, 2010 1:20:02 PM
> Subject: Rounding dates on sort and filter
>
> I've got a legacy date field that I'd like to round for sorting and
> filtering. Right now, the index is large enough that sorting or
> filtering on a date field takes 10-20 seconds (unless it's cached). I
> know this is because the date field's precision is down to the
> millisecond, and I don't really need that level of precision for most of
> my searches. So, is it possible to round my field at query time without
> having to reindex the field or add a second one?
>
>
>
> I already tried the function sorting in 1.5-dev, but my field isn't a
> TrieDate field so I can't use the ms() function (which seems to allow
> date math unlike the other functions).
>
>
>
> Thanks,
>
> Charlie

Reply | Threaded
Open this post in threaded view
|

RE: Rounding dates on sort and filter

Charlie Jackson
Good point. So it doesn't sound like there's a way to do this without
adding a new field or reindexing. Thanks anyway.

- Charlie


-----Original Message-----
From: Otis Gospodnetic [mailto:[hidden email]]
Sent: Tuesday, January 19, 2010 2:04 PM
To: [hidden email]
Subject: Re: Rounding dates on sort and filter

Charlie,

Query-time terms/tokens need to match what's in your index, and my guess
is that if you just altered query-time date field analysis, you'd get a
mismatch.  Easy enough to check through Solr Admin Analysis page.

Otis
--
Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch



----- Original Message ----

> From: Charlie Jackson <[hidden email]>
> To: [hidden email]
> Sent: Tue, January 19, 2010 1:20:02 PM
> Subject: Rounding dates on sort and filter
>
> I've got a legacy date field that I'd like to round for sorting and
> filtering. Right now, the index is large enough that sorting or
> filtering on a date field takes 10-20 seconds (unless it's cached). I
> know this is because the date field's precision is down to the
> millisecond, and I don't really need that level of precision for most
of
> my searches. So, is it possible to round my field at query time
without

> having to reindex the field or add a second one?
>
>
>
> I already tried the function sorting in 1.5-dev, but my field isn't a
> TrieDate field so I can't use the ms() function (which seems to allow
> date math unlike the other functions).
>
>
>
> Thanks,
>
> Charlie

Reply | Threaded
Open this post in threaded view
|

Re: Rounding dates on sort and filter

Lance Norskog-2
The precision of the date should not matter that much in the time for
the first sort. Lucene makes a pair of arrays for the sorted field,
one with each unique date and one with each document number in the
index. (Yes, the entire index.)  The first array will be shorter when
you cut the date precision, but the latter will not shrink at all.

On Tue, Jan 19, 2010 at 12:50 PM, Charlie Jackson
<[hidden email]> wrote:

> Good point. So it doesn't sound like there's a way to do this without
> adding a new field or reindexing. Thanks anyway.
>
> - Charlie
>
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:[hidden email]]
> Sent: Tuesday, January 19, 2010 2:04 PM
> To: [hidden email]
> Subject: Re: Rounding dates on sort and filter
>
> Charlie,
>
> Query-time terms/tokens need to match what's in your index, and my guess
> is that if you just altered query-time date field analysis, you'd get a
> mismatch.  Easy enough to check through Solr Admin Analysis page.
>
> Otis
> --
> Sematext -- http://sematext.com/ -- Solr - Lucene - Nutch
>
>
>
> ----- Original Message ----
>> From: Charlie Jackson <[hidden email]>
>> To: [hidden email]
>> Sent: Tue, January 19, 2010 1:20:02 PM
>> Subject: Rounding dates on sort and filter
>>
>> I've got a legacy date field that I'd like to round for sorting and
>> filtering. Right now, the index is large enough that sorting or
>> filtering on a date field takes 10-20 seconds (unless it's cached). I
>> know this is because the date field's precision is down to the
>> millisecond, and I don't really need that level of precision for most
> of
>> my searches. So, is it possible to round my field at query time
> without
>> having to reindex the field or add a second one?
>>
>>
>>
>> I already tried the function sorting in 1.5-dev, but my field isn't a
>> TrieDate field so I can't use the ms() function (which seems to allow
>> date math unlike the other functions).
>>
>>
>>
>> Thanks,
>>
>> Charlie
>
>



--
Lance Norskog
[hidden email]