[jira] Created: (SOLR-71) New support for "Date Math" when adding/quering date fields

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (SOLR-71) New support for "Date Math" when adding/quering date fields

JIRA jira@apache.org
New support for "Date Math" when adding/quering date fields
-----------------------------------------------------------

                 Key: SOLR-71
                 URL: http://issues.apache.org/jira/browse/SOLR-71
             Project: Solr
          Issue Type: New Feature
          Components: search, update
            Reporter: Hoss Man
         Assigned To: Hoss Man


New utility class and changes to DateField to support syntax like the following...

          startDate:[* TO NOW]
          startDate:[* TO NOW/DAY+1DAY]
          expirationDate:[NOW/DAY TO *]
          reviewDate:[NOW/DAY-1YEAR TO NOW/DAY]
          validDate:[NOW/MONTH TO NOW/MONTH+1MONTH-1MILLISECOND]

...where + and - mean what you think, and "/UNIT" rounds down to the nearest UNIT.  The motivation for this being that date range queries like these are usefull for filters, but being date sensitve can't currently be "baked in" to a config as default params.

a nice side effect of the implimentation, is that "timestamp" fields can be done with a document is added by using...

   <field name="myTimestampField">NOW</field>

...and Solr will compute the value when adding the document ... if we add default values to the schema.xml even that won't be neccessary.


Comments?  

(I'd be particularly gratefull if smarter people then I would sanity check my use of ThreadLocal for managing the DateFormat in DateField ... i've never used ThreadLocal before.  Any general comments on the syntax would also be appreciated: This left-to-right syntax seemed more intuative to write (and easier to parse) then some of the other syntaxes I'd considered)



--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-71) New support for "Date Math" when adding/quering date fields

JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/SOLR-71?page=all ]

Hoss Man updated SOLR-71:
-------------------------

    Attachment: DateMath.patch


Code, and tests ... if you readd the exampledocs the "incubationdate_dt" field is populated for the Solr product so you can play with it...

incubationdate_dt:[* TO NOW-1DAY+3HOURS/MINUTE] ... matches.
incubationdate_dt:[* TO NOW-2YEARS]  ... does not match.

> New support for "Date Math" when adding/quering date fields
> -----------------------------------------------------------
>
>                 Key: SOLR-71
>                 URL: http://issues.apache.org/jira/browse/SOLR-71
>             Project: Solr
>          Issue Type: New Feature
>          Components: search, update
>            Reporter: Hoss Man
>         Assigned To: Hoss Man
>         Attachments: DateMath.patch
>
>
> New utility class and changes to DateField to support syntax like the following...
>           startDate:[* TO NOW]
>           startDate:[* TO NOW/DAY+1DAY]
>           expirationDate:[NOW/DAY TO *]
>           reviewDate:[NOW/DAY-1YEAR TO NOW/DAY]
>           validDate:[NOW/MONTH TO NOW/MONTH+1MONTH-1MILLISECOND]
> ...where + and - mean what you think, and "/UNIT" rounds down to the nearest UNIT.  The motivation for this being that date range queries like these are usefull for filters, but being date sensitve can't currently be "baked in" to a config as default params.
> a nice side effect of the implimentation, is that "timestamp" fields can be done with a document is added by using...
>    <field name="myTimestampField">NOW</field>
> ...and Solr will compute the value when adding the document ... if we add default values to the schema.xml even that won't be neccessary.
> Comments?  
> (I'd be particularly gratefull if smarter people then I would sanity check my use of ThreadLocal for managing the DateFormat in DateField ... i've never used ThreadLocal before.  Any general comments on the syntax would also be appreciated: This left-to-right syntax seemed more intuative to write (and easier to parse) then some of the other syntaxes I'd considered)

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-71) New support for "Date Math" when adding/quering date fields

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/SOLR-71?page=comments#action_12450956 ]
           
Yonik Seeley commented on SOLR-71:
----------------------------------

Sweet!

A couple of very minor notes regarding ThreadLocal:
 - initialValue doesn't need to be synchronized... it's just that way in the javadoc example because they were incrementing a global counter
 - the trick you are trying in the ThreadLocalDateFormat aren't necessary.  no method will be called until the constructor has completed.  the java memory model also ensures that you see a completely constructed object from another thread, even if you haven't synchronized.  The extra variable won't hurt either... the JVM will completely optimize it away (so this is just an FYI).


> New support for "Date Math" when adding/quering date fields
> -----------------------------------------------------------
>
>                 Key: SOLR-71
>                 URL: http://issues.apache.org/jira/browse/SOLR-71
>             Project: Solr
>          Issue Type: New Feature
>          Components: search, update
>            Reporter: Hoss Man
>         Assigned To: Hoss Man
>         Attachments: DateMath.patch
>
>
> New utility class and changes to DateField to support syntax like the following...
>           startDate:[* TO NOW]
>           startDate:[* TO NOW/DAY+1DAY]
>           expirationDate:[NOW/DAY TO *]
>           reviewDate:[NOW/DAY-1YEAR TO NOW/DAY]
>           validDate:[NOW/MONTH TO NOW/MONTH+1MONTH-1MILLISECOND]
> ...where + and - mean what you think, and "/UNIT" rounds down to the nearest UNIT.  The motivation for this being that date range queries like these are usefull for filters, but being date sensitve can't currently be "baked in" to a config as default params.
> a nice side effect of the implimentation, is that "timestamp" fields can be done with a document is added by using...
>    <field name="myTimestampField">NOW</field>
> ...and Solr will compute the value when adding the document ... if we add default values to the schema.xml even that won't be neccessary.
> Comments?  
> (I'd be particularly gratefull if smarter people then I would sanity check my use of ThreadLocal for managing the DateFormat in DateField ... i've never used ThreadLocal before.  Any general comments on the syntax would also be appreciated: This left-to-right syntax seemed more intuative to write (and easier to parse) then some of the other syntaxes I'd considered)

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (SOLR-71) New support for "Date Math" when adding/quering date fields

Chris Hostetter-3

:  - initialValue doesn't need to be synchronized... it's just that way in
: the javadoc example because they were incrementing a global counter

Ah, good eye.

:  - the trick you are trying in the ThreadLocalDateFormat aren't
: necessary.  no method will be called until the constructor has
: completed.  the java memory model also ensures that you see a completely
: constructed object from another thread, even if you haven't
: synchronized.  The extra variable won't hurt either... the JVM will
: completely optimize it away (so this is just an FYI).

I think you are giving me more credit then i deserve ... what trick do you
think i'm trying to accomplish?

You're talking about the local "tmp" formatter and which is then assigned
to "proto" at the end? ... that wasn't out of any fear that it would be
used before i set the timezeon, it was just because my inate distaste for
declaring member variables with concrete types wouldn't let write
"SimpleDateFormat proto;" ... so i could either use a tmp var, or cast it
to call setTimeZone -- and if there is one thing i loath more then member
vars with concrete types, it's casting (I was very bitter to discover that
Clonable isn't a parametric type in java5 ... i relly wanted to write...

  private static class ThreadLocalDateFormat extends ThreadLocal<DateFormat> {
    Clonable<DateFormat> proto;
    public ThreadLocalDateFormat() {
        ...
    }
    protected synchronized DateFormat initialValue() {
      return proto.clone();
    }
  }




-Hoss

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-71) New support for "Date Math" when adding/quering date fields

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
    [ http://issues.apache.org/jira/browse/SOLR-71?page=comments#action_12450981 ]
           
Bertrand Delacretaz commented on SOLR-71:
-----------------------------------------

This looks like a very useful feature!

> New support for "Date Math" when adding/quering date fields
> -----------------------------------------------------------
>
>                 Key: SOLR-71
>                 URL: http://issues.apache.org/jira/browse/SOLR-71
>             Project: Solr
>          Issue Type: New Feature
>          Components: search, update
>            Reporter: Hoss Man
>         Assigned To: Hoss Man
>         Attachments: DateMath.patch
>
>
> New utility class and changes to DateField to support syntax like the following...
>           startDate:[* TO NOW]
>           startDate:[* TO NOW/DAY+1DAY]
>           expirationDate:[NOW/DAY TO *]
>           reviewDate:[NOW/DAY-1YEAR TO NOW/DAY]
>           validDate:[NOW/MONTH TO NOW/MONTH+1MONTH-1MILLISECOND]
> ...where + and - mean what you think, and "/UNIT" rounds down to the nearest UNIT.  The motivation for this being that date range queries like these are usefull for filters, but being date sensitve can't currently be "baked in" to a config as default params.
> a nice side effect of the implimentation, is that "timestamp" fields can be done with a document is added by using...
>    <field name="myTimestampField">NOW</field>
> ...and Solr will compute the value when adding the document ... if we add default values to the schema.xml even that won't be neccessary.
> Comments?  
> (I'd be particularly gratefull if smarter people then I would sanity check my use of ThreadLocal for managing the DateFormat in DateField ... i've never used ThreadLocal before.  Any general comments on the syntax would also be appreciated: This left-to-right syntax seemed more intuative to write (and easier to parse) then some of the other syntaxes I'd considered)

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|

[jira] Resolved: (SOLR-71) New support for "Date Math" when adding/quering date fields

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org
     [ http://issues.apache.org/jira/browse/SOLR-71?page=all ]

Hoss Man resolved SOLR-71.
--------------------------

    Resolution: Fixed

commited with some small modifications:

1) got rid of the unneeded synchronized Yonik pointed out
2) improved the javadocs a bit
3) added mention of DateMath in example schema.xml
4) added an example of a "baked in" Date Math query to the example solrconfig.xml

> New support for "Date Math" when adding/quering date fields
> -----------------------------------------------------------
>
>                 Key: SOLR-71
>                 URL: http://issues.apache.org/jira/browse/SOLR-71
>             Project: Solr
>          Issue Type: New Feature
>          Components: update, search
>            Reporter: Hoss Man
>         Assigned To: Hoss Man
>         Attachments: DateMath.patch
>
>
> New utility class and changes to DateField to support syntax like the following...
>           startDate:[* TO NOW]
>           startDate:[* TO NOW/DAY+1DAY]
>           expirationDate:[NOW/DAY TO *]
>           reviewDate:[NOW/DAY-1YEAR TO NOW/DAY]
>           validDate:[NOW/MONTH TO NOW/MONTH+1MONTH-1MILLISECOND]
> ...where + and - mean what you think, and "/UNIT" rounds down to the nearest UNIT.  The motivation for this being that date range queries like these are usefull for filters, but being date sensitve can't currently be "baked in" to a config as default params.
> a nice side effect of the implimentation, is that "timestamp" fields can be done with a document is added by using...
>    <field name="myTimestampField">NOW</field>
> ...and Solr will compute the value when adding the document ... if we add default values to the schema.xml even that won't be neccessary.
> Comments?  
> (I'd be particularly gratefull if smarter people then I would sanity check my use of ThreadLocal for managing the DateFormat in DateField ... i've never used ThreadLocal before.  Any general comments on the syntax would also be appreciated: This left-to-right syntax seemed more intuative to write (and easier to parse) then some of the other syntaxes I'd considered)

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira