[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

Hudson (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239711#comment-13239711 ]

Harley Parks commented on SOLR-2155:
------------------------------------

interesting... conversation regarding both tuning and data requirements.
I would guess that the size needs to be in power of 2's

since the example given is related to a custom type, it should be noted that a fieldValueCache is by default created, for each document id.

the example given by default in solr 3.4 if needed:
       <fieldValueCache class="solr.FastLRUCache"
                        size="512"
                        autowarmCount="128"
                        showItems="32" />

Additionally, the custom cache uses the same name, which might not matter, or does it? the custom cache may override the fieldValueCache created by default.
               

> Geospatial search using geohash prefixes
> ----------------------------------------
>
>                 Key: SOLR-2155
>                 URL: https://issues.apache.org/jira/browse/SOLR-2155
>             Project: Solr
>          Issue Type: Improvement
>            Reporter: David Smiley
>         Attachments: GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch, SOLR-2155_GeoHashPrefixFilter_with_sorting_no_poly.patch, SOLR.2155.p3.patch, SOLR.2155.p3tests.patch, Solr2155-1.0.2-project.zip, Solr2155-1.0.3-project.zip, Solr2155-1.0.4-project.zip, Solr2155-for-1.0.2-3.x-port.patch
>
>
> There currently isn't a solution in Solr for doing geospatial filtering on documents that have a variable number of points.  This scenario occurs when there is location extraction (i.e. via a "gazateer") occurring on free text.  None, one, or many geospatial locations might be extracted from any given document and users want to limit their search results to those occurring in a user-specified area.
> I've implemented this by furthering the GeoHash based work in Lucene/Solr with a geohash prefix based filter.  A geohash refers to a lat-lon box on the earth.  Each successive character added further subdivides the box into a 4x8 (or 8x4 depending on the even/odd length of the geohash) grid.  The first step in this scheme is figuring out which geohash grid squares cover the user's search query.  I've added various extra methods to GeoHashUtils (and added tests) to assist in this purpose.  The next step is an actual Lucene Filter, GeoHashPrefixFilter, that uses these geohash prefixes in TermsEnum.seek() to skip to relevant grid squares in the index.  Once a matching geohash grid is found, the points therein are compared against the user's query to see if it matches.  I created an abstraction GeoShape extended by subclasses named PointDistance... and CartesianBox.... to support different queried shapes so that the filter need not care about these details.
> This work was presented at LuceneRevolution in Boston on October 8th.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [jira] [Commented] (SOLR-2155) Geospatial search using geohash prefixes

Billnbell
Harley,

I don't really think this is the same thing. Although I think it is
poor form to name it the same, these are 2 different caches. I am
still not sure what increasing the custom cache does. I looked at the
code and it is not clear. I did confirm that this seems t be happening
NOT on commit, but throughout the day. Maybe on Garbage collection?

On Tue, Mar 27, 2012 at 11:42 AM, Harley Parks (Commented) (JIRA)
<[hidden email]> wrote:

>
>    [ https://issues.apache.org/jira/browse/SOLR-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13239711#comment-13239711 ]
>
> Harley Parks commented on SOLR-2155:
> ------------------------------------
>
> interesting... conversation regarding both tuning and data requirements.
> I would guess that the size needs to be in power of 2's
>
> since the example given is related to a custom type, it should be noted that a fieldValueCache is by default created, for each document id.
>
> the example given by default in solr 3.4 if needed:
>       <fieldValueCache class="solr.FastLRUCache"
>                        size="512"
>                        autowarmCount="128"
>                        showItems="32" />
>
> Additionally, the custom cache uses the same name, which might not matter, or does it? the custom cache may override the fieldValueCache created by default.
>
>> Geospatial search using geohash prefixes
>> ----------------------------------------
>>
>>                 Key: SOLR-2155
>>                 URL: https://issues.apache.org/jira/browse/SOLR-2155
>>             Project: Solr
>>          Issue Type: Improvement
>>            Reporter: David Smiley
>>         Attachments: GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch, GeoHashPrefixFilter.patch, SOLR-2155_GeoHashPrefixFilter_with_sorting_no_poly.patch, SOLR.2155.p3.patch, SOLR.2155.p3tests.patch, Solr2155-1.0.2-project.zip, Solr2155-1.0.3-project.zip, Solr2155-1.0.4-project.zip, Solr2155-for-1.0.2-3.x-port.patch
>>
>>
>> There currently isn't a solution in Solr for doing geospatial filtering on documents that have a variable number of points.  This scenario occurs when there is location extraction (i.e. via a "gazateer") occurring on free text.  None, one, or many geospatial locations might be extracted from any given document and users want to limit their search results to those occurring in a user-specified area.
>> I've implemented this by furthering the GeoHash based work in Lucene/Solr with a geohash prefix based filter.  A geohash refers to a lat-lon box on the earth.  Each successive character added further subdivides the box into a 4x8 (or 8x4 depending on the even/odd length of the geohash) grid.  The first step in this scheme is figuring out which geohash grid squares cover the user's search query.  I've added various extra methods to GeoHashUtils (and added tests) to assist in this purpose.  The next step is an actual Lucene Filter, GeoHashPrefixFilter, that uses these geohash prefixes in TermsEnum.seek() to skip to relevant grid squares in the index.  Once a matching geohash grid is found, the points therein are compared against the user's query to see if it matches.  I created an abstraction GeoShape extended by subclasses named PointDistance... and CartesianBox.... to support different queried shapes so that the filter need not care about these details.
>> This work was presented at LuceneRevolution in Boston on October 8th.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>



--
Bill Bell
[hidden email]
cell 720-256-8076

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]