Geographical distance searching

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Geographical distance searching

Lance Norskog-2
It is a "best practice" to store the master copy of this data in a
relational database and use Solr/Lucene as a high-speed cache.
MySQL has a geographical database option, so maybe that is a better option
than Lucene indexing.

Lance

(P.s. please start new threads for new topics.)

-----Original Message-----
From: Sandeep Shetty [mailto:[hidden email]]
Sent: Wednesday, September 26, 2007 5:15 AM
To: '[hidden email]'
Subject: custom sorting

> Hi Guys,
>
> this question as been asked before but i was unable to find an answer
> thats good for me, so hope you guys can help again i am working on a
> website where we need to sort the results by distance from the
> location entered by the user. I have indexed the lat and long info for
> each record in solr and also i can get the lat and long of the
> location input by the user.
> Previously we were using lucene to do this. by using the
> SortComparatorSource we could sort the documents returned by distance
> nicely. we are now switching over to lucene because of the features it
> provides, however i am not able to see a way to do this in Solr.
>
> If someone can point me in the right direction i would be very grateful!
>
> Thanks in advance,
> Sandeep

This email is confidential and may also be privileged. If you are not the
intended recipient please notify us immediately by telephoning +44 (0)20
7452 5300 or email [hidden email]. You should not copy it or use
it for any purpose nor disclose its contents to any other person. Touch
Local cannot accept liability for statements made which are clearly the
sender's own and are not made on behalf of the firm.

Touch Local Limited
Registered Number: 2885607
VAT Number: GB896112114
Cardinal Tower, 12 Farringdon Road, London EC1M 3NN
+44 (0)20 7452 5300

Reply | Threaded
Open this post in threaded view
|

RE: Geographical distance searching

Will Johnson-2
With the new/improved value source functions it should be pretty easy to
develop a new best practice.  You should be able to pull in the lat/lon
values from valuesource fields and then do your greater circle calculation.

- will

-----Original Message-----
From: Lance Norskog [mailto:[hidden email]]
Sent: Wednesday, September 26, 2007 3:15 PM
To: [hidden email]
Subject: Geographical distance searching

It is a "best practice" to store the master copy of this data in a
relational database and use Solr/Lucene as a high-speed cache.
MySQL has a geographical database option, so maybe that is a better option
than Lucene indexing.

Lance

(P.s. please start new threads for new topics.)

-----Original Message-----
From: Sandeep Shetty [mailto:[hidden email]]
Sent: Wednesday, September 26, 2007 5:15 AM
To: '[hidden email]'
Subject: custom sorting

> Hi Guys,
>
> this question as been asked before but i was unable to find an answer
> thats good for me, so hope you guys can help again i am working on a
> website where we need to sort the results by distance from the
> location entered by the user. I have indexed the lat and long info for
> each record in solr and also i can get the lat and long of the
> location input by the user.
> Previously we were using lucene to do this. by using the
> SortComparatorSource we could sort the documents returned by distance
> nicely. we are now switching over to lucene because of the features it
> provides, however i am not able to see a way to do this in Solr.
>
> If someone can point me in the right direction i would be very grateful!
>
> Thanks in advance,
> Sandeep

This email is confidential and may also be privileged. If you are not the
intended recipient please notify us immediately by telephoning +44 (0)20
7452 5300 or email [hidden email]. You should not copy it or use
it for any purpose nor disclose its contents to any other person. Touch
Local cannot accept liability for statements made which are clearly the
sender's own and are not made on behalf of the firm.

Touch Local Limited
Registered Number: 2885607
VAT Number: GB896112114
Cardinal Tower, 12 Farringdon Road, London EC1M 3NN
+44 (0)20 7452 5300

Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Ian Holsman (Lists)
Have you guys seen Local Lucene ?
http://www.nsshutdown.com/projects/lucene/whitepaper/*locallucene*.htm

no need for mysql if you don't want too.

rgrds
Ian

Will Johnson wrote:

> With the new/improved value source functions it should be pretty easy to
> develop a new best practice.  You should be able to pull in the lat/lon
> values from valuesource fields and then do your greater circle calculation.
>
> - will
>
> -----Original Message-----
> From: Lance Norskog [mailto:[hidden email]]
> Sent: Wednesday, September 26, 2007 3:15 PM
> To: [hidden email]
> Subject: Geographical distance searching
>
> It is a "best practice" to store the master copy of this data in a
> relational database and use Solr/Lucene as a high-speed cache.
> MySQL has a geographical database option, so maybe that is a better option
> than Lucene indexing.
>
> Lance
>
> (P.s. please start new threads for new topics.)
>
> -----Original Message-----
> From: Sandeep Shetty [mailto:[hidden email]]
> Sent: Wednesday, September 26, 2007 5:15 AM
> To: '[hidden email]'
> Subject: custom sorting
>
>  
>> Hi Guys,
>>
>> this question as been asked before but i was unable to find an answer
>> thats good for me, so hope you guys can help again i am working on a
>> website where we need to sort the results by distance from the
>> location entered by the user. I have indexed the lat and long info for
>> each record in solr and also i can get the lat and long of the
>> location input by the user.
>> Previously we were using lucene to do this. by using the
>> SortComparatorSource we could sort the documents returned by distance
>> nicely. we are now switching over to lucene because of the features it
>> provides, however i am not able to see a way to do this in Solr.
>>
>> If someone can point me in the right direction i would be very grateful!
>>
>> Thanks in advance,
>> Sandeep
>>    
>
> This email is confidential and may also be privileged. If you are not the
> intended recipient please notify us immediately by telephoning +44 (0)20
> 7452 5300 or email [hidden email]. You should not copy it or use
> it for any purpose nor disclose its contents to any other person. Touch
> Local cannot accept liability for statements made which are clearly the
> sender's own and are not made on behalf of the firm.
>
> Touch Local Limited
> Registered Number: 2885607
> VAT Number: GB896112114
> Cardinal Tower, 12 Farringdon Road, London EC1M 3NN
> +44 (0)20 7452 5300
>
>
>  

Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Patrick O'Leary-2
Might want to remove the *'s around that url
http://www.nsshutdown.com/projects/lucene/whitepaper/locallucene.htm

There's actually a download-able demo http://www.nsshutdown.com/solr-example_s1.3_ls0.2.tgz
start it up as you would a normal solr example
$ cd solr-example/apache-solr*/example
$ java -jar start.jar

Open up firefox (sorry demo ui was quick and dirty so firefox only) and go to http://localhost:8983/localcinema/
Make sure you specify localhost, there's a google maps key based upon the url's domain, and click 'Go' at the bottom of the page.

The demo comes with some sample data already indexed for the NY region, so have a play.

p.s after a little tidy up I'll be adding this to both lucene and solr's repositories if folks feel that it's a useful addition.

Thanks
Patrick

Ian Holsman wrote:
Have you guys seen Local Lucene ?
http://www.nsshutdown.com/projects/lucene/whitepaper/*locallucene*.htm

no need for mysql if you don't want too.

rgrds
Ian

Will Johnson wrote:
With the new/improved value source functions it should be pretty easy to
develop a new best practice.  You should be able to pull in the lat/lon
values from valuesource fields and then do your greater circle calculation.

- will

-----Original Message-----
From: Lance Norskog [[hidden email]] Sent: Wednesday, September 26, 2007 3:15 PM
To: [hidden email]
Subject: Geographical distance searching

It is a "best practice" to store the master copy of this data in a
relational database and use Solr/Lucene as a high-speed cache.
MySQL has a geographical database option, so maybe that is a better option
than Lucene indexing.

Lance

(P.s. please start new threads for new topics.)

-----Original Message-----
From: Sandeep Shetty [[hidden email]] Sent: Wednesday, September 26, 2007 5:15 AM
To: '[hidden email]'
Subject: custom sorting

 
Hi Guys,

this question as been asked before but i was unable to find an answer thats good for me, so hope you guys can help again i am working on a website where we need to sort the results by distance from the location entered by the user. I have indexed the lat and long info for each record in solr and also i can get the lat and long of the location input by the user.
Previously we were using lucene to do this. by using the SortComparatorSource we could sort the documents returned by distance nicely. we are now switching over to lucene because of the features it provides, however i am not able to see a way to do this in Solr.

If someone can point me in the right direction i would be very grateful!

Thanks in advance,
Sandeep
   

This email is confidential and may also be privileged. If you are not the
intended recipient please notify us immediately by telephoning +44 (0)20
7452 5300 or email [hidden email]. You should not copy it or use
it for any purpose nor disclose its contents to any other person. Touch
Local cannot accept liability for statements made which are clearly the
sender's own and are not made on behalf of the firm.

Touch Local Limited
Registered Number: 2885607
VAT Number: GB896112114
Cardinal Tower, 12 Farringdon Road, London EC1M 3NN
+44 (0)20 7452 5300


 



--
Patrick O'Leary


You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles.
 Do you understand this? 
And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.
  - Albert Einstein
View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Guillaume Smet
Hi Patrick,

On 9/27/07, patrick o'leary <[hidden email]> wrote:
>  p.s after a little tidy up I'll be adding this to both lucene and solr's repositories if folks feel that it's a useful addition.

It's definitely very interesting. Did you compare performances of
Lucene with a database allowing you to perform real GIS queries?
I'm more a PostgreSQL guy and I must admit we usually use cube contrib
or PostGIS for this sort of thing and with both, we are capable to use
indexes for proximity queries and they can be pretty fast. Using the
method you used with MySQL is definitely too slow and not used as soon
as you have a certain amount of data in your table.

Regards,

--
Guillaume
Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Patrick O'Leary-2
As far as I'm concerned nothings going to beat PG's GIS calculations, but it's tsearch was
a lot slower than myisam.

My goal was a single solution to reduce our complexity, but am interested to know if combining
both an rdbms & lucene works for you. Definitely let me know how it goes !

P

Guillaume Smet wrote:
Hi Patrick,

On 9/27/07, patrick o'leary [hidden email] wrote:
  
 p.s after a little tidy up I'll be adding this to both lucene and solr's repositories if folks feel that it's a useful addition.
    

It's definitely very interesting. Did you compare performances of
Lucene with a database allowing you to perform real GIS queries?
I'm more a PostgreSQL guy and I must admit we usually use cube contrib
or PostGIS for this sort of thing and with both, we are capable to use
indexes for proximity queries and they can be pretty fast. Using the
method you used with MySQL is definitely too slow and not used as soon
as you have a certain amount of data in your table.

Regards,

  

--
Patrick O'Leary


You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles.
 Do you understand this? 
And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.
  - Albert Einstein
View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Doug Daniels
Hi Patrick,

Are the solr components of that demo in the repository as well?  I couldn't find them there.

Best,
Doug

patrick o'leary wrote
As far as I'm concerned nothings going to beat PG's GIS calculations,
but it's tsearch was
a lot slower than myisam.

My goal was a single solution to reduce our complexity, but am
interested to know if combining
both an rdbms & lucene works for you. Definitely let me know how it goes !

P

Guillaume Smet wrote:
> Hi Patrick,
>
> On 9/27/07, patrick o'leary <polearyuk@aol.com> wrote:
>  
>>  p.s after a little tidy up I'll be adding this to both lucene and solr's repositories if folks feel that it's a useful addition.
>>    
>
> It's definitely very interesting. Did you compare performances of
> Lucene with a database allowing you to perform real GIS queries?
> I'm more a PostgreSQL guy and I must admit we usually use cube contrib
> or PostGIS for this sort of thing and with both, we are capable to use
> indexes for proximity queries and they can be pretty fast. Using the
> method you used with MySQL is definitely too slow and not used as soon
> as you have a certain amount of data in your table.
>
> Regards,
>
>  

--

Patrick O'Leary


You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles.
 Do you understand this?
And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.
  - Albert Einstein

View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
<http://www.linkedin.com/in/pjaol>
Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Patrick O'Leary-2
Hi Doug

What exactly are you looking for?
The code for localsolr is still in dev state, but I've left my work open and available for download
at http://www.nsshutdown.com/viewcvs/viewcvs.cgi/localsolr/

Once I'm happy with it, I'll donate it back in the form of patches until / unless it's accepted
as a contribution, depending on how folks feel.

If your talking about the demo ui, it's a little piece of html & JS, you can pull directly from the jar.
I've not included that in the repository.

HTH
P

Doug Daniels wrote:
Hi Patrick,

Are the solr components of that demo in the repository as well?  I couldn't
find them there.

Best,
Doug


patrick o'leary wrote:
  
As far as I'm concerned nothings going to beat PG's GIS calculations,
but it's tsearch was
a lot slower than myisam.

My goal was a single solution to reduce our complexity, but am
interested to know if combining
both an rdbms & lucene works for you. Definitely let me know how it goes !

P

Guillaume Smet wrote:
    
Hi Patrick,

On 9/27/07, patrick o'leary [hidden email] wrote:
  
      
 p.s after a little tidy up I'll be adding this to both lucene and
solr's repositories if folks feel that it's a useful addition.
    
        
It's definitely very interesting. Did you compare performances of
Lucene with a database allowing you to perform real GIS queries?
I'm more a PostgreSQL guy and I must admit we usually use cube contrib
or PostGIS for this sort of thing and with both, we are capable to use
indexes for proximity queries and they can be pretty fast. Using the
method you used with MySQL is definitely too slow and not used as soon
as you have a certain amount of data in your table.

Regards,

  
      
-- 

Patrick O'Leary


You see, wire telegraph is a kind of a very, very long cat. You pull his
tail in New York and his head is meowing in Los Angeles.
 Do you understand this? 
And radio operates exactly the same way: you send signals here, they
receive them there. The only difference is that there is no cat.
  - Albert Einstein

View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
<http://www.linkedin.com/in/pjaol>


    

  

--
Patrick O'Leary


You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles.
 Do you understand this? 
And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.
  - Albert Einstein
View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Doug Daniels
Hi Patrick,

Was mainly interested in seeing how you did the RequestHandler.  Thanks for sending the link!

Best,
Doug

patrick o'leary wrote
Hi Doug

What exactly are you looking for?
The code for localsolr is still in dev state, but I've left my work open
and available for download
at http://www.nsshutdown.com/viewcvs/viewcvs.cgi/localsolr/

Once I'm happy with it, I'll donate it back in the form of patches until
/ unless it's accepted
as a contribution, depending on how folks feel.

If your talking about the demo ui, it's a little piece of html & JS, you
can pull directly from the jar.
I've not included that in the repository.

HTH
P

Doug Daniels wrote:
> Hi Patrick,
>
> Are the solr components of that demo in the repository as well?  I couldn't
> find them there.
>
> Best,
> Doug
>
>
> patrick o'leary wrote:
>  
>> As far as I'm concerned nothings going to beat PG's GIS calculations,
>> but it's tsearch was
>> a lot slower than myisam.
>>
>> My goal was a single solution to reduce our complexity, but am
>> interested to know if combining
>> both an rdbms & lucene works for you. Definitely let me know how it goes !
>>
>> P
>>
>> Guillaume Smet wrote:
>>    
>>> Hi Patrick,
>>>
>>> On 9/27/07, patrick o'leary <polearyuk@aol.com> wrote:
>>>  
>>>      
>>>>  p.s after a little tidy up I'll be adding this to both lucene and
>>>> solr's repositories if folks feel that it's a useful addition.
>>>>    
>>>>        
>>> It's definitely very interesting. Did you compare performances of
>>> Lucene with a database allowing you to perform real GIS queries?
>>> I'm more a PostgreSQL guy and I must admit we usually use cube contrib
>>> or PostGIS for this sort of thing and with both, we are capable to use
>>> indexes for proximity queries and they can be pretty fast. Using the
>>> method you used with MySQL is definitely too slow and not used as soon
>>> as you have a certain amount of data in your table.
>>>
>>> Regards,
>>>
>>>  
>>>      
>> --
>>
>> Patrick O'Leary
>>
>>
>> You see, wire telegraph is a kind of a very, very long cat. You pull his
>> tail in New York and his head is meowing in Los Angeles.
>>  Do you understand this?
>> And radio operates exactly the same way: you send signals here, they
>> receive them there. The only difference is that there is no cat.
>>   - Albert Einstein
>>
>> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
>> <http://www.linkedin.com/in/pjaol>
>>
>>
>>    
>
>  

--

Patrick O'Leary


You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles.
 Do you understand this?
And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.
  - Albert Einstein

View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
<http://www.linkedin.com/in/pjaol>
Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Ryan McKinley
This looks good!

Are you interested in contributing it to solr core?

One major thing in the solr pipeline you may want to be aware of is the
search component interface (SOLR-281).

This would let you make simple component that adds the:
   DistanceQuery dq = new DistanceQuery(dlat,dlng,dradius);
   dsort = new DistanceSortSource(filter);

and later adds the 'distance' to each result

This way you could reuse the other standard search stuff (faceting,
debugging, etc) and would not need to make your own custom
LocalResponseWritter.

ryan


Doug Daniels wrote:

> Hi Patrick,
>
> Was mainly interested in seeing how you did the RequestHandler.  Thanks for
> sending the link!
>
> Best,
> Doug
>
>
> patrick o'leary wrote:
>> Hi Doug
>>
>> What exactly are you looking for?
>> The code for localsolr is still in dev state, but I've left my work open
>> and available for download
>> at http://www.nsshutdown.com/viewcvs/viewcvs.cgi/localsolr/
>>
>> Once I'm happy with it, I'll donate it back in the form of patches until
>> / unless it's accepted
>> as a contribution, depending on how folks feel.
>>
>> If your talking about the demo ui, it's a little piece of html & JS, you
>> can pull directly from the jar.
>> I've not included that in the repository.
>>
>> HTH
>> P
>>
>> Doug Daniels wrote:
>>> Hi Patrick,
>>>
>>> Are the solr components of that demo in the repository as well?  I
>>> couldn't
>>> find them there.
>>>
>>> Best,
>>> Doug
>>>
>>>
>>> patrick o'leary wrote:
>>>  
>>>> As far as I'm concerned nothings going to beat PG's GIS calculations,
>>>> but it's tsearch was
>>>> a lot slower than myisam.
>>>>
>>>> My goal was a single solution to reduce our complexity, but am
>>>> interested to know if combining
>>>> both an rdbms & lucene works for you. Definitely let me know how it goes
>>>> !
>>>>
>>>> P
>>>>
>>>> Guillaume Smet wrote:
>>>>    
>>>>> Hi Patrick,
>>>>>
>>>>> On 9/27/07, patrick o'leary <[hidden email]> wrote:
>>>>>  
>>>>>      
>>>>>>  p.s after a little tidy up I'll be adding this to both lucene and
>>>>>> solr's repositories if folks feel that it's a useful addition.
>>>>>>    
>>>>>>        
>>>>> It's definitely very interesting. Did you compare performances of
>>>>> Lucene with a database allowing you to perform real GIS queries?
>>>>> I'm more a PostgreSQL guy and I must admit we usually use cube contrib
>>>>> or PostGIS for this sort of thing and with both, we are capable to use
>>>>> indexes for proximity queries and they can be pretty fast. Using the
>>>>> method you used with MySQL is definitely too slow and not used as soon
>>>>> as you have a certain amount of data in your table.
>>>>>
>>>>> Regards,
>>>>>
>>>>>  
>>>>>      
>>>> --
>>>>
>>>> Patrick O'Leary
>>>>
>>>>
>>>> You see, wire telegraph is a kind of a very, very long cat. You pull his
>>>> tail in New York and his head is meowing in Los Angeles.
>>>>  Do you understand this?
>>>> And radio operates exactly the same way: you send signals here, they
>>>> receive them there. The only difference is that there is no cat.
>>>>   - Albert Einstein
>>>>
>>>> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
>>>> <http://www.linkedin.com/in/pjaol>
>>>>
>>>>
>>>>    
>>>  
>> --
>>
>> Patrick O'Leary
>>
>>
>> You see, wire telegraph is a kind of a very, very long cat. You pull his
>> tail in New York and his head is meowing in Los Angeles.
>>  Do you understand this?
>> And radio operates exactly the same way: you send signals here, they
>> receive them there. The only difference is that there is no cat.
>>   - Albert Einstein
>>
>> View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
>> <http://www.linkedin.com/in/pjaol>
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Patrick O'Leary-2
Hi Ryan

Thanks for looking at it, yes definitely I'd like to contribute back.

I'm currently working out some of the logistics to make it easier to maintain.
I think I'd need a centralized project to do the releases from, it will have 2 sets
of patches in 2 different trunks (lucene & solr), until it becomes core in lucene.
So a batteries included release will make that easier for users, and developers alike.
Once it's accepted I'll move development to ASF.

SOLR-281 looks like it will solve one of my frustrations, another being that the handlers were final ;-)
Is it close to being committed to the trunk?

Thanks
P

Ryan McKinley wrote:
This looks good!

Are you interested in contributing it to solr core?

One major thing in the solr pipeline you may want to be aware of is the search component interface (SOLR-281).

This would let you make simple component that adds the:
  DistanceQuery dq = new DistanceQuery(dlat,dlng,dradius);
  dsort = new DistanceSortSource(filter);

and later adds the 'distance' to each result

This way you could reuse the other standard search stuff (faceting, debugging, etc) and would not need to make your own custom LocalResponseWritter.

ryan


Doug Daniels wrote:
Hi Patrick,

Was mainly interested in seeing how you did the RequestHandler.  Thanks for
sending the link!

Best,
Doug


patrick o'leary wrote:
Hi Doug

What exactly are you looking for?
The code for localsolr is still in dev state, but I've left my work open
and available for download
at http://www.nsshutdown.com/viewcvs/viewcvs.cgi/localsolr/

Once I'm happy with it, I'll donate it back in the form of patches until
/ unless it's accepted
as a contribution, depending on how folks feel.

If your talking about the demo ui, it's a little piece of html & JS, you
can pull directly from the jar.
I've not included that in the repository.

HTH
P

Doug Daniels wrote:
Hi Patrick,

Are the solr components of that demo in the repository as well?  I
couldn't
find them there.

Best,
Doug


patrick o'leary wrote:
 
As far as I'm concerned nothings going to beat PG's GIS calculations,
but it's tsearch was
a lot slower than myisam.

My goal was a single solution to reduce our complexity, but am
interested to know if combining
both an rdbms & lucene works for you. Definitely let me know how it goes
!

P

Guillaume Smet wrote:
   
Hi Patrick,

On 9/27/07, patrick o'leary [hidden email] wrote:
       
 p.s after a little tidy up I'll be adding this to both lucene and
solr's repositories if folks feel that it's a useful addition.
           
It's definitely very interesting. Did you compare performances of
Lucene with a database allowing you to perform real GIS queries?
I'm more a PostgreSQL guy and I must admit we usually use cube contrib
or PostGIS for this sort of thing and with both, we are capable to use
indexes for proximity queries and they can be pretty fast. Using the
method you used with MySQL is definitely too slow and not used as soon
as you have a certain amount of data in your table.

Regards,

       
-- 

Patrick O'Leary


You see, wire telegraph is a kind of a very, very long cat. You pull his
tail in New York and his head is meowing in Los Angeles.
 Do you understand this? And radio operates exactly the same way: you send signals here, they
receive them there. The only difference is that there is no cat.
  - Albert Einstein

View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
<http://www.linkedin.com/in/pjaol>


   
 
-- 

Patrick O'Leary


You see, wire telegraph is a kind of a very, very long cat. You pull his
tail in New York and his head is meowing in Los Angeles.
 Do you understand this? And radio operates exactly the same way: you send signals here, they
receive them there. The only difference is that there is no cat.
  - Albert Einstein

View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
<http://www.linkedin.com/in/pjaol>






--
Patrick O'Leary


You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles.
 Do you understand this? 
And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.
  - Albert Einstein
View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Ryan McKinley
>
> SOLR-281 looks like it will solve one of my frustrations, another being
> that the handlers were final ;-)

What handlers are final that you found annoying?

> Is it close to being committed to the trunk?
>

I hope so ;)  Since this patch reworks the *core* query handlers
(dismax/standard) I really want someone else to look it over before
committing...

ryan

Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Patrick O'Leary-2
Actually misspoke it's the XMLWritter that's final that was a little annoying rather than a handler.
Would have been nice to just extend that, and cut down on the code.

P

Ryan McKinley wrote:

SOLR-281 looks like it will solve one of my frustrations, another being that the handlers were final ;-)

What handlers are final that you found annoying?

Is it close to being committed to the trunk?


I hope so ;)  Since this patch reworks the *core* query handlers (dismax/standard) I really want someone else to look it over before committing...

ryan



--
Patrick O'Leary

AOL Local Search Technologies
Phone: + 1 703 265 8763

You see, wire telegraph is a kind of a very, very long cat. You pull his tail in New York and his head is meowing in Los Angeles.
 Do you understand this? 
And radio operates exactly the same way: you send signals here, they receive them there. The only difference is that there is no cat.
  - Albert Einstein
View Patrick O Leary's LinkedIn profileView Patrick O Leary's profile
Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

Ryan McKinley
patrick o'leary wrote:
> Actually misspoke it's the XMLWritter that's final that was a little
> annoying rather than a handler.
> Would have been nice to just extend that, and cut down on the code.
>

aaah -- Just to be clear, if you could augment the doc list with a
calculated field ('distance') you would not need to extend XMLWritter -
correct?

And then it would also work with the JSON/pyton/php etc writers.

ryan
Reply | Threaded
Open this post in threaded view
|

Re: Geographical distance searching

hossman

: aaah -- Just to be clear, if you could augment the doc list with a calculated
: field ('distance') you would not need to extend XMLWritter - correct?
:
: And then it would also work with the JSON/pyton/php etc writers.

i don't think i ever looked at the patch in question (is it in Jira?) but
i would definitely recommend going a route like this rather then adding a
new "primitive" type that all the ResponseWriters would need to know
about.  

Personally: i don't even think "augmenting DocList" should be done ...
once the distance score calculations are done, and the documents are
ordered, returning the distance as numeric value for the clients is really
no different then returning highlighting info or score explanations ... it
should be a seperate piece of info in the response.

(i think if we had it to do all over again, i would suggest the same
approach for returning the "scores" of docs ... at the moment there is
ambiguity between the relevancy score and the possibility of a field named
"score")




-Hoss