# Can Solr solve this simple problem?

8 messages
Open this post in threaded view
|

## Can Solr solve this simple problem?

 Hi everyone :) Our company is very interesting in Solr engine for searching people. I have 3 questions below about extended capabilities of Solr, but first I'd like to present you the problem Let's say we have ~100 mln users with many characteristics - some of them described below. We want to search users by any set of these characteristics (of course we should use index clustering, replication, query distribution)    - country - text (alpha-iso-3 country code)    - language - text (alpha-iso-3 country code)    - has_photo - boolean    - has_video - boolean    - lastvisit - date    - gender - int    - age - int    - latitude - float    - longitude - float    - height - int    - updated - date    - 100+ other boolean fields to store and search by it - profile has some    property or don't * Prototype SQL query looks like this:* SELECT user_id FROM users WHERE     AND country = 'USA'     AND language = 'SPA'     AND gender = 1     AND age BETWEEN 30 AND 40     AND latitude BETWEEN 39.0 AND 41.0     AND longitude BETWEEN 73.0 AND 75.0     AND height BETWEEN 170 AND 180     AND has_photo = 1     AND has_video = 0     AND (bool_field1 = 1 OR bool_field2 = 1)     AND (bool_fieldN = 0 OR bool_fieldM = 1 OR bool_fieldK = 0)     ... ORDER BY     IF(has_photo = 1, 100, 0) +     IF(language = 'FRA', 50, 0) +     IF(has_description = 1, 150, 0) +     IF(has_video = 1, 50, 0) +     IF(lastvisit > NOW() - interval 1 month, 200, 0) DESC,     IF(age > 35, 20, 0) +     IF(gender = 2, 30, 0) +     IF(bool_field1 = 1, 50, 0) +     IF(bool_fieldN = 0, 100, 0) ASC LIMIT 200; So, these are my 3 questions: 1. Does Solr provide searching among different count fields with different types like in WHERE condition? 2. Does Solr provide such sorting, that depends on other fields (like sums in ORDER BY), other words - does it provide any kind of function, which is used to sort results from q1? 3. Does Solr provide realtime index updating or updating every N minutes? What advices can you give to provide this scheme searching with Solr? Best regards.
Open this post in threaded view
|

## Re: Can Solr solve this simple problem?

 > Hi everyone :) Hi :) > So, these are my 3 questions: > 1. Does Solr provide searching among different count fields with different > types like in WHERE condition? Yes. As long as these are not full-text you should use filter queries for these, e.g. &q=*:* &fq=country:USA &fq=language:SPA &fq=age:[30 TO 40] &fq=(bool_field1:1 OR bool_field2:1) The reason why I put multiple "fq" instead of one long is to optimize for caching of filters > 2. Does Solr provide such sorting, that depends on other fields (like sums > in ORDER BY), other words - does it provide any kind of function, which is > used to sort results from q1? Yes. In trunk version you can sort by function which can do sums and all crezy things &sort=sum(product(has_photo,10),if(exists(query(\$agequery)),50,0)) asc&agequery=age:[53 TO *] See http://wiki.apache.org/solr/FunctionQuery for more functions But you could also to much of this through boost queries &sort=score desc &bq=language:FRA^50 %bq=age:[53 TO *]^20 > 3. Does Solr provide realtime index updating or updating every N minutes? Sure, there is Near Real-time indexing in TRUNK (coming 4.0) Jan
Open this post in threaded view
|

## Re: Can Solr solve this simple problem?

 I'm wondering if Solr is the best tool for this kind of usage. Solr is a "text search engine", so even if it supports all those features, it is design for text search, which doesn't seem to be what you need. Which are the reasons for moving from a DB implementation to Solr? Don't misunderstand me, I really like Solr and it is a really cool hammer, but not everything is a nail. I would really like to hear more opinions on this. Tomás On Mon, Apr 16, 2012 at 7:12 PM, Jan Høydahl <[hidden email]> wrote: > > Hi everyone :) > > Hi :) > > > So, these are my 3 questions: > > 1. Does Solr provide searching among different count fields with > different > > types like in WHERE condition? > > Yes. As long as these are not full-text you should use filter queries for > these, e.g. > &q=*:* > &fq=country:USA > &fq=language:SPA > &fq=age:[30 TO 40] > &fq=(bool_field1:1 OR bool_field2:1) > > The reason why I put multiple "fq" instead of one long is to optimize for > caching of filters > > > 2. Does Solr provide such sorting, that depends on other fields (like > sums > > in ORDER BY), other words - does it provide any kind of function, which > is > > used to sort results from q1? > > Yes. In trunk version you can sort by function which can do sums and all > crezy things > &sort=sum(product(has_photo,10),if(exists(query(\$agequery)),50,0)) > asc&agequery=age:[53 TO *] > See http://wiki.apache.org/solr/FunctionQuery for more functions > > But you could also to much of this through boost queries > &sort=score desc > &bq=language:FRA^50 > %bq=age:[53 TO *]^20 > > > 3. Does Solr provide realtime index updating or updating every N minutes? > > Sure, there is Near Real-time indexing in TRUNK (coming 4.0) > > Jan
Open this post in threaded view
|

## Re: Can Solr solve this simple problem?

 2012/4/16 Tomás Fernández Löbbe <[hidden email]>: > I'm wondering if Solr is the best tool for this kind of usage. Solr is a > "text search engine" Well, Lucene is a "full-text search library", but Solr has always been far more. Dating back to it's first use in CNET, it was used as a browse engine (faceted search), sometimes without much of a full-text aspect at all. And we're moving more and more into the NoSQL realm (durability, realtime-get, and coming real soon - optimistic locking). -Yonik lucenerevolution.com - Lucene/Solr Open Source Search Conference. Boston May 7-10
Open this post in threaded view
|

## Re: Can Solr solve this simple problem?

Open this post in threaded view
|