Is there anything like MultiSearcher?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Is there anything like MultiSearcher?

Roman Chyla
Dear Solr experts,

Could you recommend some strategies or perhaps tell me if I approach
my problem from a wrong side? I was hoping to use MultiSearcher to
search across multiple indexes in Solr, but there is no such a thing
and MultiSearcher was removed according to this post:
http://osdir.com/ml/solr-user.lucene.apache.org/2011-01/msg00250.html

I though I had two use cases:

1. maintenance - I wanted to build two separate indexes, one for
fulltext and one for metadata (the docs have the unique ids) -
indexing them separately would make things much simpler
2. ability to switch indexes at search time (ie. for testing purposes
- one fulltext index could be built by Solr standard mechanism, the
other by a rather different process - independent instance of lucene)

I think the recommended approach is to use the Distributed search - I
found a nice solution here:
http://stackoverflow.com/questions/2139030/search-multiple-solr-cores-and-return-one-result-set
- however it seems to me, that data are sent over HTTP (5M from one
core, and 5M from the other core being merged by the 3rd solr core?)
and I would like to do it only for local indexes and without the
network overhead.

Could you please shed some light if there already exist an optimal
solution to my use cases? And if not, whether I could just try to
build a new SolrQuerySearcher that is extending lucene MultiSearcher
instead of IndexSearch - or you think there are some deeply rooted
problems there and the MultiSearch-er cannot work inside Solr?

Thank you,

  Roman
Reply | Threaded
Open this post in threaded view
|

Re: Is there anything like MultiSearcher?

Billnbell
Why not just use sharding across the 2 cores?

On 2/5/11 8:49 AM, "Roman Chyla" <[hidden email]> wrote:

>Dear Solr experts,
>
>Could you recommend some strategies or perhaps tell me if I approach
>my problem from a wrong side? I was hoping to use MultiSearcher to
>search across multiple indexes in Solr, but there is no such a thing
>and MultiSearcher was removed according to this post:
>http://osdir.com/ml/solr-user.lucene.apache.org/2011-01/msg00250.html
>
>I though I had two use cases:
>
>1. maintenance - I wanted to build two separate indexes, one for
>fulltext and one for metadata (the docs have the unique ids) -
>indexing them separately would make things much simpler
>2. ability to switch indexes at search time (ie. for testing purposes
>- one fulltext index could be built by Solr standard mechanism, the
>other by a rather different process - independent instance of lucene)
>
>I think the recommended approach is to use the Distributed search - I
>found a nice solution here:
>http://stackoverflow.com/questions/2139030/search-multiple-solr-cores-and-
>return-one-result-set
>- however it seems to me, that data are sent over HTTP (5M from one
>core, and 5M from the other core being merged by the 3rd solr core?)
>and I would like to do it only for local indexes and without the
>network overhead.
>
>Could you please shed some light if there already exist an optimal
>solution to my use cases? And if not, whether I could just try to
>build a new SolrQuerySearcher that is extending lucene MultiSearcher
>instead of IndexSearch - or you think there are some deeply rooted
>problems there and the MultiSearch-er cannot work inside Solr?
>
>Thank you,
>
>  Roman


Reply | Threaded
Open this post in threaded view
|

Re: Is there anything like MultiSearcher?

Roman Chyla
Unless I am wrong, sharding across two cores is done over HTTP and has
the limitations as listed at:
http://wiki.apache.org/solr/DistributedSearch
While MultiSearcher is just a decorator over IndexSearcher - therefore
the limitations there would (?) not apply and if indexes reside
locally, would be also faster

Cheers,

roman

On Sat, Feb 5, 2011 at 10:02 PM, Bill Bell <[hidden email]> wrote:

> Why not just use sharding across the 2 cores?
>
> On 2/5/11 8:49 AM, "Roman Chyla" <[hidden email]> wrote:
>
>>Dear Solr experts,
>>
>>Could you recommend some strategies or perhaps tell me if I approach
>>my problem from a wrong side? I was hoping to use MultiSearcher to
>>search across multiple indexes in Solr, but there is no such a thing
>>and MultiSearcher was removed according to this post:
>>http://osdir.com/ml/solr-user.lucene.apache.org/2011-01/msg00250.html
>>
>>I though I had two use cases:
>>
>>1. maintenance - I wanted to build two separate indexes, one for
>>fulltext and one for metadata (the docs have the unique ids) -
>>indexing them separately would make things much simpler
>>2. ability to switch indexes at search time (ie. for testing purposes
>>- one fulltext index could be built by Solr standard mechanism, the
>>other by a rather different process - independent instance of lucene)
>>
>>I think the recommended approach is to use the Distributed search - I
>>found a nice solution here:
>>http://stackoverflow.com/questions/2139030/search-multiple-solr-cores-and-
>>return-one-result-set
>>- however it seems to me, that data are sent over HTTP (5M from one
>>core, and 5M from the other core being merged by the 3rd solr core?)
>>and I would like to do it only for local indexes and without the
>>network overhead.
>>
>>Could you please shed some light if there already exist an optimal
>>solution to my use cases? And if not, whether I could just try to
>>build a new SolrQuerySearcher that is extending lucene MultiSearcher
>>instead of IndexSearch - or you think there are some deeply rooted
>>problems there and the MultiSearch-er cannot work inside Solr?
>>
>>Thank you,
>>
>>  Roman
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Is there anything like MultiSearcher?

tmenzel
This post has NOT been accepted by the mailing list yet.
roman,

have u solved the problem? i'm facing a similar problem. our customer wants to have separate indexes/core but still want to search accross them at times w/o the IDF limitation that solr has when using shards
Reply | Threaded
Open this post in threaded view
|

Re: Is there anything like MultiSearcher?

tmenzel
In reply to this post by Roman Chyla
roman,

have u solved the problem? i'm facing a similar problem. our customer wants to have separate indexes/core but still want to search across them at times w/o the IDF limitation that solr has when using shards

> Unless I am wrong, sharding across two cores is done over HTTP and has
> the limitations as listed at:
> http://wiki.apache.org/solr/DistributedSearch
> While MultiSearcher is just a decorator over IndexSearcher - therefore
> the limitations there would (?) not apply and if indexes reside
> locally, would be also faster
>
> Cheers,
>
> roman
>
> On Sat, Feb 5, 2011 at 10:02 PM, Bill Bell <[hidden email]> wrote:
>
> > Why not just use sharding across the 2 cores?
> >
--
NEU: FreePhone - kostenlos mobil telefonieren!
Jetzt informieren: http://www.gmx.net/de/go/freephone
Reply | Threaded
Open this post in threaded view
|

RE: Is there anything like MultiSearcher?

Kai Gülzau
In reply to this post by Roman Chyla
Hi Roman,

do you have solved your problem and how?

Regards,

Kai Gülzau

 

> -----Original Message-----
> From: Roman Chyla [mailto:[hidden email]]
> Sent: Saturday, February 05, 2011 4:50 PM
> To: [hidden email]
> Subject: Is there anything like MultiSearcher?
>
> Dear Solr experts,
>
> Could you recommend some strategies or perhaps tell me if I approach
> my problem from a wrong side? I was hoping to use MultiSearcher to
> search across multiple indexes in Solr, but there is no such a thing
> and MultiSearcher was removed according to this post:
> http://osdir.com/ml/solr-user.lucene.apache.org/2011-01/msg00250.html
>
> I though I had two use cases:
>
> 1. maintenance - I wanted to build two separate indexes, one for
> fulltext and one for metadata (the docs have the unique ids) -
> indexing them separately would make things much simpler
> 2. ability to switch indexes at search time (ie. for testing purposes
> - one fulltext index could be built by Solr standard mechanism, the
> other by a rather different process - independent instance of lucene)
>
> I think the recommended approach is to use the Distributed search - I
> found a nice solution here:
> http://stackoverflow.com/questions/2139030/search-multiple-sol
r-cores-and-return-one-result-set

> - however it seems to me, that data are sent over HTTP (5M from one
> core, and 5M from the other core being merged by the 3rd solr core?)
> and I would like to do it only for local indexes and without the
> network overhead.
>
> Could you please shed some light if there already exist an optimal
> solution to my use cases? And if not, whether I could just try to
> build a new SolrQuerySearcher that is extending lucene MultiSearcher
> instead of IndexSearch - or you think there are some deeply rooted
> problems there and the MultiSearch-er cannot work inside Solr?
>
> Thank you,
>
>   Roman
>
>