Searches across Cores

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Searches across Cores

Komal Motwani
Hi,



I have a use case where I would like a query to span across Cores
(Multi-Core); all the cores involved do have same schema. I have started
using solr just recently and have been trying to find ways to achieve this
but couldn’t find any solution so far (Distributed searches, shards are not
what I am looking for). I remember in one of the tech talks, there was a
mention of this feature to be included in future releases. Appreciate any
pointers to help me progress further.



Thanks,

Komal Motwani
Reply | Threaded
Open this post in threaded view
|

Re: Searches across Cores

sidharth228
Hi,

If the number of cores spanned is low, I guess firing parallel queries and
taking union or intersection should work since their schema is the same. Do
you notice any perceivable difference in performance?

Best,
Sidharth

On Fri, Aug 9, 2019 at 2:54 PM Komal Motwani <[hidden email]>
wrote:

> Hi,
>
>
>
> I have a use case where I would like a query to span across Cores
> (Multi-Core); all the cores involved do have same schema. I have started
> using solr just recently and have been trying to find ways to achieve this
> but couldn’t find any solution so far (Distributed searches, shards are not
> what I am looking for). I remember in one of the tech talks, there was a
> mention of this feature to be included in future releases. Appreciate any
> pointers to help me progress further.
>
>
>
> Thanks,
>
> Komal Motwani
>
Reply | Threaded
Open this post in threaded view
|

Re: Searches across Cores

Jan Høydahl / Cominvent
In reply to this post by Komal Motwani
USE request param &shards=core1,core2 or if on separate machines host:port/solr/core1,host:port/solr/core2

Jan Høydahl

> 9. aug. 2019 kl. 11:23 skrev Komal Motwani <[hidden email]>:
>
> Hi,
>
>
>
> I have a use case where I would like a query to span across Cores
> (Multi-Core); all the cores involved do have same schema. I have started
> using solr just recently and have been trying to find ways to achieve this
> but couldn’t find any solution so far (Distributed searches, shards are not
> what I am looking for). I remember in one of the tech talks, there was a
> mention of this feature to be included in future releases. Appreciate any
> pointers to help me progress further.
>
>
>
> Thanks,
>
> Komal Motwani
Reply | Threaded
Open this post in threaded view
|

Re: Searches across Cores

Erick Erickson
So my question is why do you have individual cores? Why not use SolrCloud and collections and have this happen automatically?

There may be very good reasons, this is more if a sanity check….

> On Aug 9, 2019, at 8:02 AM, Jan Høydahl <[hidden email]> wrote:
>
> USE request param &shards=core1,core2 or if on separate machines host:port/solr/core1,host:port/solr/core2
>
> Jan Høydahl
>
>> 9. aug. 2019 kl. 11:23 skrev Komal Motwani <[hidden email]>:
>>
>> Hi,
>>
>>
>>
>> I have a use case where I would like a query to span across Cores
>> (Multi-Core); all the cores involved do have same schema. I have started
>> using solr just recently and have been trying to find ways to achieve this
>> but couldn’t find any solution so far (Distributed searches, shards are not
>> what I am looking for). I remember in one of the tech talks, there was a
>> mention of this feature to be included in future releases. Appreciate any
>> pointers to help me progress further.
>>
>>
>>
>> Thanks,
>>
>> Komal Motwani

Reply | Threaded
Open this post in threaded view
|

Re: Searches across Cores

Komal Motwani
For some good reasons, SolrCloud is not an option for me.
I need to run nested graph queries so firing parallel queries and taking
union/intersection won't work.
I am aware of achieving this via shards however I am looking for ways to
achieve this via multiple cores. We already have data existing in multiple
cores on which i need to add this feature.

Thanks,
Komal Motwani

On Fri, Aug 9, 2019 at 8:57 PM Erick Erickson <[hidden email]>
wrote:

> So my question is why do you have individual cores? Why not use SolrCloud
> and collections and have this happen automatically?
>
> There may be very good reasons, this is more if a sanity check….
>
> > On Aug 9, 2019, at 8:02 AM, Jan Høydahl <[hidden email]> wrote:
> >
> > USE request param &shards=core1,core2 or if on separate machines
> host:port/solr/core1,host:port/solr/core2
> >
> > Jan Høydahl
> >
> >> 9. aug. 2019 kl. 11:23 skrev Komal Motwani <[hidden email]>:
> >>
> >> Hi,
> >>
> >>
> >>
> >> I have a use case where I would like a query to span across Cores
> >> (Multi-Core); all the cores involved do have same schema. I have started
> >> using solr just recently and have been trying to find ways to achieve
> this
> >> but couldn’t find any solution so far (Distributed searches, shards are
> not
> >> what I am looking for). I remember in one of the tech talks, there was a
> >> mention of this feature to be included in future releases. Appreciate
> any
> >> pointers to help me progress further.
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Komal Motwani
>
>
Reply | Threaded
Open this post in threaded view
|

RE: Searches across Cores

Vadim Ivanov

May be consider having one collection with implicit sharding ?
This way you can have all advantages of solrcloud and can control content of each core "manualy" as well as query them independently (&distrib=false)
... or some of them using &shards=core1,core2 as was proposed before
Quote from doc
" If you created the collection and defined the "implicit" router at the time of creation, you can additionally define a router.field parameter to use a field from each document to identify a shard where the document belongs. If the field specified is missing in the document, however, the document will be rejected. You could also use the _route_ parameter to name a specific shard."
--
Vadim


> -----Original Message-----
> From: Komal Motwani [mailto:[hidden email]]
> Sent: Friday, August 09, 2019 7:57 PM
> To: [hidden email]
> Subject: Re: Searches across Cores
>
> For some good reasons, SolrCloud is not an option for me.
> I need to run nested graph queries so firing parallel queries and taking
> union/intersection won't work.
> I am aware of achieving this via shards however I am looking for ways to
> achieve this via multiple cores. We already have data existing in multiple
> cores on which i need to add this feature.
>
> Thanks,
> Komal Motwani
>
> On Fri, Aug 9, 2019 at 8:57 PM Erick Erickson <[hidden email]>
> wrote:
>
> > So my question is why do you have individual cores? Why not use SolrCloud
> > and collections and have this happen automatically?
> >
> > There may be very good reasons, this is more if a sanity check….
> >
> > > On Aug 9, 2019, at 8:02 AM, Jan Høydahl <[hidden email]> wrote:
> > >
> > > USE request param &shards=core1,core2 or if on separate machines
> > host:port/solr/core1,host:port/solr/core2
> > >
> > > Jan Høydahl
> > >
> > >> 9. aug. 2019 kl. 11:23 skrev Komal Motwani
> <[hidden email]>:
> > >>
> > >> Hi,
> > >>
> > >>
> > >>
> > >> I have a use case where I would like a query to span across Cores
> > >> (Multi-Core); all the cores involved do have same schema. I have started
> > >> using solr just recently and have been trying to find ways to achieve
> > this
> > >> but couldn’t find any solution so far (Distributed searches, shards are
> > not
> > >> what I am looking for). I remember in one of the tech talks, there was a
> > >> mention of this feature to be included in future releases. Appreciate
> > any
> > >> pointers to help me progress further.
> > >>
> > >>
> > >>
> > >> Thanks,
> > >>
> > >> Komal Motwani
> >
> >

Reply | Threaded
Open this post in threaded view
|

Re: Searches across Cores

Nicolas Franck
He's right. You can use the parameter "shards" for a very long time,
even before the whole solr cloud existed.

e.g. http://localhost:8983/solr/core0/select

with parameters:

  shards = localhost:8983/solr/core0,example.com:8983/solr/core0<http://example.com:8983/solr/core0>
  q = *:*
  defType = lucene

Yes, I used same core name twice (in the path and in the parameter), but I do not see another way.
You need to start the query at a query handler..

I guess your data is generated by several parties,
each on their own core? That makes sense.



On 9 Aug 2019, at 19:21, Vadim Ivanov <[hidden email]<mailto:[hidden email]>> wrote:


May be consider having one collection with implicit sharding ?
This way you can have all advantages of solrcloud and can control content of each core "manualy" as well as query them independently (&distrib=false)
... or some of them using &shards=core1,core2 as was proposed before
Quote from doc
" If you created the collection and defined the "implicit" router at the time of creation, you can additionally define a router.field parameter to use a field from each document to identify a shard where the document belongs. If the field specified is missing in the document, however, the document will be rejected. You could also use the _route_ parameter to name a specific shard."
--
Vadim


-----Original Message-----
From: Komal Motwani [mailto:[hidden email]]
Sent: Friday, August 09, 2019 7:57 PM
To: [hidden email]<mailto:[hidden email]>
Subject: Re: Searches across Cores

For some good reasons, SolrCloud is not an option for me.
I need to run nested graph queries so firing parallel queries and taking
union/intersection won't work.
I am aware of achieving this via shards however I am looking for ways to
achieve this via multiple cores. We already have data existing in multiple
cores on which i need to add this feature.

Thanks,
Komal Motwani

On Fri, Aug 9, 2019 at 8:57 PM Erick Erickson <[hidden email]<mailto:[hidden email]>>
wrote:

So my question is why do you have individual cores? Why not use SolrCloud
and collections and have this happen automatically?

There may be very good reasons, this is more if a sanity check….

On Aug 9, 2019, at 8:02 AM, Jan Høydahl <[hidden email]<mailto:[hidden email]>> wrote:

USE request param &shards=core1,core2 or if on separate machines
host:port/solr/core1,host:port/solr/core2

Jan Høydahl

9. aug. 2019 kl. 11:23 skrev Komal Motwani
<[hidden email]<mailto:[hidden email]>>:

Hi,



I have a use case where I would like a query to span across Cores
(Multi-Core); all the cores involved do have same schema. I have started
using solr just recently and have been trying to find ways to achieve
this
but couldn’t find any solution so far (Distributed searches, shards are
not
what I am looking for). I remember in one of the tech talks, there was a
mention of this feature to be included in future releases. Appreciate
any
pointers to help me progress further.



Thanks,

Komal Motwani