Question about sharding and overlapping

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about sharding and overlapping

Yago Riveiro
Hi,

I created a collection with 12 shards and route.field=month (month field will have values between 1 .. 12)

I notice that I have shards with more that  a month into them. This could left empty some shard and I want the documents one month in each shard.

My question is, how I configure the sharding method to avoid overlaps?

/Yago
Best regards /Yago
Reply | Threaded
Open this post in threaded view
|

Re: Question about sharding and overlapping

Shalin Shekhar Mangar
You can't control that if using the compositeIdRouter because the routing
is dependent on the hash function. What you want is custom sharding i.e.
the ability to control the shard to which updates are routed.

You should create a collection using the Collections API with a "shards"
param specifying the names of the shards you want to create. Then when
indexing documents, include a shard=X param to route requests directly to
that shard. While querying, you can choose to query the entire collection
or again limit the shards using the same "shard" parameter.


On Wed, Oct 23, 2013 at 4:22 AM, yriveiro <[hidden email]> wrote:

> Hi,
>
> I created a collection with 12 shards and route.field=month (month field
> will have values between 1 .. 12)
>
> I notice that I have shards with more that  a month into them. This could
> left empty some shard and I want the documents one month in each shard.
>
> My question is, how I configure the sharding method to avoid overlaps?
>
> /Yago
>
>
>
> -----
> Best regards
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Question-about-sharding-and-overlapping-tp4097111.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>



--
Regards,
Shalin Shekhar Mangar.
Reply | Threaded
Open this post in threaded view
|

Re: Question about sharding and overlapping

Yago Riveiro
Can I split shards as with compositeId using this method?

On Wednesday, October 23, 2013, Shalin Shekhar Mangar wrote:

> You can't control that if using the compositeIdRouter because the routing
> is dependent on the hash function. What you want is custom sharding i.e.
> the ability to control the shard to which updates are routed.
>
> You should create a collection using the Collections API with a "shards"
> param specifying the names of the shards you want to create. Then when
> indexing documents, include a shard=X param to route requests directly to
> that shard. While querying, you can choose to query the entire collection
> or again limit the shards using the same "shard" parameter.
>
>
> On Wed, Oct 23, 2013 at 4:22 AM, yriveiro <[hidden email]<javascript:;>>
> wrote:
>
> > Hi,
> >
> > I created a collection with 12 shards and route.field=month (month field
> > will have values between 1 .. 12)
> >
> > I notice that I have shards with more that  a month into them. This could
> > left empty some shard and I want the documents one month in each shard.
> >
> > My question is, how I configure the sharding method to avoid overlaps?
> >
> > /Yago
> >
> >
> >
> > -----
> > Best regards
> > --
> > View this message in context:
> >
> http://lucene.472066.n3.nabble.com/Question-about-sharding-and-overlapping-tp4097111.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>


--
/Yago Riveiro
Best regards /Yago
Reply | Threaded
Open this post in threaded view
|

Re: Question about sharding and overlapping

Shalin Shekhar Mangar
No, shard splitting does not support collections with implicit router.


On Wed, Oct 23, 2013 at 1:21 PM, Yago Riveiro <[hidden email]>wrote:

> Can I split shards as with compositeId using this method?
>
> On Wednesday, October 23, 2013, Shalin Shekhar Mangar wrote:
>
> > You can't control that if using the compositeIdRouter because the routing
> > is dependent on the hash function. What you want is custom sharding i.e.
> > the ability to control the shard to which updates are routed.
> >
> > You should create a collection using the Collections API with a "shards"
> > param specifying the names of the shards you want to create. Then when
> > indexing documents, include a shard=X param to route requests directly to
> > that shard. While querying, you can choose to query the entire collection
> > or again limit the shards using the same "shard" parameter.
> >
> >
> > On Wed, Oct 23, 2013 at 4:22 AM, yriveiro <[hidden email]
> <javascript:;>>
> > wrote:
> >
> > > Hi,
> > >
> > > I created a collection with 12 shards and route.field=month (month
> field
> > > will have values between 1 .. 12)
> > >
> > > I notice that I have shards with more that  a month into them. This
> could
> > > left empty some shard and I want the documents one month in each shard.
> > >
> > > My question is, how I configure the sharding method to avoid overlaps?
> > >
> > > /Yago
> > >
> > >
> > >
> > > -----
> > > Best regards
> > > --
> > > View this message in context:
> > >
> >
> http://lucene.472066.n3.nabble.com/Question-about-sharding-and-overlapping-tp4097111.html
> > > Sent from the Solr - User mailing list archive at Nabble.com.
> > >
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
>
>
> --
> /Yago Riveiro
>



--
Regards,
Shalin Shekhar Mangar.