How to split index more than 2GB in size

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

How to split index more than 2GB in size

Sushant Vengurlekar
How do I split indexes which are more than 2GB in size.

I get this error when I try to use SPLITSHARD on a collection of size more
than 2GB

2018-06-20 02:25:49.810 ERROR (qtp1025799482-19) [   ] o.a.s.s.HttpSolrCall
null:org.apache.solr.common.SolrException: SPLITSHARD failed to invoke
SPLIT core admin command

solr-1_1  | at
org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)

solr-1_1  | at
org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:258)

solr-1_1  | at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:230)

solr-1_1  | at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)

solr-1_1  | at
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)

solr-1_1  | at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)

solr-1_1  | at
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)

solr-1_1  | at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)

solr-1_1  | at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)

solr-1_1  | at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)

solr-1_1  | at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)

solr-1_1  | at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)

solr-1_1  | at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)

solr-1_1  | at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)

solr-1_1  | at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)

solr-1_1  | at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)

solr-1_1  | at
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)

solr-1_1  | at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)

solr-1_1  | at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)

solr-1_1  | at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)

solr-1_1  | at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)

solr-1_1  | at
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)

solr-1_1  | at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)

solr-1_1  | at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)

solr-1_1  | at
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)

solr-1_1  | at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)

solr-1_1  | at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)

solr-1_1  | at
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)

solr-1_1  | at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)

solr-1_1  | at org.eclipse.jetty.server.Server.handle(Server.java:530)

solr-1_1  | at
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)

solr-1_1  | at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)

solr-1_1  | at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)

solr-1_1  | at
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)

solr-1_1  | at
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)

solr-1_1  | at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)

solr-1_1  | at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)

solr-1_1  | at
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)

solr-1_1  | at
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)

solr-1_1  | at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)

solr-1_1  | at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)

solr-1_1  | at java.lang.Thread.run(Thread.java:748)

Thank you
Reply | Threaded
Open this post in threaded view
|

Re: How to split index more than 2GB in size

Alessandro Benedetti
Hi,
in the first place, why do you want to split 2 Gb indexes ?
Nowadays is a fairly small index.

Secondly what you reported is incomplete.
I would expect a Caused By section in the stacktrace.

This are generic recommendations, always spend time in analysing the problem
you had scrupulously.
- SolrCloud problems often involve more than one node. Be sure to check the
logs of all the nodes possibly involved.
- Report the full stack trace to the community
- Report your full request which provoked the exception

Help is much easier this way :)

Regards




-----
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
Reply | Threaded
Open this post in threaded view
|

Re: How to split index more than 2GB in size

Sushant Vengurlekar
Thanks for the reply Alessandro! Appreciate it.

Below is the full request and the error received

curl '
http://localhost:8081/solr/admin/collections?action=SPLITSHARD&collection=dev-transactions&shard=shard1
'

{

  "responseHeader":{

    "status":500,

    "QTime":7920},

  "success":{

    "solr-1:8081_solr":{

      "responseHeader":{

        "status":0,

        "QTime":1190},

      "core":"dev-transactions_shard1_0_replica_n3"},

    "solr-1:8081_solr":{

      "responseHeader":{

        "status":0,

        "QTime":1047},

      "core":"dev-transactions_shard1_1_replica_n4"},

    "solr-1:8081_solr":{

      "responseHeader":{

        "status":0,

        "QTime":6}},

    "solr-1:8081_solr":{

      "responseHeader":{

        "status":0,

        "QTime":1009}}},

  "failure":{

    "solr-1:8081_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
from server at http://solr-1:8081/solr: java.lang.IllegalArgumentException:
Cannot merge a segment that has been created with major version 6 into this
index which has been created by major version 7"},

  "Operation splitshard caused
exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
SPLITSHARD failed to invoke SPLIT core admin command",

  "exception":{

    "msg":"SPLITSHARD failed to invoke SPLIT core admin command",

    "rspCode":500},

  "error":{

    "metadata":[

      "error-class","org.apache.solr.common.SolrException",

      "root-error-class","org.apache.solr.common.SolrException"],

    "msg":"SPLITSHARD failed to invoke SPLIT core admin command",

    "trace":"org.apache.solr.common.SolrException: SPLITSHARD failed to
invoke SPLIT core admin command\n\tat
org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:258)\n\tat
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:230)\n\tat
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)\n\tat
org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)\n\tat
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)\n\tat
org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)\n\tat
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)\n\tat
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat
org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)\n\tat
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\n\tat
org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat
org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)\n\tat
org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\n\tat
org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)\n\tat
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)\n\tat
java.lang.Thread.run(Thread.java:748)\n",

    "code":500}}


On Wed, Jun 20, 2018 at 2:25 AM, Alessandro Benedetti <[hidden email]>
wrote:

> Hi,
> in the first place, why do you want to split 2 Gb indexes ?
> Nowadays is a fairly small index.
>
> Secondly what you reported is incomplete.
> I would expect a Caused By section in the stacktrace.
>
> This are generic recommendations, always spend time in analysing the
> problem
> you had scrupulously.
> - SolrCloud problems often involve more than one node. Be sure to check the
> logs of all the nodes possibly involved.
> - Report the full stack trace to the community
> - Report your full request which provoked the exception
>
> Help is much easier this way :)
>
> Regards
>
>
>
>
> -----
> ---------------
> Alessandro Benedetti
> Search Consultant, R&D Software Engineer, Director
> Sease Ltd. - www.sease.io
> --
> Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>
Reply | Threaded
Open this post in threaded view
|

Re: How to split index more than 2GB in size

Alexandre Rafalovitch
This seems more related to an old index upgraded to latest Solr rather than
the split itself.

Regards,
    Alex

On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, <
[hidden email]> wrote:

> Thanks for the reply Alessandro! Appreciate it.
>
> Below is the full request and the error received
>
> curl '
>
> http://localhost:8081/solr/admin/collections?action=SPLITSHARD&collection=dev-transactions&shard=shard1
> '
>
> {
>
>   "responseHeader":{
>
>     "status":500,
>
>     "QTime":7920},
>
>   "success":{
>
>     "solr-1:8081_solr":{
>
>       "responseHeader":{
>
>         "status":0,
>
>         "QTime":1190},
>
>       "core":"dev-transactions_shard1_0_replica_n3"},
>
>     "solr-1:8081_solr":{
>
>       "responseHeader":{
>
>         "status":0,
>
>         "QTime":1047},
>
>       "core":"dev-transactions_shard1_1_replica_n4"},
>
>     "solr-1:8081_solr":{
>
>       "responseHeader":{
>
>         "status":0,
>
>         "QTime":6}},
>
>     "solr-1:8081_solr":{
>
>       "responseHeader":{
>
>         "status":0,
>
>         "QTime":1009}}},
>
>   "failure":{
>
>
> "solr-1:8081_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error
> from server at http://solr-1:8081/solr:
> java.lang.IllegalArgumentException:
> Cannot merge a segment that has been created with major version 6 into this
> index which has been created by major version 7"},
>
>   "Operation splitshard caused
>
> exception:":"org.apache.solr.common.SolrException:org.apache.solr.common.SolrException:
> SPLITSHARD failed to invoke SPLIT core admin command",
>
>   "exception":{
>
>     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
>
>     "rspCode":500},
>
>   "error":{
>
>     "metadata":[
>
>       "error-class","org.apache.solr.common.SolrException",
>
>       "root-error-class","org.apache.solr.common.SolrException"],
>
>     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
>
>     "trace":"org.apache.solr.common.SolrException: SPLITSHARD failed to
> invoke SPLIT core admin command\n\tat
>
> org.apache.solr.client.solrj.SolrResponse.getException(SolrResponse.java:53)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:258)\n\tat
>
> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:230)\n\tat
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)\n\tat
>
> org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736)\n\tat
>
> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717)\n\tat
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)\n\tat
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat
>
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)\n\tat
>
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)\n\tat
>
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)\n\tat
>
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat
>
> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
>
> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)\n\tat
>
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)\n\tat
> org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat
>
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)\n\tat
> org.eclipse.jetty.io
> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\n\tat
> org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat
> org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\tat
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)\n\tat
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)\n\tat
>
> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\n\tat
>
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)\n\tat
>
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)\n\tat
>
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)\n\tat
> java.lang.Thread.run(Thread.java:748)\n",
>
>     "code":500}}
>
>
> On Wed, Jun 20, 2018 at 2:25 AM, Alessandro Benedetti <
> [hidden email]>
> wrote:
>
> > Hi,
> > in the first place, why do you want to split 2 Gb indexes ?
> > Nowadays is a fairly small index.
> >
> > Secondly what you reported is incomplete.
> > I would expect a Caused By section in the stacktrace.
> >
> > This are generic recommendations, always spend time in analysing the
> > problem
> > you had scrupulously.
> > - SolrCloud problems often involve more than one node. Be sure to check
> the
> > logs of all the nodes possibly involved.
> > - Report the full stack trace to the community
> > - Report your full request which provoked the exception
> >
> > Help is much easier this way :)
> >
> > Regards
> >
> >
> >
> >
> > -----
> > ---------------
> > Alessandro Benedetti
> > Search Consultant, R&D Software Engineer, Director
> > Sease Ltd. - www.sease.io
> > --
> > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: How to split index more than 2GB in size

Sushant Vengurlekar
My old solr instance was 6.6.3 and the current solrcloud I am building is
7.3.1. Are there any issues there?

On Wed, Jun 20, 2018 at 9:11 AM, Alexandre Rafalovitch <[hidden email]>
wrote:

> This seems more related to an old index upgraded to latest Solr rather than
> the split itself.
>
> Regards,
>     Alex
>
> On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, <
> [hidden email]> wrote:
>
> > Thanks for the reply Alessandro! Appreciate it.
> >
> > Below is the full request and the error received
> >
> > curl '
> >
> > http://localhost:8081/solr/admin/collections?action=
> SPLITSHARD&collection=dev-transactions&shard=shard1
> > '
> >
> > {
> >
> >   "responseHeader":{
> >
> >     "status":500,
> >
> >     "QTime":7920},
> >
> >   "success":{
> >
> >     "solr-1:8081_solr":{
> >
> >       "responseHeader":{
> >
> >         "status":0,
> >
> >         "QTime":1190},
> >
> >       "core":"dev-transactions_shard1_0_replica_n3"},
> >
> >     "solr-1:8081_solr":{
> >
> >       "responseHeader":{
> >
> >         "status":0,
> >
> >         "QTime":1047},
> >
> >       "core":"dev-transactions_shard1_1_replica_n4"},
> >
> >     "solr-1:8081_solr":{
> >
> >       "responseHeader":{
> >
> >         "status":0,
> >
> >         "QTime":6}},
> >
> >     "solr-1:8081_solr":{
> >
> >       "responseHeader":{
> >
> >         "status":0,
> >
> >         "QTime":1009}}},
> >
> >   "failure":{
> >
> >
> > "solr-1:8081_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$
> RemoteSolrException:Error
> > from server at http://solr-1:8081/solr:
> > java.lang.IllegalArgumentException:
> > Cannot merge a segment that has been created with major version 6 into
> this
> > index which has been created by major version 7"},
> >
> >   "Operation splitshard caused
> >
> > exception:":"org.apache.solr.common.SolrException:org.
> apache.solr.common.SolrException:
> > SPLITSHARD failed to invoke SPLIT core admin command",
> >
> >   "exception":{
> >
> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
> >
> >     "rspCode":500},
> >
> >   "error":{
> >
> >     "metadata":[
> >
> >       "error-class","org.apache.solr.common.SolrException",
> >
> >       "root-error-class","org.apache.solr.common.SolrException"],
> >
> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
> >
> >     "trace":"org.apache.solr.common.SolrException: SPLITSHARD failed to
> > invoke SPLIT core admin command\n\tat
> >
> > org.apache.solr.client.solrj.SolrResponse.getException(
> SolrResponse.java:53)\n\tat
> >
> > org.apache.solr.handler.admin.CollectionsHandler.invokeAction(
> CollectionsHandler.java:258)\n\tat
> >
> > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
> CollectionsHandler.java:230)\n\tat
> >
> > org.apache.solr.handler.RequestHandlerBase.handleRequest(
> RequestHandlerBase.java:195)\n\tat
> >
> > org.apache.solr.servlet.HttpSolrCall.handleAdmin(
> HttpSolrCall.java:736)\n\tat
> >
> > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(
> HttpSolrCall.java:717)\n\tat
> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat
> >
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:384)\n\tat
> >
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:330)\n\tat
> >
> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1629)\n\tat
> >
> > org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:533)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:143)\n\tat
> >
> > org.eclipse.jetty.security.SecurityHandler.handle(
> SecurityHandler.java:548)\n\tat
> >
> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:132)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.
> nextHandle(ScopedHandler.java:190)\n\tat
> >
> > org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:1595)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.
> nextHandle(ScopedHandler.java:188)\n\tat
> >
> > org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1253)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.
> nextScope(ScopedHandler.java:168)\n\tat
> >
> > org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:473)\n\tat
> >
> > org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:1564)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.
> nextScope(ScopedHandler.java:166)\n\tat
> >
> > org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:1155)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:141)\n\tat
> >
> > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> ContextHandlerCollection.java:219)\n\tat
> >
> > org.eclipse.jetty.server.handler.HandlerCollection.
> handle(HandlerCollection.java:126)\n\tat
> >
> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:132)\n\tat
> >
> > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(
> RewriteHandler.java:335)\n\tat
> >
> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:132)\n\tat
> > org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat
> > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat
> >
> > org.eclipse.jetty.server.HttpConnection.onFillable(
> HttpConnection.java:256)\n\tat
> > org.eclipse.jetty.io
> > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\
> n\tat
> > org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat
> > org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\
> tat
> >
> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
> doProduce(EatWhatYouKill.java:247)\n\tat
> >
> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
> produce(EatWhatYouKill.java:140)\n\tat
> >
> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
> EatWhatYouKill.java:131)\n\tat
> >
> > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(
> ReservedThreadExecutor.java:382)\n\tat
> >
> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> QueuedThreadPool.java:708)\n\tat
> >
> > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
> QueuedThreadPool.java:626)\n\tat
> > java.lang.Thread.run(Thread.java:748)\n",
> >
> >     "code":500}}
> >
> >
> > On Wed, Jun 20, 2018 at 2:25 AM, Alessandro Benedetti <
> > [hidden email]>
> > wrote:
> >
> > > Hi,
> > > in the first place, why do you want to split 2 Gb indexes ?
> > > Nowadays is a fairly small index.
> > >
> > > Secondly what you reported is incomplete.
> > > I would expect a Caused By section in the stacktrace.
> > >
> > > This are generic recommendations, always spend time in analysing the
> > > problem
> > > you had scrupulously.
> > > - SolrCloud problems often involve more than one node. Be sure to check
> > the
> > > logs of all the nodes possibly involved.
> > > - Report the full stack trace to the community
> > > - Report your full request which provoked the exception
> > >
> > > Help is much easier this way :)
> > >
> > > Regards
> > >
> > >
> > >
> > >
> > > -----
> > > ---------------
> > > Alessandro Benedetti
> > > Search Consultant, R&D Software Engineer, Director
> > > Sease Ltd. - www.sease.io
> > > --
> > > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: How to split index more than 2GB in size

Sushant Vengurlekar
In reply to this post by Alexandre Rafalovitch
How can I resolve this error?

On Wed, Jun 20, 2018 at 9:11 AM, Alexandre Rafalovitch <[hidden email]>
wrote:

> This seems more related to an old index upgraded to latest Solr rather than
> the split itself.
>
> Regards,
>     Alex
>
> On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, <
> [hidden email]> wrote:
>
> > Thanks for the reply Alessandro! Appreciate it.
> >
> > Below is the full request and the error received
> >
> > curl '
> >
> > http://localhost:8081/solr/admin/collections?action=
> SPLITSHARD&collection=dev-transactions&shard=shard1
> > '
> >
> > {
> >
> >   "responseHeader":{
> >
> >     "status":500,
> >
> >     "QTime":7920},
> >
> >   "success":{
> >
> >     "solr-1:8081_solr":{
> >
> >       "responseHeader":{
> >
> >         "status":0,
> >
> >         "QTime":1190},
> >
> >       "core":"dev-transactions_shard1_0_replica_n3"},
> >
> >     "solr-1:8081_solr":{
> >
> >       "responseHeader":{
> >
> >         "status":0,
> >
> >         "QTime":1047},
> >
> >       "core":"dev-transactions_shard1_1_replica_n4"},
> >
> >     "solr-1:8081_solr":{
> >
> >       "responseHeader":{
> >
> >         "status":0,
> >
> >         "QTime":6}},
> >
> >     "solr-1:8081_solr":{
> >
> >       "responseHeader":{
> >
> >         "status":0,
> >
> >         "QTime":1009}}},
> >
> >   "failure":{
> >
> >
> > "solr-1:8081_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$
> RemoteSolrException:Error
> > from server at http://solr-1:8081/solr:
> > java.lang.IllegalArgumentException:
> > Cannot merge a segment that has been created with major version 6 into
> this
> > index which has been created by major version 7"},
> >
> >   "Operation splitshard caused
> >
> > exception:":"org.apache.solr.common.SolrException:org.
> apache.solr.common.SolrException:
> > SPLITSHARD failed to invoke SPLIT core admin command",
> >
> >   "exception":{
> >
> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
> >
> >     "rspCode":500},
> >
> >   "error":{
> >
> >     "metadata":[
> >
> >       "error-class","org.apache.solr.common.SolrException",
> >
> >       "root-error-class","org.apache.solr.common.SolrException"],
> >
> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
> >
> >     "trace":"org.apache.solr.common.SolrException: SPLITSHARD failed to
> > invoke SPLIT core admin command\n\tat
> >
> > org.apache.solr.client.solrj.SolrResponse.getException(
> SolrResponse.java:53)\n\tat
> >
> > org.apache.solr.handler.admin.CollectionsHandler.invokeAction(
> CollectionsHandler.java:258)\n\tat
> >
> > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
> CollectionsHandler.java:230)\n\tat
> >
> > org.apache.solr.handler.RequestHandlerBase.handleRequest(
> RequestHandlerBase.java:195)\n\tat
> >
> > org.apache.solr.servlet.HttpSolrCall.handleAdmin(
> HttpSolrCall.java:736)\n\tat
> >
> > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(
> HttpSolrCall.java:717)\n\tat
> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat
> >
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:384)\n\tat
> >
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> SolrDispatchFilter.java:330)\n\tat
> >
> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> doFilter(ServletHandler.java:1629)\n\tat
> >
> > org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:533)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:143)\n\tat
> >
> > org.eclipse.jetty.security.SecurityHandler.handle(
> SecurityHandler.java:548)\n\tat
> >
> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:132)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.
> nextHandle(ScopedHandler.java:190)\n\tat
> >
> > org.eclipse.jetty.server.session.SessionHandler.
> doHandle(SessionHandler.java:1595)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.
> nextHandle(ScopedHandler.java:188)\n\tat
> >
> > org.eclipse.jetty.server.handler.ContextHandler.
> doHandle(ContextHandler.java:1253)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.
> nextScope(ScopedHandler.java:168)\n\tat
> >
> > org.eclipse.jetty.servlet.ServletHandler.doScope(
> ServletHandler.java:473)\n\tat
> >
> > org.eclipse.jetty.server.session.SessionHandler.
> doScope(SessionHandler.java:1564)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.
> nextScope(ScopedHandler.java:166)\n\tat
> >
> > org.eclipse.jetty.server.handler.ContextHandler.
> doScope(ContextHandler.java:1155)\n\tat
> >
> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> ScopedHandler.java:141)\n\tat
> >
> > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> ContextHandlerCollection.java:219)\n\tat
> >
> > org.eclipse.jetty.server.handler.HandlerCollection.
> handle(HandlerCollection.java:126)\n\tat
> >
> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:132)\n\tat
> >
> > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(
> RewriteHandler.java:335)\n\tat
> >
> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> HandlerWrapper.java:132)\n\tat
> > org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat
> > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat
> >
> > org.eclipse.jetty.server.HttpConnection.onFillable(
> HttpConnection.java:256)\n\tat
> > org.eclipse.jetty.io
> > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\
> n\tat
> > org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat
> > org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\
> tat
> >
> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
> doProduce(EatWhatYouKill.java:247)\n\tat
> >
> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
> produce(EatWhatYouKill.java:140)\n\tat
> >
> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
> EatWhatYouKill.java:131)\n\tat
> >
> > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(
> ReservedThreadExecutor.java:382)\n\tat
> >
> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> QueuedThreadPool.java:708)\n\tat
> >
> > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
> QueuedThreadPool.java:626)\n\tat
> > java.lang.Thread.run(Thread.java:748)\n",
> >
> >     "code":500}}
> >
> >
> > On Wed, Jun 20, 2018 at 2:25 AM, Alessandro Benedetti <
> > [hidden email]>
> > wrote:
> >
> > > Hi,
> > > in the first place, why do you want to split 2 Gb indexes ?
> > > Nowadays is a fairly small index.
> > >
> > > Secondly what you reported is incomplete.
> > > I would expect a Caused By section in the stacktrace.
> > >
> > > This are generic recommendations, always spend time in analysing the
> > > problem
> > > you had scrupulously.
> > > - SolrCloud problems often involve more than one node. Be sure to check
> > the
> > > logs of all the nodes possibly involved.
> > > - Report the full stack trace to the community
> > > - Report your full request which provoked the exception
> > >
> > > Help is much easier this way :)
> > >
> > > Regards
> > >
> > >
> > >
> > >
> > > -----
> > > ---------------
> > > Alessandro Benedetti
> > > Search Consultant, R&D Software Engineer, Director
> > > Sease Ltd. - www.sease.io
> > > --
> > > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: How to split index more than 2GB in size

Erick Erickson
Use the indexupgrader tool or optimize your index before using splitshard.

Since this is a small index (< 5G), optimizing will not create an
overly-large segment, so that pitfall is avoided.

You haven't yet explained why you think splitting the index would be
beneficial. Splitting an index this small is unlikely to improve query
performance appreciably. This feels a lot like an "XY" problem, you're
asking how to do X thinking it will solve Y but not telling us what Y
is.

Best,
Erick

On Wed, Jun 20, 2018 at 9:40 AM, Sushant Vengurlekar
<[hidden email]> wrote:

> How can I resolve this error?
>
> On Wed, Jun 20, 2018 at 9:11 AM, Alexandre Rafalovitch <[hidden email]>
> wrote:
>
>> This seems more related to an old index upgraded to latest Solr rather than
>> the split itself.
>>
>> Regards,
>>     Alex
>>
>> On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, <
>> [hidden email]> wrote:
>>
>> > Thanks for the reply Alessandro! Appreciate it.
>> >
>> > Below is the full request and the error received
>> >
>> > curl '
>> >
>> > http://localhost:8081/solr/admin/collections?action=
>> SPLITSHARD&collection=dev-transactions&shard=shard1
>> > '
>> >
>> > {
>> >
>> >   "responseHeader":{
>> >
>> >     "status":500,
>> >
>> >     "QTime":7920},
>> >
>> >   "success":{
>> >
>> >     "solr-1:8081_solr":{
>> >
>> >       "responseHeader":{
>> >
>> >         "status":0,
>> >
>> >         "QTime":1190},
>> >
>> >       "core":"dev-transactions_shard1_0_replica_n3"},
>> >
>> >     "solr-1:8081_solr":{
>> >
>> >       "responseHeader":{
>> >
>> >         "status":0,
>> >
>> >         "QTime":1047},
>> >
>> >       "core":"dev-transactions_shard1_1_replica_n4"},
>> >
>> >     "solr-1:8081_solr":{
>> >
>> >       "responseHeader":{
>> >
>> >         "status":0,
>> >
>> >         "QTime":6}},
>> >
>> >     "solr-1:8081_solr":{
>> >
>> >       "responseHeader":{
>> >
>> >         "status":0,
>> >
>> >         "QTime":1009}}},
>> >
>> >   "failure":{
>> >
>> >
>> > "solr-1:8081_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$
>> RemoteSolrException:Error
>> > from server at http://solr-1:8081/solr:
>> > java.lang.IllegalArgumentException:
>> > Cannot merge a segment that has been created with major version 6 into
>> this
>> > index which has been created by major version 7"},
>> >
>> >   "Operation splitshard caused
>> >
>> > exception:":"org.apache.solr.common.SolrException:org.
>> apache.solr.common.SolrException:
>> > SPLITSHARD failed to invoke SPLIT core admin command",
>> >
>> >   "exception":{
>> >
>> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
>> >
>> >     "rspCode":500},
>> >
>> >   "error":{
>> >
>> >     "metadata":[
>> >
>> >       "error-class","org.apache.solr.common.SolrException",
>> >
>> >       "root-error-class","org.apache.solr.common.SolrException"],
>> >
>> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
>> >
>> >     "trace":"org.apache.solr.common.SolrException: SPLITSHARD failed to
>> > invoke SPLIT core admin command\n\tat
>> >
>> > org.apache.solr.client.solrj.SolrResponse.getException(
>> SolrResponse.java:53)\n\tat
>> >
>> > org.apache.solr.handler.admin.CollectionsHandler.invokeAction(
>> CollectionsHandler.java:258)\n\tat
>> >
>> > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
>> CollectionsHandler.java:230)\n\tat
>> >
>> > org.apache.solr.handler.RequestHandlerBase.handleRequest(
>> RequestHandlerBase.java:195)\n\tat
>> >
>> > org.apache.solr.servlet.HttpSolrCall.handleAdmin(
>> HttpSolrCall.java:736)\n\tat
>> >
>> > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(
>> HttpSolrCall.java:717)\n\tat
>> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat
>> >
>> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>> SolrDispatchFilter.java:384)\n\tat
>> >
>> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>> SolrDispatchFilter.java:330)\n\tat
>> >
>> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.
>> doFilter(ServletHandler.java:1629)\n\tat
>> >
>> > org.eclipse.jetty.servlet.ServletHandler.doHandle(
>> ServletHandler.java:533)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
>> ScopedHandler.java:143)\n\tat
>> >
>> > org.eclipse.jetty.security.SecurityHandler.handle(
>> SecurityHandler.java:548)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>> HandlerWrapper.java:132)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.ScopedHandler.
>> nextHandle(ScopedHandler.java:190)\n\tat
>> >
>> > org.eclipse.jetty.server.session.SessionHandler.
>> doHandle(SessionHandler.java:1595)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.ScopedHandler.
>> nextHandle(ScopedHandler.java:188)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.ContextHandler.
>> doHandle(ContextHandler.java:1253)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.ScopedHandler.
>> nextScope(ScopedHandler.java:168)\n\tat
>> >
>> > org.eclipse.jetty.servlet.ServletHandler.doScope(
>> ServletHandler.java:473)\n\tat
>> >
>> > org.eclipse.jetty.server.session.SessionHandler.
>> doScope(SessionHandler.java:1564)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.ScopedHandler.
>> nextScope(ScopedHandler.java:166)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.ContextHandler.
>> doScope(ContextHandler.java:1155)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
>> ScopedHandler.java:141)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
>> ContextHandlerCollection.java:219)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.HandlerCollection.
>> handle(HandlerCollection.java:126)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>> HandlerWrapper.java:132)\n\tat
>> >
>> > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(
>> RewriteHandler.java:335)\n\tat
>> >
>> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>> HandlerWrapper.java:132)\n\tat
>> > org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat
>> > org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat
>> >
>> > org.eclipse.jetty.server.HttpConnection.onFillable(
>> HttpConnection.java:256)\n\tat
>> > org.eclipse.jetty.io
>> > .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\
>> n\tat
>> > org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)\n\tat
>> > org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\
>> tat
>> >
>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
>> doProduce(EatWhatYouKill.java:247)\n\tat
>> >
>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
>> produce(EatWhatYouKill.java:140)\n\tat
>> >
>> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
>> EatWhatYouKill.java:131)\n\tat
>> >
>> > org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(
>> ReservedThreadExecutor.java:382)\n\tat
>> >
>> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>> QueuedThreadPool.java:708)\n\tat
>> >
>> > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
>> QueuedThreadPool.java:626)\n\tat
>> > java.lang.Thread.run(Thread.java:748)\n",
>> >
>> >     "code":500}}
>> >
>> >
>> > On Wed, Jun 20, 2018 at 2:25 AM, Alessandro Benedetti <
>> > [hidden email]>
>> > wrote:
>> >
>> > > Hi,
>> > > in the first place, why do you want to split 2 Gb indexes ?
>> > > Nowadays is a fairly small index.
>> > >
>> > > Secondly what you reported is incomplete.
>> > > I would expect a Caused By section in the stacktrace.
>> > >
>> > > This are generic recommendations, always spend time in analysing the
>> > > problem
>> > > you had scrupulously.
>> > > - SolrCloud problems often involve more than one node. Be sure to check
>> > the
>> > > logs of all the nodes possibly involved.
>> > > - Report the full stack trace to the community
>> > > - Report your full request which provoked the exception
>> > >
>> > > Help is much easier this way :)
>> > >
>> > > Regards
>> > >
>> > >
>> > >
>> > >
>> > > -----
>> > > ---------------
>> > > Alessandro Benedetti
>> > > Search Consultant, R&D Software Engineer, Director
>> > > Sease Ltd. - www.sease.io
>> > > --
>> > > Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>> > >
>> >
>>
Reply | Threaded
Open this post in threaded view
|

Re: How to split index more than 2GB in size

Sushant Vengurlekar
The index size is small because this is my local development copy.  The
production index is more than 20GB. So I am working on getting the index
split and replicated on different nodes. Our current instance on prod is
single instance solr 6 which we are working on moving towards solrcloud 7

On Wed, Jun 20, 2018 at 10:30 AM Erick Erickson <[hidden email]>
wrote:

> Use the indexupgrader tool or optimize your index before using splitshard.
>
> Since this is a small index (< 5G), optimizing will not create an
> overly-large segment, so that pitfall is avoided.
>
> You haven't yet explained why you think splitting the index would be
> beneficial. Splitting an index this small is unlikely to improve query
> performance appreciably. This feels a lot like an "XY" problem, you're
> asking how to do X thinking it will solve Y but not telling us what Y
> is.
>
> Best,
> Erick
>
> On Wed, Jun 20, 2018 at 9:40 AM, Sushant Vengurlekar
> <[hidden email]> wrote:
> > How can I resolve this error?
> >
> > On Wed, Jun 20, 2018 at 9:11 AM, Alexandre Rafalovitch <
> [hidden email]>
> > wrote:
> >
> >> This seems more related to an old index upgraded to latest Solr rather
> than
> >> the split itself.
> >>
> >> Regards,
> >>     Alex
> >>
> >> On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, <
> >> [hidden email]> wrote:
> >>
> >> > Thanks for the reply Alessandro! Appreciate it.
> >> >
> >> > Below is the full request and the error received
> >> >
> >> > curl '
> >> >
> >> > http://localhost:8081/solr/admin/collections?action=
> >> SPLITSHARD&collection=dev-transactions&shard=shard1
> >> > '
> >> >
> >> > {
> >> >
> >> >   "responseHeader":{
> >> >
> >> >     "status":500,
> >> >
> >> >     "QTime":7920},
> >> >
> >> >   "success":{
> >> >
> >> >     "solr-1:8081_solr":{
> >> >
> >> >       "responseHeader":{
> >> >
> >> >         "status":0,
> >> >
> >> >         "QTime":1190},
> >> >
> >> >       "core":"dev-transactions_shard1_0_replica_n3"},
> >> >
> >> >     "solr-1:8081_solr":{
> >> >
> >> >       "responseHeader":{
> >> >
> >> >         "status":0,
> >> >
> >> >         "QTime":1047},
> >> >
> >> >       "core":"dev-transactions_shard1_1_replica_n4"},
> >> >
> >> >     "solr-1:8081_solr":{
> >> >
> >> >       "responseHeader":{
> >> >
> >> >         "status":0,
> >> >
> >> >         "QTime":6}},
> >> >
> >> >     "solr-1:8081_solr":{
> >> >
> >> >       "responseHeader":{
> >> >
> >> >         "status":0,
> >> >
> >> >         "QTime":1009}}},
> >> >
> >> >   "failure":{
> >> >
> >> >
> >> > "solr-1:8081_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$
> >> RemoteSolrException:Error
> >> > from server at http://solr-1:8081/solr:
> >> > java.lang.IllegalArgumentException:
> >> > Cannot merge a segment that has been created with major version 6 into
> >> this
> >> > index which has been created by major version 7"},
> >> >
> >> >   "Operation splitshard caused
> >> >
> >> > exception:":"org.apache.solr.common.SolrException:org.
> >> apache.solr.common.SolrException:
> >> > SPLITSHARD failed to invoke SPLIT core admin command",
> >> >
> >> >   "exception":{
> >> >
> >> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
> >> >
> >> >     "rspCode":500},
> >> >
> >> >   "error":{
> >> >
> >> >     "metadata":[
> >> >
> >> >       "error-class","org.apache.solr.common.SolrException",
> >> >
> >> >       "root-error-class","org.apache.solr.common.SolrException"],
> >> >
> >> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
> >> >
> >> >     "trace":"org.apache.solr.common.SolrException: SPLITSHARD failed
> to
> >> > invoke SPLIT core admin command\n\tat
> >> >
> >> > org.apache.solr.client.solrj.SolrResponse.getException(
> >> SolrResponse.java:53)\n\tat
> >> >
> >> > org.apache.solr.handler.admin.CollectionsHandler.invokeAction(
> >> CollectionsHandler.java:258)\n\tat
> >> >
> >> > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
> >> CollectionsHandler.java:230)\n\tat
> >> >
> >> > org.apache.solr.handler.RequestHandlerBase.handleRequest(
> >> RequestHandlerBase.java:195)\n\tat
> >> >
> >> > org.apache.solr.servlet.HttpSolrCall.handleAdmin(
> >> HttpSolrCall.java:736)\n\tat
> >> >
> >> > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(
> >> HttpSolrCall.java:717)\n\tat
> >> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat
> >> >
> >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> >> SolrDispatchFilter.java:384)\n\tat
> >> >
> >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> >> SolrDispatchFilter.java:330)\n\tat
> >> >
> >> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> >> doFilter(ServletHandler.java:1629)\n\tat
> >> >
> >> > org.eclipse.jetty.servlet.ServletHandler.doHandle(
> >> ServletHandler.java:533)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> >> ScopedHandler.java:143)\n\tat
> >> >
> >> > org.eclipse.jetty.security.SecurityHandler.handle(
> >> SecurityHandler.java:548)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> >> HandlerWrapper.java:132)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.ScopedHandler.
> >> nextHandle(ScopedHandler.java:190)\n\tat
> >> >
> >> > org.eclipse.jetty.server.session.SessionHandler.
> >> doHandle(SessionHandler.java:1595)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.ScopedHandler.
> >> nextHandle(ScopedHandler.java:188)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.ContextHandler.
> >> doHandle(ContextHandler.java:1253)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.ScopedHandler.
> >> nextScope(ScopedHandler.java:168)\n\tat
> >> >
> >> > org.eclipse.jetty.servlet.ServletHandler.doScope(
> >> ServletHandler.java:473)\n\tat
> >> >
> >> > org.eclipse.jetty.server.session.SessionHandler.
> >> doScope(SessionHandler.java:1564)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.ScopedHandler.
> >> nextScope(ScopedHandler.java:166)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.ContextHandler.
> >> doScope(ContextHandler.java:1155)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> >> ScopedHandler.java:141)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> >> ContextHandlerCollection.java:219)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.HandlerCollection.
> >> handle(HandlerCollection.java:126)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> >> HandlerWrapper.java:132)\n\tat
> >> >
> >> > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(
> >> RewriteHandler.java:335)\n\tat
> >> >
> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> >> HandlerWrapper.java:132)\n\tat
> >> > org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat
> >> >
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat
> >> >
> >> > org.eclipse.jetty.server.HttpConnection.onFillable(
> >> HttpConnection.java:256)\n\tat
> >> > org.eclipse.jetty.io
> >> >
> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\
> >> n\tat
> >> > org.eclipse.jetty.io
> .FillInterest.fillable(FillInterest.java:102)\n\tat
> >> > org.eclipse.jetty.io
> .ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\
> >> tat
> >> >
> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
> >> doProduce(EatWhatYouKill.java:247)\n\tat
> >> >
> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
> >> produce(EatWhatYouKill.java:140)\n\tat
> >> >
> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
> >> EatWhatYouKill.java:131)\n\tat
> >> >
> >> >
> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(
> >> ReservedThreadExecutor.java:382)\n\tat
> >> >
> >> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> >> QueuedThreadPool.java:708)\n\tat
> >> >
> >> > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
> >> QueuedThreadPool.java:626)\n\tat
> >> > java.lang.Thread.run(Thread.java:748)\n",
> >> >
> >> >     "code":500}}
> >> >
> >> >
> >> > On Wed, Jun 20, 2018 at 2:25 AM, Alessandro Benedetti <
> >> > [hidden email]>
> >> > wrote:
> >> >
> >> > > Hi,
> >> > > in the first place, why do you want to split 2 Gb indexes ?
> >> > > Nowadays is a fairly small index.
> >> > >
> >> > > Secondly what you reported is incomplete.
> >> > > I would expect a Caused By section in the stacktrace.
> >> > >
> >> > > This are generic recommendations, always spend time in analysing the
> >> > > problem
> >> > > you had scrupulously.
> >> > > - SolrCloud problems often involve more than one node. Be sure to
> check
> >> > the
> >> > > logs of all the nodes possibly involved.
> >> > > - Report the full stack trace to the community
> >> > > - Report your full request which provoked the exception
> >> > >
> >> > > Help is much easier this way :)
> >> > >
> >> > > Regards
> >> > >
> >> > >
> >> > >
> >> > >
> >> > > -----
> >> > > ---------------
> >> > > Alessandro Benedetti
> >> > > Search Consultant, R&D Software Engineer, Director
> >> > > Sease Ltd. - www.sease.io
> >> > > --
> >> > > Sent from:
> http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> >> > >
> >> >
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: How to split index more than 2GB in size

Erick Erickson
You still haven't answered _why_ you think splitting even a 20G index
is desirable. We regularly see 200G+ indexes per replica in the field,
so what's the point? Have you measured different setups to see if it's
a good idea? A 200G index needs some beefy hardware admittedly....

If you have adequate response times with a 20G index and need to
increase the QPS rate, just add more replicas. Having more than one
shard inevitably adds overhead which may (or may not) be made up for
by parallelizing some of the work. It's nearly always better to use
only one shard if it meets your response time requirements.

Best,
Erick

On Wed, Jun 20, 2018 at 10:39 AM, Sushant Vengurlekar
<[hidden email]> wrote:

> The index size is small because this is my local development copy.  The
> production index is more than 20GB. So I am working on getting the index
> split and replicated on different nodes. Our current instance on prod is
> single instance solr 6 which we are working on moving towards solrcloud 7
>
> On Wed, Jun 20, 2018 at 10:30 AM Erick Erickson <[hidden email]>
> wrote:
>
>> Use the indexupgrader tool or optimize your index before using splitshard.
>>
>> Since this is a small index (< 5G), optimizing will not create an
>> overly-large segment, so that pitfall is avoided.
>>
>> You haven't yet explained why you think splitting the index would be
>> beneficial. Splitting an index this small is unlikely to improve query
>> performance appreciably. This feels a lot like an "XY" problem, you're
>> asking how to do X thinking it will solve Y but not telling us what Y
>> is.
>>
>> Best,
>> Erick
>>
>> On Wed, Jun 20, 2018 at 9:40 AM, Sushant Vengurlekar
>> <[hidden email]> wrote:
>> > How can I resolve this error?
>> >
>> > On Wed, Jun 20, 2018 at 9:11 AM, Alexandre Rafalovitch <
>> [hidden email]>
>> > wrote:
>> >
>> >> This seems more related to an old index upgraded to latest Solr rather
>> than
>> >> the split itself.
>> >>
>> >> Regards,
>> >>     Alex
>> >>
>> >> On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, <
>> >> [hidden email]> wrote:
>> >>
>> >> > Thanks for the reply Alessandro! Appreciate it.
>> >> >
>> >> > Below is the full request and the error received
>> >> >
>> >> > curl '
>> >> >
>> >> > http://localhost:8081/solr/admin/collections?action=
>> >> SPLITSHARD&collection=dev-transactions&shard=shard1
>> >> > '
>> >> >
>> >> > {
>> >> >
>> >> >   "responseHeader":{
>> >> >
>> >> >     "status":500,
>> >> >
>> >> >     "QTime":7920},
>> >> >
>> >> >   "success":{
>> >> >
>> >> >     "solr-1:8081_solr":{
>> >> >
>> >> >       "responseHeader":{
>> >> >
>> >> >         "status":0,
>> >> >
>> >> >         "QTime":1190},
>> >> >
>> >> >       "core":"dev-transactions_shard1_0_replica_n3"},
>> >> >
>> >> >     "solr-1:8081_solr":{
>> >> >
>> >> >       "responseHeader":{
>> >> >
>> >> >         "status":0,
>> >> >
>> >> >         "QTime":1047},
>> >> >
>> >> >       "core":"dev-transactions_shard1_1_replica_n4"},
>> >> >
>> >> >     "solr-1:8081_solr":{
>> >> >
>> >> >       "responseHeader":{
>> >> >
>> >> >         "status":0,
>> >> >
>> >> >         "QTime":6}},
>> >> >
>> >> >     "solr-1:8081_solr":{
>> >> >
>> >> >       "responseHeader":{
>> >> >
>> >> >         "status":0,
>> >> >
>> >> >         "QTime":1009}}},
>> >> >
>> >> >   "failure":{
>> >> >
>> >> >
>> >> > "solr-1:8081_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$
>> >> RemoteSolrException:Error
>> >> > from server at http://solr-1:8081/solr:
>> >> > java.lang.IllegalArgumentException:
>> >> > Cannot merge a segment that has been created with major version 6 into
>> >> this
>> >> > index which has been created by major version 7"},
>> >> >
>> >> >   "Operation splitshard caused
>> >> >
>> >> > exception:":"org.apache.solr.common.SolrException:org.
>> >> apache.solr.common.SolrException:
>> >> > SPLITSHARD failed to invoke SPLIT core admin command",
>> >> >
>> >> >   "exception":{
>> >> >
>> >> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
>> >> >
>> >> >     "rspCode":500},
>> >> >
>> >> >   "error":{
>> >> >
>> >> >     "metadata":[
>> >> >
>> >> >       "error-class","org.apache.solr.common.SolrException",
>> >> >
>> >> >       "root-error-class","org.apache.solr.common.SolrException"],
>> >> >
>> >> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
>> >> >
>> >> >     "trace":"org.apache.solr.common.SolrException: SPLITSHARD failed
>> to
>> >> > invoke SPLIT core admin command\n\tat
>> >> >
>> >> > org.apache.solr.client.solrj.SolrResponse.getException(
>> >> SolrResponse.java:53)\n\tat
>> >> >
>> >> > org.apache.solr.handler.admin.CollectionsHandler.invokeAction(
>> >> CollectionsHandler.java:258)\n\tat
>> >> >
>> >> > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
>> >> CollectionsHandler.java:230)\n\tat
>> >> >
>> >> > org.apache.solr.handler.RequestHandlerBase.handleRequest(
>> >> RequestHandlerBase.java:195)\n\tat
>> >> >
>> >> > org.apache.solr.servlet.HttpSolrCall.handleAdmin(
>> >> HttpSolrCall.java:736)\n\tat
>> >> >
>> >> > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(
>> >> HttpSolrCall.java:717)\n\tat
>> >> > org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat
>> >> >
>> >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>> >> SolrDispatchFilter.java:384)\n\tat
>> >> >
>> >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>> >> SolrDispatchFilter.java:330)\n\tat
>> >> >
>> >> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.
>> >> doFilter(ServletHandler.java:1629)\n\tat
>> >> >
>> >> > org.eclipse.jetty.servlet.ServletHandler.doHandle(
>> >> ServletHandler.java:533)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
>> >> ScopedHandler.java:143)\n\tat
>> >> >
>> >> > org.eclipse.jetty.security.SecurityHandler.handle(
>> >> SecurityHandler.java:548)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>> >> HandlerWrapper.java:132)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.ScopedHandler.
>> >> nextHandle(ScopedHandler.java:190)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.session.SessionHandler.
>> >> doHandle(SessionHandler.java:1595)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.ScopedHandler.
>> >> nextHandle(ScopedHandler.java:188)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.ContextHandler.
>> >> doHandle(ContextHandler.java:1253)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.ScopedHandler.
>> >> nextScope(ScopedHandler.java:168)\n\tat
>> >> >
>> >> > org.eclipse.jetty.servlet.ServletHandler.doScope(
>> >> ServletHandler.java:473)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.session.SessionHandler.
>> >> doScope(SessionHandler.java:1564)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.ScopedHandler.
>> >> nextScope(ScopedHandler.java:166)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.ContextHandler.
>> >> doScope(ContextHandler.java:1155)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
>> >> ScopedHandler.java:141)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
>> >> ContextHandlerCollection.java:219)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.HandlerCollection.
>> >> handle(HandlerCollection.java:126)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>> >> HandlerWrapper.java:132)\n\tat
>> >> >
>> >> > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(
>> >> RewriteHandler.java:335)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>> >> HandlerWrapper.java:132)\n\tat
>> >> > org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat
>> >> >
>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat
>> >> >
>> >> > org.eclipse.jetty.server.HttpConnection.onFillable(
>> >> HttpConnection.java:256)\n\tat
>> >> > org.eclipse.jetty.io
>> >> >
>> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\
>> >> n\tat
>> >> > org.eclipse.jetty.io
>> .FillInterest.fillable(FillInterest.java:102)\n\tat
>> >> > org.eclipse.jetty.io
>> .ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\
>> >> tat
>> >> >
>> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
>> >> doProduce(EatWhatYouKill.java:247)\n\tat
>> >> >
>> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
>> >> produce(EatWhatYouKill.java:140)\n\tat
>> >> >
>> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
>> >> EatWhatYouKill.java:131)\n\tat
>> >> >
>> >> >
>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(
>> >> ReservedThreadExecutor.java:382)\n\tat
>> >> >
>> >> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>> >> QueuedThreadPool.java:708)\n\tat
>> >> >
>> >> > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
>> >> QueuedThreadPool.java:626)\n\tat
>> >> > java.lang.Thread.run(Thread.java:748)\n",
>> >> >
>> >> >     "code":500}}
>> >> >
>> >> >
>> >> > On Wed, Jun 20, 2018 at 2:25 AM, Alessandro Benedetti <
>> >> > [hidden email]>
>> >> > wrote:
>> >> >
>> >> > > Hi,
>> >> > > in the first place, why do you want to split 2 Gb indexes ?
>> >> > > Nowadays is a fairly small index.
>> >> > >
>> >> > > Secondly what you reported is incomplete.
>> >> > > I would expect a Caused By section in the stacktrace.
>> >> > >
>> >> > > This are generic recommendations, always spend time in analysing the
>> >> > > problem
>> >> > > you had scrupulously.
>> >> > > - SolrCloud problems often involve more than one node. Be sure to
>> check
>> >> > the
>> >> > > logs of all the nodes possibly involved.
>> >> > > - Report the full stack trace to the community
>> >> > > - Report your full request which provoked the exception
>> >> > >
>> >> > > Help is much easier this way :)
>> >> > >
>> >> > > Regards
>> >> > >
>> >> > >
>> >> > >
>> >> > >
>> >> > > -----
>> >> > > ---------------
>> >> > > Alessandro Benedetti
>> >> > > Search Consultant, R&D Software Engineer, Director
>> >> > > Sease Ltd. - www.sease.io
>> >> > > --
>> >> > > Sent from:
>> http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>> >> > >
>> >> >
>> >>
>>
Reply | Threaded
Open this post in threaded view
|

Re: How to split index more than 2GB in size

Sushant Vengurlekar
Thank you for the detailed response Eric. Very much appreciated. The reason
I am looking into splitting the index into two is because it’s much faster
to search across a smaller index than a larger one.

On Wed, Jun 20, 2018 at 10:46 AM Erick Erickson <[hidden email]>
wrote:

> You still haven't answered _why_ you think splitting even a 20G index
> is desirable. We regularly see 200G+ indexes per replica in the field,
> so what's the point? Have you measured different setups to see if it's
> a good idea? A 200G index needs some beefy hardware admittedly....
>
> If you have adequate response times with a 20G index and need to
> increase the QPS rate, just add more replicas. Having more than one
> shard inevitably adds overhead which may (or may not) be made up for
> by parallelizing some of the work. It's nearly always better to use
> only one shard if it meets your response time requirements.
>
> Best,
> Erick
>
> On Wed, Jun 20, 2018 at 10:39 AM, Sushant Vengurlekar
> <[hidden email]> wrote:
> > The index size is small because this is my local development copy.  The
> > production index is more than 20GB. So I am working on getting the index
> > split and replicated on different nodes. Our current instance on prod is
> > single instance solr 6 which we are working on moving towards solrcloud 7
> >
> > On Wed, Jun 20, 2018 at 10:30 AM Erick Erickson <[hidden email]
> >
> > wrote:
> >
> >> Use the indexupgrader tool or optimize your index before using
> splitshard.
> >>
> >> Since this is a small index (< 5G), optimizing will not create an
> >> overly-large segment, so that pitfall is avoided.
> >>
> >> You haven't yet explained why you think splitting the index would be
> >> beneficial. Splitting an index this small is unlikely to improve query
> >> performance appreciably. This feels a lot like an "XY" problem, you're
> >> asking how to do X thinking it will solve Y but not telling us what Y
> >> is.
> >>
> >> Best,
> >> Erick
> >>
> >> On Wed, Jun 20, 2018 at 9:40 AM, Sushant Vengurlekar
> >> <[hidden email]> wrote:
> >> > How can I resolve this error?
> >> >
> >> > On Wed, Jun 20, 2018 at 9:11 AM, Alexandre Rafalovitch <
> >> [hidden email]>
> >> > wrote:
> >> >
> >> >> This seems more related to an old index upgraded to latest Solr
> rather
> >> than
> >> >> the split itself.
> >> >>
> >> >> Regards,
> >> >>     Alex
> >> >>
> >> >> On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, <
> >> >> [hidden email]> wrote:
> >> >>
> >> >> > Thanks for the reply Alessandro! Appreciate it.
> >> >> >
> >> >> > Below is the full request and the error received
> >> >> >
> >> >> > curl '
> >> >> >
> >> >> > http://localhost:8081/solr/admin/collections?action=
> >> >> SPLITSHARD&collection=dev-transactions&shard=shard1
> >> >> > '
> >> >> >
> >> >> > {
> >> >> >
> >> >> >   "responseHeader":{
> >> >> >
> >> >> >     "status":500,
> >> >> >
> >> >> >     "QTime":7920},
> >> >> >
> >> >> >   "success":{
> >> >> >
> >> >> >     "solr-1:8081_solr":{
> >> >> >
> >> >> >       "responseHeader":{
> >> >> >
> >> >> >         "status":0,
> >> >> >
> >> >> >         "QTime":1190},
> >> >> >
> >> >> >       "core":"dev-transactions_shard1_0_replica_n3"},
> >> >> >
> >> >> >     "solr-1:8081_solr":{
> >> >> >
> >> >> >       "responseHeader":{
> >> >> >
> >> >> >         "status":0,
> >> >> >
> >> >> >         "QTime":1047},
> >> >> >
> >> >> >       "core":"dev-transactions_shard1_1_replica_n4"},
> >> >> >
> >> >> >     "solr-1:8081_solr":{
> >> >> >
> >> >> >       "responseHeader":{
> >> >> >
> >> >> >         "status":0,
> >> >> >
> >> >> >         "QTime":6}},
> >> >> >
> >> >> >     "solr-1:8081_solr":{
> >> >> >
> >> >> >       "responseHeader":{
> >> >> >
> >> >> >         "status":0,
> >> >> >
> >> >> >         "QTime":1009}}},
> >> >> >
> >> >> >   "failure":{
> >> >> >
> >> >> >
> >> >> >
> "solr-1:8081_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$
> >> >> RemoteSolrException:Error
> >> >> > from server at http://solr-1:8081/solr:
> >> >> > java.lang.IllegalArgumentException:
> >> >> > Cannot merge a segment that has been created with major version 6
> into
> >> >> this
> >> >> > index which has been created by major version 7"},
> >> >> >
> >> >> >   "Operation splitshard caused
> >> >> >
> >> >> > exception:":"org.apache.solr.common.SolrException:org.
> >> >> apache.solr.common.SolrException:
> >> >> > SPLITSHARD failed to invoke SPLIT core admin command",
> >> >> >
> >> >> >   "exception":{
> >> >> >
> >> >> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
> >> >> >
> >> >> >     "rspCode":500},
> >> >> >
> >> >> >   "error":{
> >> >> >
> >> >> >     "metadata":[
> >> >> >
> >> >> >       "error-class","org.apache.solr.common.SolrException",
> >> >> >
> >> >> >       "root-error-class","org.apache.solr.common.SolrException"],
> >> >> >
> >> >> >     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
> >> >> >
> >> >> >     "trace":"org.apache.solr.common.SolrException: SPLITSHARD
> failed
> >> to
> >> >> > invoke SPLIT core admin command\n\tat
> >> >> >
> >> >> > org.apache.solr.client.solrj.SolrResponse.getException(
> >> >> SolrResponse.java:53)\n\tat
> >> >> >
> >> >> > org.apache.solr.handler.admin.CollectionsHandler.invokeAction(
> >> >> CollectionsHandler.java:258)\n\tat
> >> >> >
> >> >> > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
> >> >> CollectionsHandler.java:230)\n\tat
> >> >> >
> >> >> > org.apache.solr.handler.RequestHandlerBase.handleRequest(
> >> >> RequestHandlerBase.java:195)\n\tat
> >> >> >
> >> >> > org.apache.solr.servlet.HttpSolrCall.handleAdmin(
> >> >> HttpSolrCall.java:736)\n\tat
> >> >> >
> >> >> > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(
> >> >> HttpSolrCall.java:717)\n\tat
> >> >> >
> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat
> >> >> >
> >> >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> >> >> SolrDispatchFilter.java:384)\n\tat
> >> >> >
> >> >> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> >> >> SolrDispatchFilter.java:330)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> >> >> doFilter(ServletHandler.java:1629)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.servlet.ServletHandler.doHandle(
> >> >> ServletHandler.java:533)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> >> >> ScopedHandler.java:143)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.security.SecurityHandler.handle(
> >> >> SecurityHandler.java:548)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> >> >> HandlerWrapper.java:132)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.ScopedHandler.
> >> >> nextHandle(ScopedHandler.java:190)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.session.SessionHandler.
> >> >> doHandle(SessionHandler.java:1595)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.ScopedHandler.
> >> >> nextHandle(ScopedHandler.java:188)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.ContextHandler.
> >> >> doHandle(ContextHandler.java:1253)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.ScopedHandler.
> >> >> nextScope(ScopedHandler.java:168)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.servlet.ServletHandler.doScope(
> >> >> ServletHandler.java:473)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.session.SessionHandler.
> >> >> doScope(SessionHandler.java:1564)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.ScopedHandler.
> >> >> nextScope(ScopedHandler.java:166)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.ContextHandler.
> >> >> doScope(ContextHandler.java:1155)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.ScopedHandler.handle(
> >> >> ScopedHandler.java:141)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
> >> >> ContextHandlerCollection.java:219)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.HandlerCollection.
> >> >> handle(HandlerCollection.java:126)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> >> >> HandlerWrapper.java:132)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(
> >> >> RewriteHandler.java:335)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.handler.HandlerWrapper.handle(
> >> >> HandlerWrapper.java:132)\n\tat
> >> >> > org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat
> >> >> >
> >> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.server.HttpConnection.onFillable(
> >> >> HttpConnection.java:256)\n\tat
> >> >> > org.eclipse.jetty.io
> >> >> >
> >> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\
> >> >> n\tat
> >> >> > org.eclipse.jetty.io
> >> .FillInterest.fillable(FillInterest.java:102)\n\tat
> >> >> > org.eclipse.jetty.io
> >> .ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\
> >> >> tat
> >> >> >
> >> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
> >> >> doProduce(EatWhatYouKill.java:247)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
> >> >> produce(EatWhatYouKill.java:140)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
> >> >> EatWhatYouKill.java:131)\n\tat
> >> >> >
> >> >> >
> >> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(
> >> >> ReservedThreadExecutor.java:382)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
> >> >> QueuedThreadPool.java:708)\n\tat
> >> >> >
> >> >> > org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
> >> >> QueuedThreadPool.java:626)\n\tat
> >> >> > java.lang.Thread.run(Thread.java:748)\n",
> >> >> >
> >> >> >     "code":500}}
> >> >> >
> >> >> >
> >> >> > On Wed, Jun 20, 2018 at 2:25 AM, Alessandro Benedetti <
> >> >> > [hidden email]>
> >> >> > wrote:
> >> >> >
> >> >> > > Hi,
> >> >> > > in the first place, why do you want to split 2 Gb indexes ?
> >> >> > > Nowadays is a fairly small index.
> >> >> > >
> >> >> > > Secondly what you reported is incomplete.
> >> >> > > I would expect a Caused By section in the stacktrace.
> >> >> > >
> >> >> > > This are generic recommendations, always spend time in analysing
> the
> >> >> > > problem
> >> >> > > you had scrupulously.
> >> >> > > - SolrCloud problems often involve more than one node. Be sure to
> >> check
> >> >> > the
> >> >> > > logs of all the nodes possibly involved.
> >> >> > > - Report the full stack trace to the community
> >> >> > > - Report your full request which provoked the exception
> >> >> > >
> >> >> > > Help is much easier this way :)
> >> >> > >
> >> >> > > Regards
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > -----
> >> >> > > ---------------
> >> >> > > Alessandro Benedetti
> >> >> > > Search Consultant, R&D Software Engineer, Director
> >> >> > > Sease Ltd. - www.sease.io
> >> >> > > --
> >> >> > > Sent from:
> >> http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
> >> >> > >
> >> >> >
> >> >>
> >>
>
Reply | Threaded
Open this post in threaded view
|

Re: How to split index more than 2GB in size

Michael Kuhlmann-5
Hi Sushant,

while this is true in general, it won't hold here. If you split your
index, searching on each splitted shard might be a bit faster, but
you'll increase search time much more because Solr needs to send your
search queries to all shards and then combine the results. So instead of
having one medium fast search request, you'll have several fast requests
plus the aggregation step.

Erick is totally right, splitting an index of that size has no
performance benefit. Sharding is not a technique to improve performance,
it's a technique to be able to handle indexes of hundreds of megabytes
in size, which won't fit into an individual machine.

Best,
Michael


Am 20.06.2018 um 19:58 schrieb Sushant Vengurlekar:

> Thank you for the detailed response Eric. Very much appreciated. The reason
> I am looking into splitting the index into two is because it’s much faster
> to search across a smaller index than a larger one.
>
> On Wed, Jun 20, 2018 at 10:46 AM Erick Erickson <[hidden email]>
> wrote:
>
>> You still haven't answered _why_ you think splitting even a 20G index
>> is desirable. We regularly see 200G+ indexes per replica in the field,
>> so what's the point? Have you measured different setups to see if it's
>> a good idea? A 200G index needs some beefy hardware admittedly....
>>
>> If you have adequate response times with a 20G index and need to
>> increase the QPS rate, just add more replicas. Having more than one
>> shard inevitably adds overhead which may (or may not) be made up for
>> by parallelizing some of the work. It's nearly always better to use
>> only one shard if it meets your response time requirements.
>>
>> Best,
>> Erick
>>
>> On Wed, Jun 20, 2018 at 10:39 AM, Sushant Vengurlekar
>> <[hidden email]> wrote:
>>> The index size is small because this is my local development copy.  The
>>> production index is more than 20GB. So I am working on getting the index
>>> split and replicated on different nodes. Our current instance on prod is
>>> single instance solr 6 which we are working on moving towards solrcloud 7
>>>
>>> On Wed, Jun 20, 2018 at 10:30 AM Erick Erickson <[hidden email]
>>>
>>> wrote:
>>>
>>>> Use the indexupgrader tool or optimize your index before using
>> splitshard.
>>>>
>>>> Since this is a small index (< 5G), optimizing will not create an
>>>> overly-large segment, so that pitfall is avoided.
>>>>
>>>> You haven't yet explained why you think splitting the index would be
>>>> beneficial. Splitting an index this small is unlikely to improve query
>>>> performance appreciably. This feels a lot like an "XY" problem, you're
>>>> asking how to do X thinking it will solve Y but not telling us what Y
>>>> is.
>>>>
>>>> Best,
>>>> Erick
>>>>
>>>> On Wed, Jun 20, 2018 at 9:40 AM, Sushant Vengurlekar
>>>> <[hidden email]> wrote:
>>>>> How can I resolve this error?
>>>>>
>>>>> On Wed, Jun 20, 2018 at 9:11 AM, Alexandre Rafalovitch <
>>>> [hidden email]>
>>>>> wrote:
>>>>>
>>>>>> This seems more related to an old index upgraded to latest Solr
>> rather
>>>> than
>>>>>> the split itself.
>>>>>>
>>>>>> Regards,
>>>>>>     Alex
>>>>>>
>>>>>> On Wed, Jun 20, 2018, 12:07 PM Sushant Vengurlekar, <
>>>>>> [hidden email]> wrote:
>>>>>>
>>>>>>> Thanks for the reply Alessandro! Appreciate it.
>>>>>>>
>>>>>>> Below is the full request and the error received
>>>>>>>
>>>>>>> curl '
>>>>>>>
>>>>>>> http://localhost:8081/solr/admin/collections?action=
>>>>>> SPLITSHARD&collection=dev-transactions&shard=shard1
>>>>>>> '
>>>>>>>
>>>>>>> {
>>>>>>>
>>>>>>>   "responseHeader":{
>>>>>>>
>>>>>>>     "status":500,
>>>>>>>
>>>>>>>     "QTime":7920},
>>>>>>>
>>>>>>>   "success":{
>>>>>>>
>>>>>>>     "solr-1:8081_solr":{
>>>>>>>
>>>>>>>       "responseHeader":{
>>>>>>>
>>>>>>>         "status":0,
>>>>>>>
>>>>>>>         "QTime":1190},
>>>>>>>
>>>>>>>       "core":"dev-transactions_shard1_0_replica_n3"},
>>>>>>>
>>>>>>>     "solr-1:8081_solr":{
>>>>>>>
>>>>>>>       "responseHeader":{
>>>>>>>
>>>>>>>         "status":0,
>>>>>>>
>>>>>>>         "QTime":1047},
>>>>>>>
>>>>>>>       "core":"dev-transactions_shard1_1_replica_n4"},
>>>>>>>
>>>>>>>     "solr-1:8081_solr":{
>>>>>>>
>>>>>>>       "responseHeader":{
>>>>>>>
>>>>>>>         "status":0,
>>>>>>>
>>>>>>>         "QTime":6}},
>>>>>>>
>>>>>>>     "solr-1:8081_solr":{
>>>>>>>
>>>>>>>       "responseHeader":{
>>>>>>>
>>>>>>>         "status":0,
>>>>>>>
>>>>>>>         "QTime":1009}}},
>>>>>>>
>>>>>>>   "failure":{
>>>>>>>
>>>>>>>
>>>>>>>
>> "solr-1:8081_solr":"org.apache.solr.client.solrj.impl.HttpSolrClient$
>>>>>> RemoteSolrException:Error
>>>>>>> from server at http://solr-1:8081/solr:
>>>>>>> java.lang.IllegalArgumentException:
>>>>>>> Cannot merge a segment that has been created with major version 6
>> into
>>>>>> this
>>>>>>> index which has been created by major version 7"},
>>>>>>>
>>>>>>>   "Operation splitshard caused
>>>>>>>
>>>>>>> exception:":"org.apache.solr.common.SolrException:org.
>>>>>> apache.solr.common.SolrException:
>>>>>>> SPLITSHARD failed to invoke SPLIT core admin command",
>>>>>>>
>>>>>>>   "exception":{
>>>>>>>
>>>>>>>     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
>>>>>>>
>>>>>>>     "rspCode":500},
>>>>>>>
>>>>>>>   "error":{
>>>>>>>
>>>>>>>     "metadata":[
>>>>>>>
>>>>>>>       "error-class","org.apache.solr.common.SolrException",
>>>>>>>
>>>>>>>       "root-error-class","org.apache.solr.common.SolrException"],
>>>>>>>
>>>>>>>     "msg":"SPLITSHARD failed to invoke SPLIT core admin command",
>>>>>>>
>>>>>>>     "trace":"org.apache.solr.common.SolrException: SPLITSHARD
>> failed
>>>> to
>>>>>>> invoke SPLIT core admin command\n\tat
>>>>>>>
>>>>>>> org.apache.solr.client.solrj.SolrResponse.getException(
>>>>>> SolrResponse.java:53)\n\tat
>>>>>>>
>>>>>>> org.apache.solr.handler.admin.CollectionsHandler.invokeAction(
>>>>>> CollectionsHandler.java:258)\n\tat
>>>>>>>
>>>>>>> org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
>>>>>> CollectionsHandler.java:230)\n\tat
>>>>>>>
>>>>>>> org.apache.solr.handler.RequestHandlerBase.handleRequest(
>>>>>> RequestHandlerBase.java:195)\n\tat
>>>>>>>
>>>>>>> org.apache.solr.servlet.HttpSolrCall.handleAdmin(
>>>>>> HttpSolrCall.java:736)\n\tat
>>>>>>>
>>>>>>> org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(
>>>>>> HttpSolrCall.java:717)\n\tat
>>>>>>>
>> org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498)\n\tat
>>>>>>>
>>>>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>>>>>> SolrDispatchFilter.java:384)\n\tat
>>>>>>>
>>>>>>> org.apache.solr.servlet.SolrDispatchFilter.doFilter(
>>>>>> SolrDispatchFilter.java:330)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.servlet.ServletHandler$CachedChain.
>>>>>> doFilter(ServletHandler.java:1629)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.servlet.ServletHandler.doHandle(
>>>>>> ServletHandler.java:533)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(
>>>>>> ScopedHandler.java:143)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.security.SecurityHandler.handle(
>>>>>> SecurityHandler.java:548)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>>>>>> HandlerWrapper.java:132)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.
>>>>>> nextHandle(ScopedHandler.java:190)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.session.SessionHandler.
>>>>>> doHandle(SessionHandler.java:1595)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.
>>>>>> nextHandle(ScopedHandler.java:188)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.ContextHandler.
>>>>>> doHandle(ContextHandler.java:1253)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.
>>>>>> nextScope(ScopedHandler.java:168)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.servlet.ServletHandler.doScope(
>>>>>> ServletHandler.java:473)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.session.SessionHandler.
>>>>>> doScope(SessionHandler.java:1564)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.
>>>>>> nextScope(ScopedHandler.java:166)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.ContextHandler.
>>>>>> doScope(ContextHandler.java:1155)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.ScopedHandler.handle(
>>>>>> ScopedHandler.java:141)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(
>>>>>> ContextHandlerCollection.java:219)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.HandlerCollection.
>>>>>> handle(HandlerCollection.java:126)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>>>>>> HandlerWrapper.java:132)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(
>>>>>> RewriteHandler.java:335)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.handler.HandlerWrapper.handle(
>>>>>> HandlerWrapper.java:132)\n\tat
>>>>>>> org.eclipse.jetty.server.Server.handle(Server.java:530)\n\tat
>>>>>>>
>>>> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.server.HttpConnection.onFillable(
>>>>>> HttpConnection.java:256)\n\tat
>>>>>>> org.eclipse.jetty.io
>>>>>>>
>>>> .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)\
>>>>>> n\tat
>>>>>>> org.eclipse.jetty.io
>>>> .FillInterest.fillable(FillInterest.java:102)\n\tat
>>>>>>> org.eclipse.jetty.io
>>>> .ChannelEndPoint$2.run(ChannelEndPoint.java:124)\n\
>>>>>> tat
>>>>>>>
>>>>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
>>>>>> doProduce(EatWhatYouKill.java:247)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.
>>>>>> produce(EatWhatYouKill.java:140)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(
>>>>>> EatWhatYouKill.java:131)\n\tat
>>>>>>>
>>>>>>>
>>>> org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(
>>>>>> ReservedThreadExecutor.java:382)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
>>>>>> QueuedThreadPool.java:708)\n\tat
>>>>>>>
>>>>>>> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
>>>>>> QueuedThreadPool.java:626)\n\tat
>>>>>>> java.lang.Thread.run(Thread.java:748)\n",
>>>>>>>
>>>>>>>     "code":500}}
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 20, 2018 at 2:25 AM, Alessandro Benedetti <
>>>>>>> [hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> in the first place, why do you want to split 2 Gb indexes ?
>>>>>>>> Nowadays is a fairly small index.
>>>>>>>>
>>>>>>>> Secondly what you reported is incomplete.
>>>>>>>> I would expect a Caused By section in the stacktrace.
>>>>>>>>
>>>>>>>> This are generic recommendations, always spend time in analysing
>> the
>>>>>>>> problem
>>>>>>>> you had scrupulously.
>>>>>>>> - SolrCloud problems often involve more than one node. Be sure to
>>>> check
>>>>>>> the
>>>>>>>> logs of all the nodes possibly involved.
>>>>>>>> - Report the full stack trace to the community
>>>>>>>> - Report your full request which provoked the exception
>>>>>>>>
>>>>>>>> Help is much easier this way :)
>>>>>>>>
>>>>>>>> Regards
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> -----
>>>>>>>> ---------------
>>>>>>>> Alessandro Benedetti
>>>>>>>> Search Consultant, R&D Software Engineer, Director
>>>>>>>> Sease Ltd. - www.sease.io
>>>>>>>> --
>>>>>>>> Sent from:
>>>> http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>
>