[jira] Created: (SOLR-1506) Search multiple cores using MultiReader

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (SOLR-1506) Search multiple cores using MultiReader

Prajeeth Emanuel (Jira)
Search multiple cores using MultiReader
---------------------------------------

                 Key: SOLR-1506
                 URL: https://issues.apache.org/jira/browse/SOLR-1506
             Project: Solr
          Issue Type: Improvement
          Components: search
    Affects Versions: 1.4
            Reporter: Jason Rutherglen
            Priority: Trivial
             Fix For: 1.5


I need to search over multiple cores, and SOLR-1477 is more
complicated than expected, so here we'll create a MultiReader
over the cores to allow searching on them.

Maybe in the future we can add parallel searching however
SOLR-1477, if it gets completed, provides that out of the box.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1506) Search multiple cores using MultiReader

Prajeeth Emanuel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated SOLR-1506:
-----------------------------------

    Attachment: SOLR-1506.patch

Well, it seems to work, though I had to comment out the reader.directory() call in SolrCore.  I'm not sure what to do there yet, but this is good enough for now.  

> Search multiple cores using MultiReader
> ---------------------------------------
>
>                 Key: SOLR-1506
>                 URL: https://issues.apache.org/jira/browse/SOLR-1506
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.4
>            Reporter: Jason Rutherglen
>            Priority: Trivial
>             Fix For: 1.5
>
>         Attachments: SOLR-1506.patch
>
>
> I need to search over multiple cores, and SOLR-1477 is more
> complicated than expected, so here we'll create a MultiReader
> over the cores to allow searching on them.
> Maybe in the future we can add parallel searching however
> SOLR-1477, if it gets completed, provides that out of the box.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1506) Search multiple cores using MultiReader

Prajeeth Emanuel (Jira)
In reply to this post by Prajeeth Emanuel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated SOLR-1506:
-----------------------------------

    Attachment: SOLR-1506.patch

Fixes a bug, added Apache headers

> Search multiple cores using MultiReader
> ---------------------------------------
>
>                 Key: SOLR-1506
>                 URL: https://issues.apache.org/jira/browse/SOLR-1506
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.4
>            Reporter: Jason Rutherglen
>            Priority: Trivial
>             Fix For: 1.5
>
>         Attachments: SOLR-1506.patch, SOLR-1506.patch
>
>
> I need to search over multiple cores, and SOLR-1477 is more
> complicated than expected, so here we'll create a MultiReader
> over the cores to allow searching on them.
> Maybe in the future we can add parallel searching however
> SOLR-1477, if it gets completed, provides that out of the box.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1506) Search multiple cores using MultiReader

Prajeeth Emanuel (Jira)
In reply to this post by Prajeeth Emanuel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12772793#action_12772793 ]

Jason Rutherglen commented on SOLR-1506:
----------------------------------------

There's a bug here with getting the status of multiple cores:

SEVERE: org.apache.solr.common.SolrException: Error handling 'status' action
        at org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:362)
        at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:131)
        at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
        at org.apache.solr.servlet.SolrDispatchFilter.handleAdminRequest(SolrDispatchFilter.java:298)
        at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
        at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:849)
        at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454)
        at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.UnsupportedOperationException: This reader does not support this method.
        at org.apache.lucene.index.IndexReader.directory(IndexReader.java:592)
        at org.apache.solr.search.SolrIndexReader.directory(SolrIndexReader.java:222)
        at org.apache.solr.handler.admin.LukeRequestHandler.getIndexInfo(LukeRequestHandler.java:442)
        at org.apache.solr.handler.admin.CoreAdminHandler.getCoreStatus(CoreAdminHandler.java:449)
        at org.apache.solr.handler.admin.CoreAdminHandler.handleStatusAction(CoreAdminHandler.java:353


> Search multiple cores using MultiReader
> ---------------------------------------
>
>                 Key: SOLR-1506
>                 URL: https://issues.apache.org/jira/browse/SOLR-1506
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.4
>            Reporter: Jason Rutherglen
>            Priority: Trivial
>             Fix For: 1.5
>
>         Attachments: SOLR-1506.patch, SOLR-1506.patch
>
>
> I need to search over multiple cores, and SOLR-1477 is more
> complicated than expected, so here we'll create a MultiReader
> over the cores to allow searching on them.
> Maybe in the future we can add parallel searching however
> SOLR-1477, if it gets completed, provides that out of the box.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1506) Search multiple cores using MultiReader

Prajeeth Emanuel (Jira)
In reply to this post by Prajeeth Emanuel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773104#action_12773104 ]

Jason Rutherglen commented on SOLR-1506:
----------------------------------------

Commit doesn't work because reopen isn't supported by MultiReader.

> Search multiple cores using MultiReader
> ---------------------------------------
>
>                 Key: SOLR-1506
>                 URL: https://issues.apache.org/jira/browse/SOLR-1506
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.4
>            Reporter: Jason Rutherglen
>            Priority: Trivial
>             Fix For: 1.5
>
>         Attachments: SOLR-1506.patch, SOLR-1506.patch
>
>
> I need to search over multiple cores, and SOLR-1477 is more
> complicated than expected, so here we'll create a MultiReader
> over the cores to allow searching on them.
> Maybe in the future we can add parallel searching however
> SOLR-1477, if it gets completed, provides that out of the box.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1506) Search multiple cores using MultiReader

Prajeeth Emanuel (Jira)
In reply to this post by Prajeeth Emanuel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12773213#action_12773213 ]

Koji Sekiguchi commented on SOLR-1506:
--------------------------------------

bq. Commit doesn't work because reopen isn't supported by MultiReader.

Regarding MultiReader and reopen, I've set reopenReaders to false:

{code:title=solrconfig.xml}
<reopenReaders>false</reopenReaders>
  :
<indexReaderFactory name="IndexReaderFactory" class="mypackage.MultiReaderFactory"/>
{code}


> Search multiple cores using MultiReader
> ---------------------------------------
>
>                 Key: SOLR-1506
>                 URL: https://issues.apache.org/jira/browse/SOLR-1506
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.4
>            Reporter: Jason Rutherglen
>            Priority: Trivial
>             Fix For: 1.5
>
>         Attachments: SOLR-1506.patch, SOLR-1506.patch
>
>
> I need to search over multiple cores, and SOLR-1477 is more
> complicated than expected, so here we'll create a MultiReader
> over the cores to allow searching on them.
> Maybe in the future we can add parallel searching however
> SOLR-1477, if it gets completed, provides that out of the box.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (SOLR-1506) Search multiple cores using MultiReader

Prajeeth Emanuel (Jira)
In reply to this post by Prajeeth Emanuel (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jason Rutherglen updated SOLR-1506:
-----------------------------------

    Attachment: SOLR-1506.patch

MultiReader doesn't support reopen with the readOnly parameter.  This patch adds a test case for commit on the proxy, and a workaround (if unsupported is caught, then regular reopen is called).

> Search multiple cores using MultiReader
> ---------------------------------------
>
>                 Key: SOLR-1506
>                 URL: https://issues.apache.org/jira/browse/SOLR-1506
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.4
>            Reporter: Jason Rutherglen
>            Priority: Trivial
>             Fix For: 1.5
>
>         Attachments: SOLR-1506.patch, SOLR-1506.patch, SOLR-1506.patch
>
>
> I need to search over multiple cores, and SOLR-1477 is more
> complicated than expected, so here we'll create a MultiReader
> over the cores to allow searching on them.
> Maybe in the future we can add parallel searching however
> SOLR-1477, if it gets completed, provides that out of the box.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-1506) Search multiple cores using MultiReader

Prajeeth Emanuel (Jira)
In reply to this post by Prajeeth Emanuel (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-1506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789600#action_12789600 ]

Jason Rutherglen commented on SOLR-1506:
----------------------------------------

There's a different bug here, where because CoreContainer loads
the cores sequentially, and MultiCoreReaderFactory looks for all
the cores, when the proxy core isn't last, not all the cores are
searchable, if the proxy is first, an exception is thrown.

The workaround is to place the proxy core last, however that's
not possible when using the core admin HTTP API. Hmm... Not sure
what the best workaround is.

> Search multiple cores using MultiReader
> ---------------------------------------
>
>                 Key: SOLR-1506
>                 URL: https://issues.apache.org/jira/browse/SOLR-1506
>             Project: Solr
>          Issue Type: Improvement
>          Components: search
>    Affects Versions: 1.4
>            Reporter: Jason Rutherglen
>            Priority: Trivial
>             Fix For: 1.5
>
>         Attachments: SOLR-1506.patch, SOLR-1506.patch, SOLR-1506.patch
>
>
> I need to search over multiple cores, and SOLR-1477 is more
> complicated than expected, so here we'll create a MultiReader
> over the cores to allow searching on them.
> Maybe in the future we can add parallel searching however
> SOLR-1477, if it gets completed, provides that out of the box.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.