[jira] Created: (LUCENE-2128) Further parallelizaton of ParallelMultiSearcher

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (LUCENE-2128) Further parallelizaton of ParallelMultiSearcher

JIRA jira@apache.org
Further parallelizaton of ParallelMultiSearcher
-----------------------------------------------

                 Key: LUCENE-2128
                 URL: https://issues.apache.org/jira/browse/LUCENE-2128
             Project: Lucene - Java
          Issue Type: Improvement
          Components: Search
    Affects Versions: 3.0
            Reporter: Israel Tsadok
            Priority: Minor


When calling {{search(Query, Filter, int)}} on a ParallelMultiSearcher, the {{createWeights}} function of MultiSearcher is called, and sequentially calls {{docFreqs()}} on every sub-searcher. This can take a significant amount of time when there are lots of remote sub-searchers.



--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2128) Further parallelizaton of ParallelMultiSearcher

JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Israel Tsadok updated LUCENE-2128:
----------------------------------

    Attachment: LUCENE-2128.patch

I extracted step 3 of createWeights into its own method, and added an overriding method on ParallelMultiSearcher.

This is untested code, sorry.

> Further parallelizaton of ParallelMultiSearcher
> -----------------------------------------------
>
>                 Key: LUCENE-2128
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2128
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 3.0
>            Reporter: Israel Tsadok
>            Priority: Minor
>         Attachments: LUCENE-2128.patch
>
>
> When calling {{search(Query, Filter, int)}} on a ParallelMultiSearcher, the {{createWeights}} function of MultiSearcher is called, and sequentially calls {{docFreqs()}} on every sub-searcher. This can take a significant amount of time when there are lots of remote sub-searchers.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2128) Further parallelizaton of ParallelMultiSearcher

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786934#action_12786934 ]

Simon Willnauer commented on LUCENE-2128:
-----------------------------------------

bq. This is untested code, sorry.
And it does not compile either.

Can you revise your patch?



> Further parallelizaton of ParallelMultiSearcher
> -----------------------------------------------
>
>                 Key: LUCENE-2128
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2128
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 3.0
>            Reporter: Israel Tsadok
>            Priority: Minor
>         Attachments: LUCENE-2128.patch
>
>
> When calling {{search(Query, Filter, int)}} on a ParallelMultiSearcher, the {{createWeights}} function of MultiSearcher is called, and sequentially calls {{docFreqs()}} on every sub-searcher. This can take a significant amount of time when there are lots of remote sub-searchers.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2128) Further parallelizaton of ParallelMultiSearcher

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Israel Tsadok updated LUCENE-2128:
----------------------------------

    Attachment: LUCENE-2128.patch

Attaching a patch that actually compiles.

> Further parallelizaton of ParallelMultiSearcher
> -----------------------------------------------
>
>                 Key: LUCENE-2128
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2128
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 3.0
>            Reporter: Israel Tsadok
>            Priority: Minor
>         Attachments: LUCENE-2128.patch, LUCENE-2128.patch
>
>
> When calling {{search(Query, Filter, int)}} on a ParallelMultiSearcher, the {{createWeights}} function of MultiSearcher is called, and sequentially calls {{docFreqs()}} on every sub-searcher. This can take a significant amount of time when there are lots of remote sub-searchers.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2128) Further parallelizaton of ParallelMultiSearcher

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-2128:
------------------------------------

    Attachment: LUCENE-2128.patch

Updated patch with cleaner structure, extracted classes and added testcase for it.

> Further parallelizaton of ParallelMultiSearcher
> -----------------------------------------------
>
>                 Key: LUCENE-2128
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2128
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 3.0
>            Reporter: Israel Tsadok
>            Priority: Minor
>         Attachments: LUCENE-2128.patch, LUCENE-2128.patch, LUCENE-2128.patch
>
>
> When calling {{search(Query, Filter, int)}} on a ParallelMultiSearcher, the {{createWeights}} function of MultiSearcher is called, and sequentially calls {{docFreqs()}} on every sub-searcher. This can take a significant amount of time when there are lots of remote sub-searchers.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Assigned: (LUCENE-2128) Further parallelizaton of ParallelMultiSearcher

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler reassigned LUCENE-2128:
-------------------------------------

    Assignee: Uwe Schindler

> Further parallelizaton of ParallelMultiSearcher
> -----------------------------------------------
>
>                 Key: LUCENE-2128
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2128
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 3.0
>            Reporter: Israel Tsadok
>            Assignee: Uwe Schindler
>            Priority: Minor
>         Attachments: LUCENE-2128.patch, LUCENE-2128.patch, LUCENE-2128.patch
>
>
> When calling {{search(Query, Filter, int)}} on a ParallelMultiSearcher, the {{createWeights}} function of MultiSearcher is called, and sequentially calls {{docFreqs()}} on every sub-searcher. This can take a significant amount of time when there are lots of remote sub-searchers.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2128) Further parallelizaton of ParallelMultiSearcher

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-2128:
------------------------------------

    Attachment: LUCENE-2128.patch

updated patch - createDocFrequencyMap change from protected to package private.

> Further parallelizaton of ParallelMultiSearcher
> -----------------------------------------------
>
>                 Key: LUCENE-2128
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2128
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 3.0
>            Reporter: Israel Tsadok
>            Assignee: Uwe Schindler
>            Priority: Minor
>         Attachments: LUCENE-2128.patch, LUCENE-2128.patch, LUCENE-2128.patch, LUCENE-2128.patch
>
>
> When calling {{search(Query, Filter, int)}} on a ParallelMultiSearcher, the {{createWeights}} function of MultiSearcher is called, and sequentially calls {{docFreqs()}} on every sub-searcher. This can take a significant amount of time when there are lots of remote sub-searchers.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2128) Further parallelizaton of ParallelMultiSearcher

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12787773#action_12787773 ]

Uwe Schindler commented on LUCENE-2128:
---------------------------------------

I'll commit shortly.

> Further parallelizaton of ParallelMultiSearcher
> -----------------------------------------------
>
>                 Key: LUCENE-2128
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2128
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 3.0
>            Reporter: Israel Tsadok
>            Assignee: Uwe Schindler
>            Priority: Minor
>         Attachments: LUCENE-2128.patch, LUCENE-2128.patch, LUCENE-2128.patch, LUCENE-2128.patch
>
>
> When calling {{search(Query, Filter, int)}} on a ParallelMultiSearcher, the {{createWeights}} function of MultiSearcher is called, and sequentially calls {{docFreqs()}} on every sub-searcher. This can take a significant amount of time when there are lots of remote sub-searchers.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Resolved: (LUCENE-2128) Further parallelizaton of ParallelMultiSearcher

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-2128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler resolved LUCENE-2128.
-----------------------------------

    Resolution: Fixed

Committed revision: 888595

Thanks Simon & Israel

> Further parallelizaton of ParallelMultiSearcher
> -----------------------------------------------
>
>                 Key: LUCENE-2128
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2128
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 3.0
>            Reporter: Israel Tsadok
>            Assignee: Uwe Schindler
>            Priority: Minor
>         Attachments: LUCENE-2128.patch, LUCENE-2128.patch, LUCENE-2128.patch, LUCENE-2128.patch
>
>
> When calling {{search(Query, Filter, int)}} on a ParallelMultiSearcher, the {{createWeights}} function of MultiSearcher is called, and sequentially calls {{docFreqs()}} on every sub-searcher. This can take a significant amount of time when there are lots of remote sub-searchers.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]