[jira] Created: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
Complete parallelizaton of ParallelMultiSearcher
------------------------------------------------

                 Key: LUCENE-2041
                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
             Project: Lucene - Java
          Issue Type: Improvement
          Components: Search
    Affects Versions: 2.9
            Reporter: Joey Surls
            Priority: Minor
             Fix For: 3.0


ParallelMultiSearcher is parallel only for the method signatures of 'search'.

Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.



--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joey Surls updated LUCENE-2041:
-------------------------------

    Attachment: LUCENE-2041.patch

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Priority: Minor
>             Fix For: 3.0
>
>         Attachments: LUCENE-2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774492#action_12774492 ]

Simon Willnauer commented on LUCENE-2041:
-----------------------------------------

Hey Joey,
good to see you coming up with patches, nice that you catch up on that TODO. You added support for parallel DocFreq which is good! Did you think about using an interface from the new concurrent utils instead of subclassing thread? I would like to see people using the new features which make code slightly more readable too. You would be able to simply throw the exception in the Callable#call() method and retrieve it once it has terminated.
you could also use a Pooled Executor to run those threads so you don't have to spawn them each time you call docFreq on the multi searcher.

one more thing, your patch is hard to read as you formated the source code. please try to do not format it - that would make it way easier to read and see what has been added  / changed.

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Priority: Minor
>             Fix For: 3.0
>
>         Attachments: LUCENE-2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-2041:
------------------------------------

    Fix Version/s:     (was: 3.0)
                   3.1
         Assignee: Simon Willnauer

moving to 3.1 for now. we will see how this turns out

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Simon Willnauer
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774498#action_12774498 ]

Joey Surls commented on LUCENE-2041:
------------------------------------

I agree with you on the Pooled Executor and concurrent utils, I was actually trying to copy the semantics of the search methods in place so someone could quickly grasp the code.

Ya, I thought that's why the patch was full was from the formatted code. Sorry about that. I'll look for that in the future. I noticed the link in the wiki for the formatting under Helpful Resources doesn't work anymore.

http://wiki.apache.org/lucene-java/HowToContribute




> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Simon Willnauer
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774501#action_12774501 ]

Uwe Schindler commented on LUCENE-2041:
---------------------------------------

bq. Ya, I thought that's why the patch was full was from the formatted code. Sorry about that. I'll look for that in the future. I noticed the link in the wiki for the formatting under Helpful Resources doesn't work anymore.

Fixed that and uploaded a new Eclipse Galieo style also containing our way for Java 5 Generics.

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Simon Willnauer
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-2041:
------------------------------------

    Attachment: LUCENE_2041.patch

I drove this a bit further and refactored the whole code to use the java 1.5 concurrent utils. I seem to be somewhat cleaner with executors / callable and a little refactoring.

simon

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Simon Willnauer
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Assigned: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler reassigned LUCENE-2041:
-------------------------------------

    Assignee: Uwe Schindler  (was: Simon Willnauer)

I take it as communicated.

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774974#action_12774974 ]

Simon Willnauer commented on LUCENE-2041:
-----------------------------------------

thanks uwe!

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774976#action_12774976 ]

Uwe Schindler commented on LUCENE-2041:
---------------------------------------

String.isEmpty() is Java 6 only, changed this to length() == 0.

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2041:
----------------------------------

    Attachment: LUCENE-2041.patch

Updated patch.

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-2041:
------------------------------------

    Attachment: LUCENE-2041.patch

Fixed the Java5 issue.
Damned I should use java 5 on my notebook.

simon

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774982#action_12774982 ]

Simon Willnauer commented on LUCENE-2041:
-----------------------------------------

hehe - my patch was quicker but you comment appears first :)

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774983#action_12774983 ]

Uwe Schindler commented on LUCENE-2041:
---------------------------------------

I also changed some formatting, we require whitespace after control statements like for, while, catch,... and before {

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774987#action_12774987 ]

Uwe Schindler commented on LUCENE-2041:
---------------------------------------

I will also readd support for MaxScore in the fielded case (it is still supported by Searchable.search(SortField))!!!

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2041:
----------------------------------

    Attachment: LUCENE-2041.patch

Advanced patch:
- Removed @SuppressWarnings by duplicating some code, but it is type-safe now
- Added support for maxScore in sorted mode
- Refactoring: made all internal classes static inner classes
- Fix thread safety: hq.setFields() synchronized

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2041:
----------------------------------

    Attachment: LUCENE-2041-refactor.patch

Here another possibility, removing more duplicated code: MultiSearcher contains now the merge Callables as static inner classes. These Callables are used directly without Executor in MultiSearcher, but with Executor in ParallelMultiSearcher.

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Simon Willnauer updated LUCENE-2041:
------------------------------------

    Attachment: LUCENE-2041.patch

Added some more final keywords and generalized HitQueue to PriorityQueue<ScoreDoc>  in search (no sort)

I guess we are set with this patch, good team work Uwe! Lets get this out for heavy committing! :)

I get the impression that we should think about moving stuff like the Function interface out to utils. This could be useful in many situations though. Thoughts?

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-2041:
----------------------------------

    Attachment: LUCENE-2041-refactor.patch

Here my latest patch, will commit soon.

Uwe

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.1
>
>         Attachments: LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-2041) Complete parallelizaton of ParallelMultiSearcher

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-2041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12775927#action_12775927 ]

Uwe Schindler commented on LUCENE-2041:
---------------------------------------

Oh I didn't see your patch. But its not so different, I also did lots of code cleanup.

I also used ReentrantLocks. I added a new DummyLock to utils, that is a replacement for ReentrantLock, but does simply nothing. By this, the synchronized around the HitQueue in MultiSearcher can be ommitted.

> Complete parallelizaton of ParallelMultiSearcher
> ------------------------------------------------
>
>                 Key: LUCENE-2041
>                 URL: https://issues.apache.org/jira/browse/LUCENE-2041
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: Search
>    Affects Versions: 2.9
>            Reporter: Joey Surls
>            Assignee: Uwe Schindler
>            Priority: Minor
>             Fix For: 3.0
>
>         Attachments: LUCENE-2041-refactor.patch, LUCENE-2041-refactor.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE-2041.patch, LUCENE_2041.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> ParallelMultiSearcher is parallel only for the method signatures of 'search'.
> Part of a query process calls the method docFreq(). There was a TODO comment to parallelize this. Parallelizing this method actually increases the performance of a query on multiple indexes, especially remotely.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

12