[jira] Created: (LUCENE-997) Add search timeout support to Lucene

classic Classic list List threaded Threaded
63 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562964#action_12562964 ]

Timo Nentwig commented on LUCENE-997:
-------------------------------------

@Sean: "The biggest changes are that it uses a long instead of an int for the counter in TimerThread" - didn't I use a volatile long already? :)

I hope this will become part of the next release. IMHO this is Priority Major or above...

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562972#action_12562972 ]

Paul Elschot commented on LUCENE-997:
-------------------------------------

In the timeout.patch, instead of this:
{code}
time += resolution;
{code}
I'd rather have this:
{code}
time = System.currentTimeMillis();
{code}
because all of the wait() methods can become unreliable, especially at high loads.

With (or without) this change, 100 msecs or even 200 msecs could be used as the
update frequency instead of the current 10 msecs.

By computing the time out moment up front, one subtraction can be saved at each document collection. Then only TIMER_THREAD.getMilliseconds() method is needed at document collection time, and the getElapsedMilliseconds() method is superfluous.


> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563155#action_12563155 ]

Sean Timm commented on LUCENE-997:
----------------------------------

@Timo: "didn't I use a volatile long already?"  Indeed.  I guess that wasn't a very big change then. ;-)

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Timm updated LUCENE-997:
-----------------------------

    Attachment: timeout.patch

This is a minor update to timeout.patch which fixes the comment about updates to 32-bit-sized variables being atomic and instead talks about volatile longs, as pointed out by Andrzej.  It also computes the time out moment up front to save a subtraction on each document collection as suggested by Paul.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563632#action_12563632 ]

Sean Timm commented on LUCENE-997:
----------------------------------

I could go either way on the System.currentTimeMillis() versus a TimerThread issue.  I agree nanoTime is the correct implementation when using 1.5.

It doesn't seem like on Linux running ntp it matters much either way.  NTP tries to perform smoothing and makes clock changes slowly over a longer period of time when it can rather than have an abrupt change, but YMMV if your system is having clock issues.  On a really overloaded Windows box, the TimerThread implementation will not behave well as demonstrated above.  I can't speak to any other platforms.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12563698#action_12563698 ]

Paul Elschot commented on LUCENE-997:
-------------------------------------

The idea of System.currentTimeMillis() is to guard against misbehaviour of the java wait() method and against unexpected delays because of thread scheduling during the jump back for the loop around the wait() call.
One way to reduce such misbehaviour under heavy load is by increasing the scheduling priority of the timing thread, but I don't think that is necessary.

Also System.currentTimeMillis() is obviously correct, whereas timeout += resolution is never more than an assumption about correct wait() behaviour.

Clock changes by NTP are normally so slow that they don't really matter for query time outs.


> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12564126#action_12564126 ]

Doron Cohen commented on LUCENE-997:
------------------------------------

Sean, can you add a Junit test to timeout.patch?

I think such test should check (1) search correctness (regardless of timeout), (2) expected timeout behavior, and (3) some sanity test with multiple searching threads. This can also serve as an example for using this new collector.

Cheers.
Doron



> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565720#action_12565720 ]

Timo Nentwig commented on LUCENE-997:
-------------------------------------

IIRC I did time=System.currentTimeMillis() first but was surprised how slow this method actally is.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565809#action_12565809 ]

Paul Elschot commented on LUCENE-997:
-------------------------------------

Would that still be a problem with a 200ms resolution?

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Timm updated LUCENE-997:
-----------------------------

    Attachment: timeout.patch

Patch adds JUnit test cases as suggested by Doren.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Issue Comment Edited: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565835#action_12565835 ]

timmsc edited comment on LUCENE-997 at 2/5/08 9:51 AM:
----------------------------------------------------------

Patch adds JUnit test cases as suggested by Doron.

      was (Author: timmsc):
    Patch adds JUnit test cases as suggested by Doren.
 

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565838#action_12565838 ]

Timo Nentwig commented on LUCENE-997:
-------------------------------------

200ms? No, probably not. I don't recall what resolution I used in my test but actually the timeout check took more time than the Lucene query...

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12565906#action_12565906 ]

Sean Timm commented on LUCENE-997:
----------------------------------

Paul,
I think that if we were to use System.currentTimeMillis(), we would eschew the TimerThread as Doron suggests in his Dec. 15 comment.  I haven't seen any performance issues with System.currentTimeMillis().

As far as 200ms, I think that is too large of a default resolution (and with the current implementation it is not configurable).  With a 200 ms resolution, a query with a 1 second time allowed could timeout in 800 ms, and one with a time allowed of 500 ms could timeout in 300 ms.  I think it is much worse to timeout a query early than to timeout late.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doron Cohen updated LUCENE-997:
-------------------------------

    Attachment: timeout.patch

Sean thanks for adding the test.

In the attached I tightened the check of allowed elapsed time until timeout.
Also added info in the exception, and added ability to modify the resolution - default is 20ms (was 5ms).
Please let me know what you think.

As for System.currentTimeMillis() vs. Timer thread - IMHO Mike's comment on 'system clock changes' makes the timer thread favorable.

I checked this with up to 10,000 threads and with that number the test sometimes fails because it is quite tight on the max elapsed time required comparing to the timeout, so I don't see this is a problem. In the attached N_THREADS = 50 and this number of threads always passes for me.

If there are no more major concerns I think this is now ready to go in, question is where to - under core o.a.l.search or under contrib (query or misc).
Others?

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12566171#action_12566171 ]

Sean Timm commented on LUCENE-997:
----------------------------------

Doron, your comment for setResolution(long) says "The default timer resolution is 50 milliseconds", however, the default is actually 20 ms (public static final int DEFAULT_RESOLUTION = 20;).  Other than that, everything looks great.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12566175#action_12566175 ]

Doron Cohen commented on LUCENE-997:
------------------------------------

Oh wrote comment that was before I decided to change the default...
Thanks for catching this.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doron Cohen updated LUCENE-997:
-------------------------------

    Attachment: timeout.patch

Attached patch corrects default resolution comment.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doron Cohen updated LUCENE-997:
-------------------------------

    Attachment: timeout.patch

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568280#action_12568280 ]

Sean Timm commented on LUCENE-997:
----------------------------------

"If there are no more major concerns I think this is now ready to go in, question is where to - under core o.a.l.search or under contrib (query or misc)."

My preference would be for core o.a.l.search.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12568286#action_12568286 ]

Timo Nentwig commented on LUCENE-997:
-------------------------------------

I agree, core.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

1234