[jira] Created: (LUCENE-997) Add search timeout support to Lucene

classic Classic list List threaded Threaded
63 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12552174 ]

Nadav Har'El commented on LUCENE-997:
-------------------------------------

I'd like to add my 2 cents on this issue.

The more I use Lucene in various ways, the more I'm getting convinced that the "Hits" API should be de-emphasized (if not outright depracated), and users should be encouraged to use the HitCollector API (search() taking a hitcollector, TopDocCollector, and so on) - especially for advanced usage.

I believe that your TimeLimitedCollector is an interesting idea. However, there is simply no justification to change TopDocCollector, TopFieldDocCollector, Topdocs, Hits and Searcher. There's a MUCH simpler, and in my opinion cleaner, approach: Just make TimeLimitedCollector be a front-end for another collector (for example, TopDocCollector) which will be passed to TimeLimitedCollector's constructor. For every document that is presented to TimeLimitedCollector's collect(), it would call the inner collector's collect().

This way, without any changes to Lucene's core APIs, and just the addition of a new class, you can add the new functionality that you wanted. This, I think, is the beauty of the HitCollector interface over the "monolithic" Hits approach, and I think we should encourage this way of thinking instead of adding more and more features to Hits.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: LuceneTimeoutTest.java, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12552295 ]

Yonik Seeley commented on LUCENE-997:
-------------------------------------

{quote}make TimeLimitedCollector be a front-end for another collector (for example, TopDocCollector) which will be passed to TimeLimitedCollector's constructor.{quote}
+1

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: LuceneTimeoutTest.java, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12552322 ]

Doron Cohen commented on LUCENE-997:
------------------------------------

Sean, can you revise your patch to follow the suggestions above?

That is, create a TimeLimitedCollector that takes and Collector parameter
for its constructor. You should be able to hide all the TimerThread
details (use long instead of int) within the implementation of this
new collector class, and so when moving to Java5  the TimeThread
can be replaced by nanoTime.

Then we can add to either core search or under contrib.

On a related point - I am not happy with programming by a
runtimException - if others agree that this should become
a standard functionality, how about modifying Collector.collect()
to return a stop-or-continue indicator?

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: LuceneTimeoutTest.java, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12553586 ]

Sean Timm commented on LUCENE-997:
----------------------------------

Thanks for all of the feedback.  I'll take another stab at this.  I'm on vacation now and probably won't have time until I get back.  It'll probably be early Jan. before I have a new patch ready.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: LuceneTimeoutTest.java, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Timo Nentwig updated LUCENE-997:
--------------------------------

    Attachment: HitCollectorTimeoutDecorator.java

Example of the timeout HitCollector implemented as an decorator.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12555190 ]

Timo Nentwig commented on LUCENE-997:
-------------------------------------

IMHO it definitely should be part of the core. Being able to control the runtime of queries/your ressources is crucial for every live system and I really wonder this it has taken so long to address this.

Otherwise I totally agree with Navdav: that Hits thingie is nice and fine for simple full-text queries but as soon as things become somewhat more complex you don't get around writing your own HitCollector (and do stuff like Facets).

I also strongly agree that the timeout HC should be implemented as an decorator (what's been called "front-end" here), I just quickly wrote an example and attached it (no, I'm not happy throwing an runtime exception either):

MyHitCollector hc = new MyHitCollector();
s.search(q, null, HitCollectorTimeoutDecorator.decorate( hc, 10 ) );

And finally, why return partial results? I don't think that this is reasonable.

BTW I'm not sure whether volatile in the timer thread is really working reliably in 1.4...


> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Timo Nentwig updated LUCENE-997:
--------------------------------

    Attachment: MyHitCollector.java

Example HitCollector to be decorated by HitCollectorTimeoutDecorator.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12555203 ]

Paul Elschot commented on LUCENE-997:
-------------------------------------

By using only a HitCollector it cannot be guaranteed that searches will not take too long. The reason is that there are searches that take a long time but do not collect any docs.

For example consider the case where every even doc has term A and every odd doc has term B, and the query requires both A and B. This is going to take an amount of time that is linear in the size of the index but no document will be collected.

This means that every conjunction (boolean queries with more than one required clause, phrase queries, span near queries) will need to check for timeout. Also a HitCollector with a timeout facility will need a way
to be informed of a timeout when no document is collected.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12555205 ]

Timo Nentwig commented on LUCENE-997:
-------------------------------------

True, unfortunately, but still better than nothing (->current situation). This approach isn't very precise in matters of timing either. Also, throwing a RuntimeException feels more like a hack than well thought out code...

I don't know Lucene's code good enough in order to estimate whether it's possible to build a real timeout machanism at all/without changing the API/rewriting a lot of code but it's incredibly important to be able to cancel running queries. You don't want to servers under high load suffering from lucene queries running up to multiple *minutes* at the same time consuming quite a lot of memory. And it makes no sense either because nobody is waiting so long for results...

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Timm updated LUCENE-997:
-----------------------------

    Attachment: timeout.patch

Created a patch using Timo's HitCollectorTimeoutDecorator.  This patch just has a few mostly minor changes.  The biggest changes are that it uses a long instead of an int for the counter in TimerThread and the TimerThread interval is fixed at 10 ms.  It also throws a TimeLimitedCollector.TimeExceeded exception on a timeout.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Timm updated LUCENE-997:
-----------------------------

    Attachment:     (was: timeout.patch)

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Timm updated LUCENE-997:
-----------------------------

    Attachment: timeout.patch

Updated bad patch with good copy.

Created a patch using Timo's HitCollectorTimeoutDecorator. This patch just has a few mostly minor changes. The biggest changes are that it uses a long instead of an int for the counter in TimerThread and the TimerThread interval is fixed at 10 ms. It also throws a TimeLimitedCollector.TimeExceeded exception on a timeout.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Timm updated LUCENE-997:
-----------------------------

    Attachment: LuceneTimeoutTest.java

Updated to work with latest patch.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Timm updated LUCENE-997:
-----------------------------

    Attachment: TimerThreadTest.java

The TimerThreadTest illustrates the accuracy of the TimerThread under load.  On my 2GHz Xeon 4 CPU dual core RH AS 4 Linux box, it get a 20% difference between the TimerThread implementation and System.currentTimeMillis() and is independent of load.

java TimerThreadTest 8
[...]
10010   12020   [...]

With my single core single CPU Windows XP laptop I see a 20% difference at load, but when adding additional threads, I see an increasing difference.

java TimerThreadTest 0
[...]
10000   11819 [...]

java TimerThreadTest 2
[...]
10040   18890 [...]



> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562765#action_12562765 ]

Andrzej Bialecki  commented on LUCENE-997:
------------------------------------------

I believe this version of the patch won't work properly unless you add synchronization - writes to long values are non-atomic (Java Language Specification 17.7, as the comment says), that's why Nutch uses an int there. Perhaps using AtomicLong would be an answer, if you really need a long value.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562838#action_12562838 ]

Nicolas Lalevée commented on LUCENE-997:
----------------------------------------

AtomicLong is a Java 1.5 feature, so it doesn't fit either.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562845#action_12562845 ]

Paul Elschot commented on LUCENE-997:
-------------------------------------

After LUCENE-584 more work will be needed to get all conjunctions in the same place, but it is a starting point.

Once all conjunctions are in the same place, it would make sense to add a timeout there.


> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562869#action_12562869 ]

Sean Timm commented on LUCENE-997:
----------------------------------

Andrzej--

JLS 17.7 Non-atomic Treatment of double and long

"Writes and reads of volatile long and double values are always atomic."

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562936#action_12562936 ]

Andrzej Bialecki  commented on LUCENE-997:
------------------------------------------

Indeed, thanks for the correction - I forgot about the special treatment of volatile values.

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-997) Add search timeout support to Lucene

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562952#action_12562952 ]

Sean Timm commented on LUCENE-997:
----------------------------------

You are right that I forgot to change the comment when I changed it from an int to a long though.  "* updates to 32-bit-sized variables are atomic" is a pretty pointless comment now. :-)

> Add search timeout support to Lucene
> ------------------------------------
>
>                 Key: LUCENE-997
>                 URL: https://issues.apache.org/jira/browse/LUCENE-997
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Sean Timm
>            Priority: Minor
>         Attachments: HitCollectorTimeoutDecorator.java, LuceneTimeoutTest.java, LuceneTimeoutTest.java, MyHitCollector.java, timeout.patch, timeout.patch, timeout.patch, TimerThreadTest.java
>
>
> This patch is based on Nutch-308.
> This patch adds support for a maximum search time limit. After this time is exceeded, the search thread is stopped, partial results (if any) are returned and the total number of results is estimated.
> This patch tries to minimize the overhead related to time-keeping by using a version of safe unsynchronized timer.
> This was also discussed in an e-mail thread.
> http://www.nabble.com/search-timeout-tf3410206.html#a9501029

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

1234