[jira] Created: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a Scorer which gets cached rather than a bit set

classic Classic list List threaded Threaded
38 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a Scorer which gets cached rather than a bit set

JIRA jira@apache.org
When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a Scorer which gets cached rather than a bit set
------------------------------------------------------------------------------------------------------------------------------------

                 Key: LUCENE-1911
                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
             Project: Lucene - Java
          Issue Type: Bug
            Reporter: Mark Miller
             Fix For: 2.9


there is a large performance cost to this.

The old impl for this type of thing, QueryFilter, recommends :

@deprecated use a CachingWrapperFilter with QueryWrapperFilter

see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Miller updated LUCENE-1911:
--------------------------------

    Summary: When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set  (was: When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a Scorer which gets cached rather than a bit set)

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>             Fix For: 2.9
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Assigned: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Miller reassigned LUCENE-1911:
-----------------------------------

    Assignee: Uwe Schindler

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756133#action_12756133 ]

Mark Miller commented on LUCENE-1911:
-------------------------------------

sounds like Uwe is working on a patch so that you can tell the CachingWrapperFilter to cache the results as a bitset (optionally), which will allow you to duplicated the deprecated QueryFilter.

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Miller updated LUCENE-1911:
--------------------------------

    Description:
there is a large performance cost to this.

The old impl for this type of thing, QueryFilter, recommends :

@deprecated use a CachingWrapperFilter with QueryWrapperFilter

The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.

see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

  was:
there is a large performance cost to this.

The old impl for this type of thing, QueryFilter, recommends :

@deprecated use a CachingWrapperFilter with QueryWrapperFilter

see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1


> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756138#action_12756138 ]

Mark Miller commented on LUCENE-1911:
-------------------------------------

whether we do an RC or not after this I don't know.

Technically, if we don't want to - its not required. All thats required is that we get the 3 votes and no -1 votes with valid tech arguments.

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

     [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Uwe Schindler updated LUCENE-1911:
----------------------------------

    Attachment: LUCENE-1911.patch

Here a patch, that implements my proposal.

Thomas Becker: Could you apply this patch and test if itr resolves your speed problem?

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756145#action_12756145 ]

Mark Miller commented on LUCENE-1911:
-------------------------------------

Very nice Uwe! Patch looks very good. Nicely takes care of the QueryFilter issue as well.

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756148#action_12756148 ]

Uwe Schindler commented on LUCENE-1911:
---------------------------------------

By the way, the added IOException to the protected method is no BW break, as the method is new in 2.9.

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756151#action_12756151 ]

Mark Miller commented on LUCENE-1911:
-------------------------------------

bq. the protected method

Its funny, because literally this morning, I was looking at CachingWrapperFilter for other reasons and I saw that method and briefly wondered what it was for - it just return what was passed to it by default. I moved on without thinking too much of it - but it makes perfect sense now ;)

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756160#action_12756160 ]

Uwe Schindler commented on LUCENE-1911:
---------------------------------------

I will post a new patch later with some added tests. I will also add SortedVIntList to the cacheable implementations.

Maybe in 3.1 we could add a DocIdSet.isCacheable() method (like the proposed isRandomAccess()), that defaults to false. The 4 utils impls of DocIdSet could then return true here. This would be cleaner than the instanceof check.

Or do somebody want isCacheable to DocIdSet added for 2.9, too?

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756171#action_12756171 ]

Mark Miller commented on LUCENE-1911:
-------------------------------------

Could we commit now Uwe? Its easy to tweak if we have to. Its likely to work and stick like this though - so if we commit, I can make the release artifcats and run tests and all that jazz - then if we are good, it will be a very fast turnaround. Otherwise I have to start all that much later in either case. I'd prefer to be optimistic and get everything ready by tonight.

I'll leave it up to you though.

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756180#action_12756180 ]

Uwe Schindler commented on LUCENE-1911:
---------------------------------------

I understand you, but:

After reviewing other DocIdSets, I found more of them, that could be easily cached, like e.g. one of the DocIdSets returned by FieldCacheRangeFilter (the only-FieldCache-non-TermDocs-backed one). You cannot check all of them with instanceof and may some of them twice, which is a bad idea. A simple check with DocIdSet.isCacheable() looks like a good solution. If we default it to false, we have no backwards-problem.

If we do not want to do this, I will add at least SortedVIntList to the cacheable implementations.

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756181#action_12756181 ]

Mark Miller commented on LUCENE-1911:
-------------------------------------

Okay - no problem - if we already know theres more to do, no point yet. Lets not do it before it makes sense.

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756188#action_12756188 ]

Mark Miller commented on LUCENE-1911:
-------------------------------------

I guess I would lean toward isCacheable myself. Its much cleaner than trying to juggle a list and overrides.

I guess the downside is that someone could be using a custom one that was cacheable, and so it would be copied. But they can
update to the new API as a workaround.
And you get that with the other solution in certain cases too (unless the user updates to use the new API and does the override).
That also requires tracking which CachingWrapperFilters get passed what Filters and its just much uglier right?

+1 on isCacheable.

You could argue to keep the override too - if someone was using a jar that says its not cachaeable, but it is, and can't change the source. A
Likely scenario I'm sure ;) But worth the method it would seem. It still becomes much less necessary though.

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756194#action_12756194 ]

Michael McCandless commented on LUCENE-1911:
--------------------------------------------

bq. we could add a DocIdSet.isCacheable() method

+1 to this, and to keeping docIdSetToCache() for overriding.  This is much cleaner than trying to check for every known DocIdSet impl.

Thanks for fixing this Uwe!

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756207#action_12756207 ]

Mark Miller commented on LUCENE-1911:
-------------------------------------

Now I regret not pushing for the isCachable route to fix the Highlighter caching issue :(



> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

Paul Elschot
In reply to this post by JIRA jira@apache.org
I have not yet looked at the the patch, but after the comments at LUCENE-1911
I think isCacheable() could also return true for OpenBitSet, not only for SortedVIntList.


Regards,
Paul Elschot.



On Wednesday 16 September 2009 21:33:58 Michael McCandless (JIRA) wrote:
>
> [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756194#action_12756194 ]
>
> Michael McCandless commented on LUCENE-1911:
> --------------------------------------------
>
> bq. we could add a DocIdSet.isCacheable() method
>
> +1 to this, and to keeping docIdSetToCache() for overriding. This is much cleaner than trying to check for every known DocIdSet impl.
>
> Thanks for fixing this Uwe!
>
> > When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> > -------------------------------------------------------------------------------------------------------------------------------------------------------------
> >
> > Key: LUCENE-1911
> > URL: https://issues.apache.org/jira/browse/LUCENE-1911
> > Project: Lucene - Java
> > Issue Type: Bug
> > Reporter: Mark Miller
> > Assignee: Uwe Schindler
> > Fix For: 2.9
> >
> > Attachments: LUCENE-1911.patch
> >
> >
> > there is a large performance cost to this.
> > The old impl for this type of thing, QueryFilter, recommends :
> > @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> > The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> > see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>



Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756209#action_12756209 ]

Uwe Schindler commented on LUCENE-1911:
---------------------------------------

What do you mean?

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1911) When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set

JIRA jira@apache.org
In reply to this post by JIRA jira@apache.org

    [ https://issues.apache.org/jira/browse/LUCENE-1911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756210#action_12756210 ]

Mark Miller commented on LUCENE-1911:
-------------------------------------

Remember the CachingTokenFilter issue that first popped with the Highlighter? And you suggested we add the isCachable? I was hoping the RC process was going to be smoother at the time, so I just pushed to have the simple override for now. But looking back with what I know now, I wish we had gone for it.

> When using QueryWrapperFilter with CachingWrapperFilter, QueryWrapperFilter returns a DocIdSet that creates a Scorer, which gets cached rather than a bit set
> -------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: LUCENE-1911
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1911
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Mark Miller
>            Assignee: Uwe Schindler
>             Fix For: 2.9
>
>         Attachments: LUCENE-1911.patch
>
>
> there is a large performance cost to this.
> The old impl for this type of thing, QueryFilter, recommends :
> @deprecated use a CachingWrapperFilter with QueryWrapperFilter
> The deprecated QueryFilter itself also suffers from the problem because its now implemented using a CachingWrapperFilter and QueryWrapperFilter.
> see http://search.lucidimagination.com/search/document/7f54715f14b8b7a/lucene_2_9_0rc4_slower_than_2_4_1

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

12