[jira] Created: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

classic Classic list List threaded Threaded
65 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
-----------------------------------------------------------

                 Key: LUCENE-1424
                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
             Project: Lucene - Java
          Issue Type: New Feature
            Reporter: Mark Miller
            Priority: Minor


If we want to be able to highlight these queries, they need to be added to Lucene core (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Miller updated LUCENE-1424:
--------------------------------

    Description: If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.  (was: If we want to be able to highlight these queries, they need to be added to Lucene core (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.)

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Priority: Minor
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Miller updated LUCENE-1424:
--------------------------------

    Attachment: LUCENE-1424.patch

What do people think about putting these classes in? They have no max clause limitations, and reports have been that they can be *much* more efficient on large indexes. Solr has switched from using the boolean rewriting WildcardQuery,PrefixQuery to ConstantScore queries. We already put in the ConstantScoreRange query for just these reasons, so I think it makes since to add the rest of the family. An upside will be that I can add support for these queries to the Span highlighter. That will put our Highlighter in a position of being able to highlight pretty much every major query type, which I think is an important goal.

- Mark

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Priority: Minor
>         Attachments: LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642605#action_12642605 ]

Michael McCandless commented on LUCENE-1424:
--------------------------------------------

bq. What do people think about putting these classes in?

+1

I think in general we should more aggressively slurp useful things like this from Solr into Lucene.

Is there any reason to keep the rewrite-to-BooleanQuery version of these (vs, deprecating them in favor of the ConstantScore variety)?  Are the score differences caused by the rewrite-to-BooleanQuery implementations ever "useful"?

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Priority: Minor
>         Attachments: LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Assigned: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael McCandless reassigned LUCENE-1424:
------------------------------------------

    Assignee: Michael McCandless

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642606#action_12642606 ]

Michael McCandless commented on LUCENE-1424:
--------------------------------------------

Woops... patch uses @Override (Java 5 only):

{code}
common.compile-core:
    [mkdir] Created dir: /lucene/lucene.constant/build/classes/java
    [javac] Compiling 333 source files to /lucene/lucene.constant/build/classes/java
    [javac] /lucene/lucene.constant/src/java/org/apache/lucene/search/ConstantScoreWildcardQuery.java:69: annotations are not supported in -source 1.4
    [javac] (use -source 5 or higher to enable annotations)
    [javac]   @Override
    [javac]    ^
    [javac] /lucene/lucene.constant/src/java/org/apache/lucene/search/WildcardFilter.java:49: annotations are not supported in -source 1.4
    [javac] (use -source 5 or higher to enable annotations)
    [javac]   @Override
    [javac]    ^
    [javac] 2 errors
{code}

I'll fix in my local checkout.

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642611#action_12642611 ]

Michael McCandless commented on LUCENE-1424:
--------------------------------------------

Patch looks good.  I plan to commit in a day or two.

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642615#action_12642615 ]

Mark Harwood commented on LUCENE-1424:
--------------------------------------

>> Are the score differences caused by the rewrite-to-BooleanQuery implementations ever "useful"?

So we need to consider what we are losing - TF, IDF, coordination, length norm, doc boosts.

I can only think of one use case which relates to coordination factor.

If you have a "category" field for a product e.g. given Lucene docs for these books:

Title:            Lucene in Action
Category:   /Books/Computing/Languages/Java
                    /Books/Computing/InformationRetrieval

Title:           The Long Tail
Category:  /Books/Business/Internet
                   /Books/Computing

You might then use a wildcard search of /Books/Computing/* and "Lucene in Action" would rank higher than "The Long Tail" because a BooleanQuery would score a higher coordination factor suggesting LIA got more hits under this "/Books/Computing.." category. There would still be the issue of IDF potentially skewing results but the coordination factor is potentially useful here.

I think in general IDF tends to be useless for "auto-expanded" terms e.g. Wildcard, fuzzy etc. Incidentally, we still see that IDF issue in fuzzy queries ranking rare mis-spellings higher but that's another issue (one I resolved in contrib's FuzzyLikeThisQuery).

I suppose one other consideration is for people who have created any doc boosts e.g. trying to use this to boost by date.

I don't think any of these cases necessarily outweigh the benefit to be obtained from switching "wildcard/prefix to constant score queries"


Cheers,
Mark







> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642617#action_12642617 ]

Uwe Schindler commented on LUCENE-1424:
---------------------------------------

Just a suggestion:
How about *not* inventing new Class names. Just include both code parts into e.g. (Range|Prefix|Wildcard)Query's rewrite() method. It would then be possible to just switch the QueryParser or other code by e.g. setting a global (static) flag in (Range|Prefix|Wildcard)Query that switches between both implementations).

To your patch:
For printing out the boost in toString(): there is a nice helper routine (instead of "if (getBoost()!=1.0f)...."). I think it should be consistent with toString() of all other core query types.

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642628#action_12642628 ]

Hoss Man commented on LUCENE-1424:
----------------------------------

bq. Is there any reason to keep the rewrite-to-BooleanQuery

In addition to the coord example Mark mentioned, there are also cases where the tf and idf values for terms matching a wildcard/prefix are meaningful ... but the other advantage to keeping the existing implementations is use cases where the number of unique terms a query expands to is  is very small, but the number of documents matched is very large ... in theory (but i haven't tested this) the expanding queries should be more efficient in space and equally efficient in time as the ConstantScore equivalents.

bq. It would then be possible to just switch the QueryParser or other code by e.g. setting a global (static) flag.

I'm loath to see more static "setter" methods added to query clauses, but there's no reason this couldn't be a member property on instances of the classes with corresponding properties on QueryParser.

In theory it could even be a tertiary state property: REWRITE_TO_BOOLEAN, REWRITE_TO_CONSTANT_SCORE. REWRITE_GUESS ... where the third option caused the REWRITE method to make it's best guess using a first pass at the TermEnum iteration and aborting if the number of terms get's above some threshold.

(but that kind of "optimization" would be premature without testing .. the point is it would be possible)

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Miller updated LUCENE-1424:
--------------------------------

    Attachment: LUCENE-1424.patch

Sorry bout the @overrides...grabbed/copied from solr and forgot the java 1.4 cleanse. There was also one StringBuilder in the wildcardfilter that this patch pulls.

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch, LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12643219#action_12643219 ]

Michael McCandless commented on LUCENE-1424:
--------------------------------------------

OK, it sounds like we should preserve the rewrite-to-BooleanQuery
option for each of these Query classes since there are clear use cases
where it makes sense.

I do like the idea of adding "constant score capability" to the
existing query classes, instead of adding new ConstantScoreXXXQuery
classes.

I don't really like the REWRITE_GUESS option because it could lead to
strange inconsistent results as seen by the end user -- eg prefix1
sorts one way but then a relaxed prefix2 sorts another way.  I think a
simple static binary choice is good?

Couldn't we build this functionality (get/setRewriteMethod(...)) into
MultiTermQuery?  And then fix those queries that don't already to
subclass MultiTermQuery (at least RangeQuery and PrefixQuery)?

Finally, I would also prefer non-static methods to instruct
QueryParser which rewrite method to use.  In fact we already have
setUseOldRangeQuery -- maybe change this to
setUseConstantScoreRewriteMethod (defaulting to true)?


> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch, LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

Mark Miller-3
Michael McCandless (JIRA) wrote:
> I do like the idea of adding "constant score capability" to the
> existing query classes, instead of adding new ConstantScoreXXXQuery
> classes.
>  
I am torn on this one. Not against the idea, but I think it should be
consistent with ConstantScoreRangeQuery, and it seems a bit awkward to
deprecate that at this point. Not a huge deal either way I guess though.

- Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: [jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

Uwe Schindler
> Michael McCandless (JIRA) wrote:
> > I do like the idea of adding "constant score capability" to the
> > existing query classes, instead of adding new ConstantScoreXXXQuery
> > classes.
> >
> I am torn on this one. Not against the idea, but I think it should be
> consistent with ConstantScoreRangeQuery, and it seems a bit awkward to
> deprecate that at this point. Not a huge deal either way I guess though.

In my opinion, RangeQuery should also have this boolean flag for
consistency. Deprecating ConstantScoreRangeQuery is not so nice, but needed.
A workaround would be:

... RangeQuery ... {
        public Query rewrite(IndexReader r) {
                if (constantScore)
                        return new ConstantScoreRangeQuery(...).rewrite(r);
                else...
        }
}


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

Michael McCandless-2
I think it would be quite clean if all XXXQuery classes that simply
enumerate certain terms and then match all docs containing those
terms, would subclass MultiTermQuery which itself would implement
set/getRewriteMethod?

I guess we'd also create a MultiTermFilter, which is basically the
same as TermsFilter from contrib/queries.

And then no more separate ConstantScoreXXXQuery classes... so we'd
deprecate ConstantScoreRangeQuery.

At least these [core] queries should fit this refactoring: RangeQuery,
PrefixQuery, WildcardQuery, FuzzyQuery.

I don't think I'll have time near-term to do this myself... but I do
think it's a good step forward, to bring constant-score rewriting to
all of these XXXQuery classes.

Mike

Uwe Schindler wrote:

>> Michael McCandless (JIRA) wrote:
>>> I do like the idea of adding "constant score capability" to the
>>> existing query classes, instead of adding new ConstantScoreXXXQuery
>>> classes.
>>>
>> I am torn on this one. Not against the idea, but I think it should be
>> consistent with ConstantScoreRangeQuery, and it seems a bit awkward  
>> to
>> deprecate that at this point. Not a huge deal either way I guess  
>> though.
>
> In my opinion, RangeQuery should also have this boolean flag for
> consistency. Deprecating ConstantScoreRangeQuery is not so nice, but  
> needed.
> A workaround would be:
>
> ... RangeQuery ... {
> public Query rewrite(IndexReader r) {
> if (constantScore)
> return new ConstantScoreRangeQuery(...).rewrite(r);
> else...
> }
> }
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

hossman

: I think it would be quite clean if all XXXQuery classes that simply
: enumerate certain terms and then match all docs containing those
: terms, would subclass MultiTermQuery which itself would implement
: set/getRewriteMethod?

FWIW: that may be tricky for the collator logic that was added to
RangeQuery.

: > > I am torn on this one. Not against the idea, but I think it should be
: > > consistent with ConstantScoreRangeQuery, and it seems a bit awkward to
: > > deprecate that at this point. Not a huge deal either way I guess though.

it wouldn't really be that awkward.  assuming RangeQuery had an option for
rewriting to a ConstantScoreQuery, ConstantScoreRangeQuery becomes as
simple as...

   public class ConstantScoreRangeQuery extends RangeQuery {
     public ConstantScoreRangeQuery(...) {
        super(...);
        this.setRewriteConstantScore(true);
     }
   }



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

     [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mark Miller updated LUCENE-1424:
--------------------------------

    Attachment: LUCENE-1424.patch

Heres an early review patch.

I switched range and prefix to use multiterm query, but its debatable if thats necessary. Prob a few clock cycles slower, and the benefits are slim beyond overall code design niceness (which counts a lot to me <g>). Just not sure every mutlitermquery would want to have to implement a constantscore version, and the amount of code reuse saved is low (it could do a bit more with something like getFilteredEnum, but thats not much either).

TODO:

look over
some comments maybe
some tests for the constant score versions
rangequery is not as expressive as constantscorequery, which can have separate inclusive/exclusive ends.


When I was talking about deprecating constantscorequery being awkward, its wasnt really in the implementation sense (i think we can leave it as is), but more the deprecating one of the newest queries already :) Still don't consider that a huge deal though.


> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch, LUCENE-1424.patch, LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Issue Comment Edited: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644756#action_12644756 ]

[hidden email] edited comment on LUCENE-1424 at 11/3/08 10:44 AM:
---------------------------------------------------------------

Heres an early review patch.

I switched range and prefix to use multiterm query, but its debatable if thats necessary. Prob a few clock cycles slower, and the benefits are slim beyond overall code design niceness (which counts a lot to me <g>). Just not sure every mutlitermquery would want to have to implement a constantscore version, and the amount of code reuse saved is low (it could do a bit more with something like getConstantScoreQuery, but thats not much either - I may put it in come to think of it).

TODO:

look over
some comments maybe
some tests for the constant score versions
rangequery is not as expressive as constantscorequery, which can have separate inclusive/exclusive ends.


When I was talking about deprecating constantscorequery being awkward, its wasnt really in the implementation sense (i think we can leave it as is), but more the deprecating one of the newest queries already :) Still don't consider that a huge deal though.


      was (Author: [hidden email]):
    Heres an early review patch.

I switched range and prefix to use multiterm query, but its debatable if thats necessary. Prob a few clock cycles slower, and the benefits are slim beyond overall code design niceness (which counts a lot to me <g>). Just not sure every mutlitermquery would want to have to implement a constantscore version, and the amount of code reuse saved is low (it could do a bit more with something like getFilteredEnum, but thats not much either).

TODO:

look over
some comments maybe
some tests for the constant score versions
rangequery is not as expressive as constantscorequery, which can have separate inclusive/exclusive ends.


When I was talking about deprecating constantscorequery being awkward, its wasnt really in the implementation sense (i think we can leave it as is), but more the deprecating one of the newest queries already :) Still don't consider that a huge deal though.

 

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch, LUCENE-1424.patch, LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644802#action_12644802 ]

Michael McCandless commented on LUCENE-1424:
--------------------------------------------

Patch looks good!  Thanks Mark.

I think we should indeed "factor up" into MultiTermQuery the ability to create a ConstantScoreQuery out of the filter generated by enumerating the terms and walking the docs for those terms?  Then we don't need to special case in each of the subclasses.

Maybe we can then fix Wildcard/Prefix/RangeFilter to create the corresponding query and then ask it for its filter (assuming we make a method eg "getDocIdSet" in MultiTermQuery)?  Or I guess we could just make a QueryWrapperFilter around the corresponding query, though that seems rather roundabout.  I don't think we need to have duplicated code in PrefixFilter, RangeFilter, WildcardFilter.

{quote}
When I was talking about deprecating constantscorequery being awkward, its wasnt really in the implementation sense (i think we can leave it as is), but more the deprecating one of the newest queries already  Still don't consider that a huge deal though.
{quote}

Well ... this is just how software evolves :)  You can't control which code will be the "victim" of a nice refactoring.  It's a healthy sign of progress, and progress is good!

bq. rangequery is not as expressive as constantscorequery, which can have separate inclusive/exclusive ends.

If we do deprecate ConstantScoreRangeQuery (I think we should) then we could add a ctor to RangeQuery matching ConstantScoreRangeQuery's more expressive one?

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch, LUCENE-1424.patch, LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (LUCENE-1424) Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery

ASF GitHub Bot (Jira)
In reply to this post by ASF GitHub Bot (Jira)

    [ https://issues.apache.org/jira/browse/LUCENE-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12644805#action_12644805 ]

Mark Miller commented on LUCENE-1424:
-------------------------------------

bq. I think we should indeed "factor up" into MultiTermQuery...

Ah, I see...I hadn't thought all way up that path - it goes a bit further than was in my mind. The way to go for sure.

All the comments make perfect sense, I'll work up a new patch while I'm at apachcon.

> Add ConstantScorePrefixQuery and ConstantScoreWildcardQuery
> -----------------------------------------------------------
>
>                 Key: LUCENE-1424
>                 URL: https://issues.apache.org/jira/browse/LUCENE-1424
>             Project: Lucene - Java
>          Issue Type: New Feature
>            Reporter: Mark Miller
>            Assignee: Michael McCandless
>            Priority: Minor
>         Attachments: LUCENE-1424.patch, LUCENE-1424.patch, LUCENE-1424.patch
>
>
> If we want to be able to highlight these queries, they need to be added to Lucene core or contrib (solr's WildCardFilter can be used to create the ConstantScoreWildcardQuery). They are very useful anyway.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

1234