[jira] Created: (SOLR-2429) ability to not cache a filter

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (SOLR-2429) ability to not cache a filter

Nick Burch (Jira)
ability to not cache a filter
-----------------------------

                 Key: SOLR-2429
                 URL: https://issues.apache.org/jira/browse/SOLR-2429
             Project: Solr
          Issue Type: New Feature
            Reporter: Yonik Seeley


A user should be able to add {!cache=false} to a query or filter query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-2429) ability to not cache a filter

Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007122#comment-13007122 ]

Yonik Seeley commented on SOLR-2429:
------------------------------------

The annoying part here is we need more metadata than just "Query" that we use now for a filter.
Unfortunately, SolrIndexSearcher uses List<Query> everywhere.

We could create something like a SolrQuery extends Query that wrapped a normal query and added additional metadata (like cache options).  That's a bit messier since we'd have instanceof checks and casts everywhere though.

Another option is to create a SolrQuery class that does not extend Query - hence methods taking List<Query> would now need to take List<SolrQuery>

{code}
class SolrQuery {
  Query q;
  QParser qparser;
  boolean cache;
  ...
}
{code}

Thoughts?

> ability to not cache a filter
> -----------------------------
>
>                 Key: SOLR-2429
>                 URL: https://issues.apache.org/jira/browse/SOLR-2429
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>
> A user should be able to add {!cache=false} to a query or filter query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-2429) ability to not cache a filter

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007217#comment-13007217 ]

Hoss Man commented on SOLR-2429:
--------------------------------

why not extend Query? ... it could actually rewrite to the Query it wraps, giving us the best of both worlds.

FWIW: it also seems like it would make sense for this type of syntax/decoration to work with the "q" param (skipping the queryResultCache)

> ability to not cache a filter
> -----------------------------
>
>                 Key: SOLR-2429
>                 URL: https://issues.apache.org/jira/browse/SOLR-2429
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>
> A user should be able to add {!cache=false} to a query or filter query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-2429) ability to not cache a filter

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007265#comment-13007265 ]

Ryan McKinley commented on SOLR-2429:
-------------------------------------

I'm not sure this is related -- it could be -- I'm looking writing a custom query from:
{code:java}
  @Override
  public Query getFieldQuery(QParser parser, SchemaField field, String externalVal)
{code}

and it would be great to know if this is used as a filter or not -- should it include scoring?  Are there ways to build the query where parts are cached and some is not?  



> ability to not cache a filter
> -----------------------------
>
>                 Key: SOLR-2429
>                 URL: https://issues.apache.org/jira/browse/SOLR-2429
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>
> A user should be able to add {!cache=false} to a query or filter query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-2429) ability to not cache a filter

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007324#comment-13007324 ]

Otis Gospodnetic commented on SOLR-2429:
----------------------------------------

I'm with Hoss.  For many months now, I've been dreaming about the possibility of telling Solr to execute a query without caching the results.

> ability to not cache a filter
> -----------------------------
>
>                 Key: SOLR-2429
>                 URL: https://issues.apache.org/jira/browse/SOLR-2429
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>
> A user should be able to add {!cache=false} to a query or filter query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (SOLR-2429) ability to not cache a filter

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13007340#comment-13007340 ]

David Smiley commented on SOLR-2429:
------------------------------------

Heh, me too!  I was pondering this last night; I know specific queries will needlessly pollute the cache.  I was imagining a syntax such as this:  fq={!cache=no}queryhere

> ability to not cache a filter
> -----------------------------
>
>                 Key: SOLR-2429
>                 URL: https://issues.apache.org/jira/browse/SOLR-2429
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>
> A user should be able to add {!cache=false} to a query or filter query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (SOLR-2429) ability to not cache a filter

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yonik Seeley updated SOLR-2429:
-------------------------------

    Attachment: SOLR-2429.patch

Here's a patch that allows one to add cache=false to top level queries (main queries, filter queries, facet queries, etc).

Currently (without this patch) Solr generates the set of documents that match each filter individually (this is so they can be cached and reused).

Adding cache=false to the main query prevents lookup/storing in the query cache.  Adding cache=false to any filter query causes the filterCache to not be used.  Further, the filter query is actually run in parallel to the main query and any other non-cached filter queries (which can speed things up if the base query or other filter queries are relatively sparse).

There is also an optional "cost" parameter that controls the order in which non-cached filter queries are evaluated so knowledgable users can order less expensive non-cached filters before expensive non-cached filters.

As an additional feature for very high cost filters, if cache=false and cost>=100 and the query implements the PostFilter interface, a Collector will be requested from that query and used to filter documents after they have matched the main query and all other filter queries.  There can be multiple post filters, and they are also ordered by cost.

The frange query (a range over function queries, background here:
http://www.lucidimagination.com/blog/2009/07/06/ranges-over-functions-in-solr-14/
) also now implements PostFilter.

Examples:
{code}
// normal function range query used as a filter, all matching documents generated up front and cached
fq={!frange l=10 u=100}mul(popularity,price)

// function range query run in parallel with the main query like a traditional lucene filter
fq={!frange l=10 u=100 cache=false}mul(popularity,price)

// function range query checked after each document that already matches the query and all other filters.  Good for really expensive function queries.
fq={!frange l=10 u=100 cache=false cost=100}mul(popularity,price)
{code}

> ability to not cache a filter
> -----------------------------
>
>                 Key: SOLR-2429
>                 URL: https://issues.apache.org/jira/browse/SOLR-2429
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>         Attachments: SOLR-2429.patch
>
>
> A user should be able to add {!cache=false} to a query or filter query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Resolved] (SOLR-2429) ability to not cache a filter

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

     [ https://issues.apache.org/jira/browse/SOLR-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Yonik Seeley resolved SOLR-2429.
--------------------------------

       Resolution: Fixed
    Fix Version/s: 3.4

> ability to not cache a filter
> -----------------------------
>
>                 Key: SOLR-2429
>                 URL: https://issues.apache.org/jira/browse/SOLR-2429
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>             Fix For: 3.4
>
>         Attachments: SOLR-2429.patch
>
>
> A user should be able to add {!cache=false} to a query or filter query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (SOLR-2429) ability to not cache a filter

Nick Burch (Jira)
In reply to this post by Nick Burch (Jira)

    [ https://issues.apache.org/jira/browse/SOLR-2429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13068351#comment-13068351 ]

Koji Sekiguchi commented on SOLR-2429:
--------------------------------------

As Queries are wrapped into WrappedQuery when {!cache} is specified, Highlighter and FastVectorHighlighter cannot highlight them.

Using solr example:

http://localhost:8983/solr/select/?q=headphones&hl=on&hl.fl=includes
=> a snippet returned

http://localhost:8983/solr/select/?q={!cache=false}headphones&hl=on&hl.fl=includes
=> no snippets returned

> ability to not cache a filter
> -----------------------------
>
>                 Key: SOLR-2429
>                 URL: https://issues.apache.org/jira/browse/SOLR-2429
>             Project: Solr
>          Issue Type: New Feature
>            Reporter: Yonik Seeley
>             Fix For: 3.4
>
>         Attachments: SOLR-2429.patch
>
>
> A user should be able to add {!cache=false} to a query or filter query.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]