putting UnInvertedField instances in a SolrCache?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

putting UnInvertedField instances in a SolrCache?

hossman

recent wiki updates have be looking at UnInvertedField for the first time
(i haven't been very good at keeping up with commits the last few months)
and i'm wondering about the use of a "static Cache multiValuedFieldCache"
keyed off of SolrIndexSearcher.

Lucene-Java is trying to move away from this pattern in FieldCache, and in
Solr we already have a nice and robust cache mechanism on each
SolrIndexSearcher -- that includes the possibility of doing auto-warming
via regenerators -- so why don't we suse that for UnInvertedField?

suggested changes...

1) add a new "special" (as opposed to "user") SolrCache instance named
"facetCache" to SolrIndexSearcher (just like filterCache and
queryResultCache) where the key is a field name and the value is an
UnInvertedField instance.

2) I think the way the "special" caches are
initialized they eist with defaults even if they aren't declared in
solrconfig.xml, but if i'm wrong we should consier making facetCache work
that way.

2) add a regenerator for facetCache (relatively trivial)

3) remove all of the static cashing code from UnInvertedField

  thoughts?


-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: putting UnInvertedField instances in a SolrCache?

Yonik Seeley-2
Right.
  - we need a "blocking cache" to avoid more than one thread
attempting to generate, but that can be done outside the SolrCache for
now.
  - prob want to expose the statistics collected... (see logging
output of new faceting stuff)
  - might want a way to dynamically add caches.. but for now adding a
"magic" facetCache that exists even when not on solrconfig.xml is prob
easiest (the current solr caches do not get instantiated if they are
not in solrconfig.xml - they are seen as optional).

-Yonik

On Tue, Dec 2, 2008 at 6:27 PM, Chris Hostetter
<[hidden email]> wrote:

>
> recent wiki updates have be looking at UnInvertedField for the first time (i
> haven't been very good at keeping up with commits the last few months) and
> i'm wondering about the use of a "static Cache multiValuedFieldCache" keyed
> off of SolrIndexSearcher.
>
> Lucene-Java is trying to move away from this pattern in FieldCache, and in
> Solr we already have a nice and robust cache mechanism on each
> SolrIndexSearcher -- that includes the possibility of doing auto-warming via
> regenerators -- so why don't we suse that for UnInvertedField?
>
> suggested changes...
>
> 1) add a new "special" (as opposed to "user") SolrCache instance named
> "facetCache" to SolrIndexSearcher (just like filterCache and
> queryResultCache) where the key is a field name and the value is an
> UnInvertedField instance.
>
> 2) I think the way the "special" caches are initialized they eist with
> defaults even if they aren't declared in solrconfig.xml, but if i'm wrong we
> should consier making facetCache work that way.
>
> 2) add a regenerator for facetCache (relatively trivial)
>
> 3) remove all of the static cashing code from UnInvertedField
>
>        thoughts?
>
>
> -Hoss
Reply | Threaded
Open this post in threaded view
|

Re: putting UnInvertedField instances in a SolrCache?

hossman

:   - might want a way to dynamically add caches.. but for now adding a
: "magic" facetCache that exists even when not on solrconfig.xml is prob
: easiest (the current solr caches do not get instantiated if they are
: not in solrconfig.xml - they are seen as optional).

thinking about it more -- keeping the same behavior as filterCache isn't
really a bad idea, it would certianly obey the law of least suprise for
users who are upgrading: if a facetCache doesn't exist, default to the
"old" enum behavior.

(however: facetCache doesn't exist but facet.method=fc then an error
pointing outhte facetCache needs to be defined seems appropriate)



-Hoss