How to merge an "autofacet" with a predefined facet

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

How to merge an "autofacet" with a predefined facet

Xavier
Hi everyone,

Like explained in this post : http://lucene.472066.n3.nabble.com/How-to-index-a-facetfield-by-searching-words-matching-from-another-Textfield-td3761201.html

I have created a dynamic facet at indexation by searching terms in a fulltext field.

But i don't know if it's possible to merge this "autocreated" facet with a facet already predefined ? i tried to used <copyField> (adding this to my code in my previous post) :
<copyField source="text_tag_facet" dest="predefined_facet" />

 but it's not seems to work ... (my text_tag_facet is always working, but didnt merged with my predefined_facet)

It's maybe because (As I understood) the real (stored) value of this dynamic facet is still the initial fulltext  ?? (or maybe i'm wrong ...)

I'm a little confused about this and i'm certainly doing it wrong but i begin to feel that those kinds of manipulation arent feasible into schema.xml ????

Best regards.
Em
Reply | Threaded
Open this post in threaded view
|

Re: How to merge an "autofacet" with a predefined facet

Em
Hi Xavier,

> It's maybe because (As I understood) the real (stored) value of this
dynamic
> facet is still the initial fulltext  ?? (or maybe i'm wrong ...)
Exactly.
CopyField does not copy the analyzed result of a field into another one.
Instead, the original content given to that field (the unanalyzed raw
input) is getting copied.

Could you explain what is the difference between your text_tag_facets
and your predefined facets?

Kind regards,
Em

Am 21.02.2012 17:11, schrieb Xavier:

> Hi everyone,
>
> Like explained in this post :
> http://lucene.472066.n3.nabble.com/How-to-index-a-facetfield-by-searching-words-matching-from-another-Textfield-td3761201.html
>
> I have created a dynamic facet at indexation by searching terms in a
> fulltext field.
>
> But i don't know if it's possible to merge this "autocreated" facet with a
> facet already predefined ? i tried to used <copyField> (adding this to my
> code in my previous post) :
> *<copyField source="text_tag_facet" dest="predefined_facet" />*
>
>  but it's not seems to work ... (my text_tag_facet is always working, but
> didnt merged with my predefined_facet)
>
> It's maybe because (As I understood) the real (stored) value of this dynamic
> facet is still the initial fulltext  ?? (or maybe i'm wrong ...)
>
> I'm a little confused about this and i'm certainly doing it wrong but i
> begin to feel that those kinds of manipulation arent feasible into
> schema.xml ????
>
> Best regards.
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-merge-an-autofacet-with-a-predefined-facet-tp3763988p3763988.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: How to merge an "autofacet" with a predefined facet

Xavier
Sure, the difference between my 2 facets are :

- 'predefined_facets' contains values already filled in my database like :
'web langage', 'cooking', 'fishing' ....

- 'text_tag_facets' will contain the same possible value but determined automatically from a given wordslist by searching in the document text as shown in my previous post


Why i want to do that ? because sometimes my 'predefined_facets' is not defined, and even if it is, i want to defined it the more as possible.

Best regards,
Xavier
Em
Reply | Threaded
Open this post in threaded view
|

Re: How to merge an "autofacet" with a predefined facet

Em
Wouldn't it be easier to store both types in different fields?
At query-time you are able to do a facet on both and can combine the
results client-side to present them within the GUI.

Kind regards,
Em

Am 21.02.2012 17:52, schrieb Xavier:

> Sure, the difference between my 2 facets are :
>
> - 'predefined_facets' contains values already filled in my database like :
> 'web langage', 'cooking', 'fishing' ....
>
> - 'text_tag_facets' will contain the same possible value but determined
> automatically from a given wordslist by searching in the document text as
> shown in my previous post
>
>
> Why i want to do that ? because sometimes my 'predefined_facets' is not
> defined, and even if it is, i want to defined it the more as possible.
>
> Best regards,
> Xavier
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-merge-an-autofacet-with-a-predefined-facet-tp3763988p3764116.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: How to merge an "autofacet" with a predefined facet

Xavier
In a way I agree that it would be easier to do that but i really wants to avoid this solution because it prefer to work "harder" on preparing my index than adding field requests on my front query :)

So the only solution i see right now is to do that on my own in order to have my database fully prepared to be indexed ... but i had hope that solr could handle it ... so if anyone see any solution to handle it directly with solr you are welcome :p

Anyways thanks for your help Em ;)

Best regards,
Xavier
Em
Reply | Threaded
Open this post in threaded view
|

Re: How to merge an "autofacet" with a predefined facet

Em
Well, you could create a keyword-file out of your database and join it
with your self-maintained keywordslist.
Doing so, keep in mind that you have to reload your SolrCore in order to
make the changes visible to the indexing-process (and keep in mind that
you have to reindex those documents that match your new keywordslist but
currently do not have those keywords assigned).

Kind regards,
Em

Am 21.02.2012 19:53, schrieb Xavier:

> In a way I agree that it would be easier to do that but i really wants to
> avoid this solution because it prefer to work "harder" on preparing my index
> than adding field requests on my front query :)
>
> So the only solution i see right now is to do that on my own in order to
> have my database fully prepared to be indexed ... but i had hope that solr
> could handle it ... so if anyone see any solution to handle it directly with
> solr you are welcome :p
>
> Anyways thanks for your help Em ;)
>
> Best regards,
> Xavier
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-merge-an-autofacet-with-a-predefined-facet-tp3763988p3764506.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: How to merge an "autofacet" with a predefined facet

Chris Hostetter-3
In reply to this post by Xavier

: But i don't know if it's possible to merge this "autocreated" facet with a
: facet already predefined ? i tried to used <copyField> (adding this to my
: code in my previous post) :
: *<copyField source="text_tag_facet" dest="predefined_facet" />*

copyField applies to the raw input of those fields -- so the special logic
you have in the analyzer for your text_tag_facet won't be applied yet when
it's copied to your predefined_facet field (copyField happens first)

: It's maybe because (As I understood) the real (stored) value of this dynamic
: facet is still the initial fulltext  ?? (or maybe i'm wrong ...)

stored values are differnet from indexed values -- but stored values are
also not ever a factor in dealing with faceting, the stored value is just
what is returned when you get results back (ie: the "doc list") ... your
problem has nothing to do with stored values.


-Hoss
Reply | Threaded
Open this post in threaded view
|

Re: How to merge an "autofacet" with a predefined facet

Xavier
In reply to this post by Em
I'm not sure to understand your solution ?

When (and how) will be the 'word' detection in the fulltext ? before (by my own) or during (with) solr indexation ?
Em
Reply | Threaded
Open this post in threaded view
|

Re: How to merge an "autofacet" with a predefined facet

Em
If you use the suggested solution, it will detect the words at indexing
time.
However, Solr's FilterFactory's lifecycle keeps no track on whether a
file for synonyms, keywords etc. has been changed since Solr's last startup.
Therefore a change within these files is not visible until you reload
your core.

Furthermore keywords for old documents aren't added automatically if you
change your keywords (and reload the core) - you have to write a routine
that finds documents matching the new keywords and reindex those documents.

Example:

Your keywordslist at time t1 contains two words:
keyword
codeword

You are indexing two documents:
doc1: {"content":"I am about a secret codeword."}
doc1: {"content":"Happy keyword and the gang."}

Your filter will mark "codeword" in doc1 and "keyword" in doc2 as words
to keep and remove everything else. Therefore their content for your
keepWordField contains only

doc1: {"indexedContent":"codeword"}
doc2: {"indexedContent":"keyword"}

However, if you add the word "gang" to your keywordlist AND reload your
SolrCore, doc2 will still only contain the term "keyword" until it gets
reindexed again.

Kind regards,
Em

Am 22.02.2012 17:56, schrieb Xavier:
> I'm not sure to understand your solution ?
>
> When (and how) will be the 'word' detection in the fulltext ? before (by my
> own) or during (with) solr indexation ?
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-merge-an-autofacet-with-a-predefined-facet-tp3763988p3767059.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Em
Reply | Threaded
Open this post in threaded view
|

Re: How to merge an "autofacet" with a predefined facet

Em
In reply to this post by Xavier
Btw.:
Solr has no downtime while reloading the core.
It loads the new core and while loading the new one it still serves
requests with the old one.
When the new one is ready (and warmed up) it finally replaces the old core.

Best,
Em

Am 22.02.2012 17:56, schrieb Xavier:
> I'm not sure to understand your solution ?
>
> When (and how) will be the 'word' detection in the fulltext ? before (by my
> own) or during (with) solr indexation ?
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/How-to-merge-an-autofacet-with-a-predefined-facet-tp3763988p3767059.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
Reply | Threaded
Open this post in threaded view
|

Re: How to merge an "autofacet" with a predefined facet

Xavier
Thank you for theses informations, I'll keep that in mind.

But i'm sorry, i don't get it about the process to do it ???

Em wrote
Well, you could create a keyword-file out of your database and join it
with your self-maintained keywordslist.

By that you mean :
- 'self-maintained keywordslist' is my 'predefined_facet' already filled in database that i'll still import with DIH ?
- The keyword-file isnt the same thing that i've created with synonyms/keepsword combination ?

And still don't get how to 'merge' those both way of getting facets values in an only one facet !

Thanks for advance,
Xavier