question related to solr LTR plugin

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

question related to solr LTR plugin

Saurabh Agarwal (BLOOMBERG/ 731 LEX)
Hi,

I do have a question related to solr LTR plugin. I have a use case of personalization and wondering whether you can help me there. I would like to rerank my query based on the relationship of searcher with the author of the returned documents. I do have relationship score in the external datastore in form of user1(searcher), user2(author), relationship score. In my query, I can pass searcher id as external feature. My question is that during querying, how do I retrieve relationship score for each documents as a feature and rerank the documents. Would I need to implement a custom feature to do so? and How to implement the custom feature.

Thanks,
Saurabh
Reply | Threaded
Open this post in threaded view
|

Re: question related to solr LTR plugin

Michael Nilsson
Hey Saurabh,

So there are a few things you can do to with the LTR plugin and your Solr
collection to solve different degrees of the kind of personalization you
might want.  I'll start with the simplest, which isn't exactly what you're
looking for but is very quick to implement and play around with, and move
on to some more complex features you could write.


   1. Make a simple binary feature to see if the user is the author using
   the existing SolrFeature class.
   {
       "store" : "myFeatureStore",
       "name" : "isUserTheDocumentAuthor ",
       "class" : "org.apache.solr.ltr.feature.SolrFeature",
       "params" : {
   "fq" : [ "{!field f=authorName}${userName}" ]
   }
   } Pass in the userName at request time using external feature
   information (efi)
   ...&rq={!ltr model="myModel" reRankDocs=20 efi.userName="Saurabh"}
   2. Similar to the previous one, except this time instead of seeing if
   the user is the document's author, see if the user has a relationship to
   the document's author.  The only difference here is that you would add a
   new multivalued field to your document which could index all the author's
   relationships that you would match against.
   {
       "store" : "myFeatureStore",
       "name" : "doesUserHaveRelationshipToDocumentAuthor ",
       "class" : "org.apache.solr.ltr.feature.SolrFeature",
       "params" : {
   "fq" : [ "{!field f=authorRelationships}${userName}" ]
   }
   } Ranking request doesn't change
   ...&rq={!ltr model="myModel" reRankDocs=20 efi.userName="Saurabh"}
   3. Implement your own custom feature by subclassing Feature.java
   <http://lucene.apache.org/solr/6_4_0/solr-ltr/org/apache/solr/ltr/feature/Feature.html>.
   You can look at the implementation of ValueFeature.java
   <http://lucene.apache.org/solr/6_4_0/solr-ltr/org/apache/solr/ltr/feature/ValueFeature.html>
for
   a simple concrete example of how to do that.  Depending on how large your
   user-user map is, and how often it changes, you could do it in a couple
   different ways.

   If the map is pretty static and relatively small, you can upload it
   along with your feature definition in the params section.  Your custom
   feature can parse that dump once at creation time, and then at query time
   you can look up the value of userName+authorName in your map and return
   that in the score() function for the feature's value.  If it wasn't found
   in the map you could return a default value of 0.

   If the map changes more frequently, you could instead pass in only that
   user's relationship map at request time through an efi.  Your custom
   feature can parse that user's specific relationship map (based on whatever
   format you send it in) and then return the authorName lookup value in the
   score() function.

   If neither of those options are good for you, you could of course just
   make your custom feature class do whatever you want and lookup the
   appropriate data in whatever store you have it in.  You could index this
   info in a parallel collection and look it up there.  Solr recently added
   jdbc driver support, so you might even be able to use that to get data from
   a sql store as well.  I haven't messed around with these two options myself
   so you'd be treading somewhat new ground.  If you or anyone does test this
   out and it seems to work nicely, I think this would make an
*excellent *generic
   feature class to contribute back to the community.

Hope that helps,
Michael
















On Mon, Mar 6, 2017 at 4:24 PM, Saurabh Agarwal (BLOOMBERG/ 731 LEX) <
[hidden email]> wrote:

> Hi,
>
> I do have a question related to solr LTR plugin. I have a use case of
> personalization and wondering whether you can help me there. I would like
> to rerank my query based on the relationship of searcher with the author of
> the returned documents. I do have relationship score in the external
> datastore in form of user1(searcher), user2(author), relationship score. In
> my query, I can pass searcher id as external feature. My question is that
> during querying, how do I retrieve relationship score for each documents as
> a feature and rerank the documents. Would I need to implement a custom
> feature to do so? and How to implement the custom feature.
>
> Thanks,
> Saurabh