Bugs with Re-ranking/LtR and ExplainAugmenterFactory

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Bugs with Re-ranking/LtR and ExplainAugmenterFactory

Sambhav Kothari (BLOOMBERG/ LONDON)

Currently, if we use the ExplainAugmenterFactory with LtR, instead of using the model/re-rankers explain method, it uses the default query explain (tf-idf explanation). This happens because the BasicResultContext doesn't wrap the query(https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/solr/core/src/java/org/apache/solr/response/BasicResultContext.java#L67)  with the RankQuery when its set to context's query, which is then used by the ExplainAugmenterFactory. (https://github.com/apache/lucene-solr/blob/1d85cd783863f75cea133fb9c452302214165a4d/solr/core/src/java/org/apache/solr/response/transform/ExplainAugmenterFactory.java#L111).

As a result there are discrepancies between queries like -

http://localhost:8983/solr/collection1/select?q=*:*&collection=collectionName&wt=json&fl=[explain style=nl],score&rq={!ltr model=linear-model}

http://localhost:8983/solr/collection1/select?q=*:*&collection=collectionName&wt=json&fl=score&rq={!ltr model=linear-model}&debugQuery=true

the former outputs the explain from the SimilarityScorer's explain while the latter uses the correct LtR ModelScorer's explain.

There are a few other problems with the explain augmenter - for eg. it doesn't work with grouping (although the other doc transformers like LtR's LTRFeatureLoggerTransformerFactory work with grouping).

Just wanted to discuss these issues before creating tickets on Jira.