retrieve document boost

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

retrieve document boost

Ryan McKinley
I'm trying to make sure the document boost i'm sending in is actually
getting used.  I don't see it showing up anywhere.  To illustrate, I
augmented the BasicFunctionalityTest.testDocBoost() with:

public void testDocBoost() throws Exception {
   ...

    LocalSolrQueryRequest lqr = lrf.makeRequest("q", "hello");
    Term term = new Term( "id", "2" );
    int docID = lqr.getSearcher().getFirstMatch( term );
    assertTrue( "term not found", docID >= 0 );
    Document doc = lqr.getSearcher().doc( docID );
    assertNotNull( "document not found", doc );
    assertEquals( "incorrect boost.", 2.0f, doc.getBoost() );
}

it retrieves the document, but gets the wrong boost:  "expected:<2.0>
but was:<1.0>"

Can you get the boost of an indexed document?  Am I missing something
basic?  Is the stored document boost lost once it is indexed?

thanks
ryan
Reply | Threaded
Open this post in threaded view
|

Re: retrieve document boost

Chris Hostetter-3

: Can you get the boost of an indexed document?  Am I missing something
: basic?  Is the stored document boost lost once it is indexed?

Bingo.  In Lucene, Document boosts aren't stored in the docs for later
recovered - the getBoost method is meaningless from a Document returned by
a search (or retrieved from an IndexReader)

Background on my bitterness about this...
http://www.nabble.com/Re%3A--jira--Commented%3A-%28LUCENE-778%29-Allow-overriding-a-Document-p8418224.html

Boosts are folded into the fieldNorm - doc boosts are folded into the
fieldNorm of every field the doc has (that doesn't have OMIT_NORMS used)



-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: retrieve document boost

Ryan McKinley
On 2/20/07, Chris Hostetter <[hidden email]> wrote:
>
> : Can you get the boost of an indexed document?  Am I missing something
> : basic?  Is the stored document boost lost once it is indexed?
>
> Bingo.  In Lucene, Document boosts aren't stored in the docs for later
> recovered - the getBoost method is meaningless from a Document returned by
> a search (or retrieved from an IndexReader)
>

Aaaah, thanks.
Reply | Threaded
Open this post in threaded view
|

Re: retrieve document boost

Brian Whitman
In reply to this post by Chris Hostetter-3
On Feb 20, 2007, at 2:59 PM, Chris Hostetter wrote:

> In Lucene, Document boosts aren't stored in the docs for later
> recovered - the getBoost method is meaningless from a Document  
> returned by
> a search (or retrieved from an IndexReader)
>
> Boosts are folded into the fieldNorm - doc boosts are folded into the
> fieldNorm of every field the doc has (that doesn't have OMIT_NORMS  
> used)


Chris-- then how does Luke show the document boost? Does it compute  
it somehow?

If I set a doc boost in Solr, should there be a field boost = to the  
doc boost for each field where omitNorms=false during the explain  
chunk? Because I am, and there's not. :) It doesn't show in Luke either.

-Brian




Reply | Threaded
Open this post in threaded view
|

Re: retrieve document boost

Mike Klaas
On 2/20/07, Brian Whitman <[hidden email]> wrote:

> On Feb 20, 2007, at 2:59 PM, Chris Hostetter wrote:
>
> > In Lucene, Document boosts aren't stored in the docs for later
> > recovered - the getBoost method is meaningless from a Document
> > returned by
> > a search (or retrieved from an IndexReader)
> >
> > Boosts are folded into the fieldNorm - doc boosts are folded into the
> > fieldNorm of every field the doc has (that doesn't have OMIT_NORMS
> > used)
>
>
> Chris-- then how does Luke show the document boost? Does it compute
> it somehow?

If you calculate the length of a document field, run it through the
Similarity used for indexing (lenghtNorm), and multiply its reciprocal
with the existing fieldNorm, you get the doc boost.

> If I set a doc boost in Solr, should there be a field boost = to the
> doc boost for each field where omitNorms=false during the explain
> chunk? Because I am, and there's not. :) It doesn't show in Luke either.

If omitNorms=false on a field, the document boost and field boost are
moot.  Unfortunate, but true.

-Mike
Reply | Threaded
Open this post in threaded view
|

Re: retrieve document boost

Brian Whitman
> If omitNorms=false on a field, the document boost and field boost are
> moot.  Unfortunate, but true.

hm? The reason I have omitNorms=false on for a field is because I  
want the boost. I did so because I read here:
http://wiki.apache.org/solr/UpdateXmlMessages

"NOTE: make sure norms are enabled (omitNorms="false" in the  
schema.xml) for any fields where the index-time boost should be stored."

-brian

Reply | Threaded
Open this post in threaded view
|

Re: retrieve document boost

Mike Klaas
On 2/20/07, Brian Whitman <[hidden email]> wrote:
> > If omitNorms=false on a field, the document boost and field boost are
> > moot.  Unfortunate, but true.
>
> hm? The reason I have omitNorms=false on for a field is because I
> want the boost. I did so because I read here:
> http://wiki.apache.org/solr/UpdateXmlMessages
>
> "NOTE: make sure norms are enabled (omitNorms="false" in the
> schema.xml) for any fields where the index-time boost should be stored."

Sorry, I inverted the logic in my head. You're right.

I believe that docBoost does not translate into a fieldBoost display
(which is only the query-time boost), but is factored into the
fieldNorm.

-Mike
Reply | Threaded
Open this post in threaded view
|

Re: retrieve document boost

kkrugler
In reply to this post by Chris Hostetter-3
>: Can you get the boost of an indexed document?  Am I missing something
>: basic?  Is the stored document boost lost once it is indexed?
>
>Bingo.  In Lucene, Document boosts aren't stored in the docs for later
>recovered - the getBoost method is meaningless from a Document returned by
>a search (or retrieved from an IndexReader)
>
>Background on my bitterness about this...
>http://www.nabble.com/Re%3A--jira--Commented%3A-%28LUCENE-778%29-Allow-overriding-a-Document-p8418224.html
>
>Boosts are folded into the fieldNorm - doc boosts are folded into the
>fieldNorm of every field the doc has (that doesn't have OMIT_NORMS used)

Ah - yes. That would explain the oddity I saw with the order of hits
not matching the document boost, since I was querying against a field
that had omit norms (because it's really short).

-- Ken
--
Ken Krugler
Krugle, Inc.
+1 530-210-6378
"Find Code, Find Answers"
Reply | Threaded
Open this post in threaded view
|

Re: retrieve document boost

Brian Whitman
In reply to this post by Mike Klaas
On Feb 20, 2007, at 4:09 PM, Mike Klaas wrote:

> Sorry, I inverted the logic in my head. You're right.

You're not alone, I do it all the time too.

> I believe that docBoost does not translate into a fieldBoost display
> (which is only the query-time boost), but is factored into the
> fieldNorm.
>

OK, that's enough confirmation for me.. I'll feel safe that  
somewhere, somehow, my doc boosts are being used.

thanks
Brian