Explain Scoring function in LMJelinekMercerSimilarity Class

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Explain Scoring function in LMJelinekMercerSimilarity Class

Dwaipayan Roy
Hello,

Can anyone help me understand the scoring function in the
LMJelinekMercerSimilarity class?

The scoring function in LMJelinekMercerSimilarity is shown below:
--------------------------------------------------------
float score = stats.getTotalBoost() *
(float)Math.log(1 + ((1 - lambda) * freq / docLen) / (lambda *
((LMStats)stats).getCollectionProbability()));
--------------------------------------------------------

Can anyone help explain the equation? I can understand the scoring effect
when calculating the stat in the document, i.e.: (1 - lambda) * freq /
docLen).

I hope getCollectionProbability() returns col_freq(t) / col_size. Am I
right?

Also the boosting part is not clear to me (stats.getTotalBoost()).

I want to reproduce the result of the scoring using LM-JM. Hence I want the
details.

Thanks.
Dwaipayan Roy..
Reply | Threaded
Open this post in threaded view
|

Re: Explain Scoring function in LMJelinekMercerSimilarity Class

wmartinusa
https://doi.org/10.3115/981574.981579



On 12/20/2016 12:21 PM, Dwaipayan Roy wrote:

> Hello,
>
> Can anyone help me understand the scoring function in the
> LMJelinekMercerSimilarity class?
>
> The scoring function in LMJelinekMercerSimilarity is shown below:
> --------------------------------------------------------
> float score = stats.getTotalBoost() *
> (float)Math.log(1 + ((1 - lambda) * freq / docLen) / (lambda *
> ((LMStats)stats).getCollectionProbability()));
> --------------------------------------------------------
>
> Can anyone help explain the equation? I can understand the scoring effect
> when calculating the stat in the document, i.e.: (1 - lambda) * freq /
> docLen).
>
> I hope getCollectionProbability() returns col_freq(t) / col_size. Am I
> right?
>
> Also the boosting part is not clear to me (stats.getTotalBoost()).
>
> I want to reproduce the result of the scoring using LM-JM. Hence I want the
> details.
>
> Thanks.
> Dwaipayan Roy..
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Explain Scoring function in LMJelinekMercerSimilarity Class

Dwaipayan Roy
In reply to this post by Dwaipayan Roy
Waiting for an explanation for my query. Thank you very much.

On Tue, Dec 20, 2016 at 10:51 PM, Dwaipayan Roy <[hidden email]>
wrote:

> Hello,
>
> Can anyone help me understand the scoring function in the
> LMJelinekMercerSimilarity class?
>
> The scoring function in LMJelinekMercerSimilarity is shown below:
> --------------------------------------------------------
> float score = stats.getTotalBoost() *
> (float)Math.log(1 + ((1 - lambda) * freq / docLen) / (lambda *
> ((LMStats)stats).getCollectionProbability()));
> --------------------------------------------------------
>
> Can anyone help explain the equation? I can understand the scoring effect
> when calculating the stat in the document, i.e.: (1 - lambda) * freq /
> docLen).
>
> I hope getCollectionProbability() returns col_freq(t) / col_size. Am I
> right?
>
> Also the boosting part is not clear to me (stats.getTotalBoost()).
>
> I want to reproduce the result of the scoring using LM-JM. Hence I want
> the details.
>
> Thanks.
> Dwaipayan Roy..
>



--
Dwaipayan Roy.