group.ngroups is set to an incorrect value - specific field types

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

group.ngroups is set to an incorrect value - specific field types

海老澤 志信
Hi,

I'm using Solr version 4.1.
I found a bug in group.ngroups. So could anyone kindly take a look at my bug report?

If I specify the type Double as group.field, the value of group.ngroups is set to be an incorrect value.

[condition]
- Double is defined in group.field
- Documents without the field which is defined as group.field,

[Sample query and Example]
-------------------------------
solr/select?q=*:*&group=true&group.ngroups=true&group.field=Double_Field

* "Double_Field" is defined "solr.TrieDoubleField" type.
-------------------------------
When documents with group.field are 4 and documents without group.field are 6,
then it turns out 10 of group.ngroups as result of the query.

But I think that group.ngroups should be 5 rightly in this case.

[Root Cause]
It seems there is a bug in the source code of Lucene.
There is a function that compares a list of whether these groups contain the same group.field,
It calls MutableValueDouble.compareSameType().

See below the point which seems to be a root cause.
-----------------------------
    if (!exists) return -1;
    if (!b.exists) return 1;
-----------------------------
If "exists" is false, it return -1.

But I think it should return 0, when "exists" and "b.exists" are equal.

[Similar problem]
There is a similar problem to MutableValueBool.compareSameType().
Therefore, when you grouping the field of type Boolean (solr.BoolField),
value of group.ngroups is always 0 or 1 .

[Solution]
I propose the following modifications: MutableValueDouble.compareSameType()

===================================================================
--- MutableValueDouble.java
+++ MutableValueDouble.java
@@ -54,9 +54,8 @@
     MutableValueDouble b = (MutableValueDouble)other;
     int c = Double.compare(value, b.value);
     if (c != 0) return c;
-    if (!exists) return -1;
-    if (!b.exists) return 1;
-    return 0;
+    if (exists == b.exists) return 0;
+    return exists ? 1 : -1;
   }
===================================================================

I propose the following modifications: MutableValueBool.compareSameType()

===================================================================
--- MutableValueBool.java
+++ MutableValueBool.java
@@ -52,7 +52,7 @@
   @Override
   public int compareSameType(Object other) {
     MutableValueBool b = (MutableValueBool)other;
-    if (value != b.value) return value ? 1 : 0;
+    if (value != b.value) return value ? 1 : -1;
     if (exists == b.exists) return 0;
     return exists ? 1 : -1;
   }
===================================================================


Thanks,

Ebisawa

Reply | Threaded
Open this post in threaded view
|

RE: group.ngroups is set to an incorrect value - specific field types

海老澤 志信
Hi all

Could anyone have comments on my bug report?

Regards,
Ebisawa


>-----Original Message-----
>From: 海老澤 志信
>Sent: Friday, June 13, 2014 7:45 PM
>To: '[hidden email]'
>Subject: group.ngroups is set to an incorrect value - specific field types
>
>Hi,
>
>I'm using Solr version 4.1.
>I found a bug in group.ngroups. So could anyone kindly take a look at my
>bug report?
>
>If I specify the type Double as group.field, the value of group.ngroups is
>set to be an incorrect value.
>
>[condition]
>- Double is defined in group.field
>- Documents without the field which is defined as group.field,
>
>[Sample query and Example]
>-------------------------------
>solr/select?q=*:*&group=true&group.ngroups=true&group.field=Double_Fiel
>d
>
>* "Double_Field" is defined "solr.TrieDoubleField" type.
>-------------------------------
>When documents with group.field are 4 and documents without group.field are
>6,
>then it turns out 10 of group.ngroups as result of the query.
>
>But I think that group.ngroups should be 5 rightly in this case.
>
>[Root Cause]
>It seems there is a bug in the source code of Lucene.
>There is a function that compares a list of whether these groups contain
>the same group.field,
>It calls MutableValueDouble.compareSameType().
>
>See below the point which seems to be a root cause.
>-----------------------------
>    if (!exists) return -1;
>    if (!b.exists) return 1;
>-----------------------------
>If "exists" is false, it return -1.
>
>But I think it should return 0, when "exists" and "b.exists" are equal.
>
>[Similar problem]
>There is a similar problem to MutableValueBool.compareSameType().
>Therefore, when you grouping the field of type Boolean (solr.BoolField),
>value of group.ngroups is always 0 or 1 .
>
>[Solution]
>I propose the following modifications:
>MutableValueDouble.compareSameType()
>
>===================================================================
>--- MutableValueDouble.java
>+++ MutableValueDouble.java
>@@ -54,9 +54,8 @@
>     MutableValueDouble b = (MutableValueDouble)other;
>     int c = Double.compare(value, b.value);
>     if (c != 0) return c;
>-    if (!exists) return -1;
>-    if (!b.exists) return 1;
>-    return 0;
>+    if (exists == b.exists) return 0;
>+    return exists ? 1 : -1;
>   }
>===================================================================
>
>I propose the following modifications: MutableValueBool.compareSameType()
>
>===================================================================
>--- MutableValueBool.java
>+++ MutableValueBool.java
>@@ -52,7 +52,7 @@
>   @Override
>   public int compareSameType(Object other) {
>     MutableValueBool b = (MutableValueBool)other;
>-    if (value != b.value) return value ? 1 : 0;
>+    if (value != b.value) return value ? 1 : -1;
>     if (exists == b.exists) return 0;
>     return exists ? 1 : -1;
>   }
>===================================================================
>
>
>Thanks,
>
>Ebisawa

Reply | Threaded
Open this post in threaded view
|

Re: group.ngroups is set to an incorrect value - specific field types

alxsss
Hi,


I see similar problem in our solr application. Sometime it gives number in a group as number of all documents. This starting to happen after upgrade from 4.6.1 to 4.8.1


Thanks.
Alex.



-----Original Message-----
From: 海老澤 志信 <[hidden email]>
To: solr-user <[hidden email]>
Sent: Tue, Jun 17, 2014 5:24 am
Subject: RE: group.ngroups is set to an incorrect value - specific field types


Hi all

Could anyone have comments on my bug report?

Regards,
Ebisawa


>-----Original Message-----
>From: 海老澤 志信
>Sent: Friday, June 13, 2014 7:45 PM
>To: '[hidden email]'
>Subject: group.ngroups is set to an incorrect value - specific field types
>
>Hi,
>
>I'm using Solr version 4.1.
>I found a bug in group.ngroups. So could anyone kindly take a look at my
>bug report?
>
>If I specify the type Double as group.field, the value of group.ngroups is
>set to be an incorrect value.
>
>[condition]
>- Double is defined in group.field
>- Documents without the field which is defined as group.field,
>
>[Sample query and Example]
>-------------------------------
>solr/select?q=*:*&group=true&group.ngroups=true&group.field=Double_Fiel
>d
>
>* "Double_Field" is defined "solr.TrieDoubleField" type.
>-------------------------------
>When documents with group.field are 4 and documents without group.field are
>6,
>then it turns out 10 of group.ngroups as result of the query.
>
>But I think that group.ngroups should be 5 rightly in this case.
>
>[Root Cause]
>It seems there is a bug in the source code of Lucene.
>There is a function that compares a list of whether these groups contain
>the same group.field,
>It calls MutableValueDouble.compareSameType().
>
>See below the point which seems to be a root cause.
>-----------------------------
>    if (!exists) return -1;
>    if (!b.exists) return 1;
>-----------------------------
>If "exists" is false, it return -1.
>
>But I think it should return 0, when "exists" and "b.exists" are equal.
>
>[Similar problem]
>There is a similar problem to MutableValueBool.compareSameType().
>Therefore, when you grouping the field of type Boolean (solr.BoolField),
>value of group.ngroups is always 0 or 1 .
>
>[Solution]
>I propose the following modifications:
>MutableValueDouble.compareSameType()
>
>===================================================================
>--- MutableValueDouble.java
>+++ MutableValueDouble.java
>@@ -54,9 +54,8 @@
>     MutableValueDouble b = (MutableValueDouble)other;
>     int c = Double.compare(value, b.value);
>     if (c != 0) return c;
>-    if (!exists) return -1;
>-    if (!b.exists) return 1;
>-    return 0;
>+    if (exists == b.exists) return 0;
>+    return exists ? 1 : -1;
>   }
>===================================================================
>
>I propose the following modifications: MutableValueBool.compareSameType()
>
>===================================================================
>--- MutableValueBool.java
>+++ MutableValueBool.java
>@@ -52,7 +52,7 @@
>   @Override
>   public int compareSameType(Object other) {
>     MutableValueBool b = (MutableValueBool)other;
>-    if (value != b.value) return value ? 1 : 0;
>+    if (value != b.value) return value ? 1 : -1;
>     if (exists == b.exists) return 0;
>     return exists ? 1 : -1;
>   }
>===================================================================
>
>
>Thanks,
>
>Ebisawa


 
Reply | Threaded
Open this post in threaded view
|

Re: group.ngroups is set to an incorrect value - specific field types

Chris Hostetter-3
In reply to this post by 海老澤 志信

I think you are correct -- deinitely looks like a bug to me...

https://issues.apache.org/jira/browse/LUCENE-5790

: Date: Fri, 13 Jun 2014 10:45:12 +0000
: From: 海老澤 志信 <[hidden email]>
: Reply-To: [hidden email]
: To: "[hidden email]" <[hidden email]>
: Subject: group.ngroups is set to an incorrect value - specific field types
:
: Hi,
:
: I'm using Solr version 4.1.
: I found a bug in group.ngroups. So could anyone kindly take a look at my bug report?
:
: If I specify the type Double as group.field, the value of group.ngroups is set to be an incorrect value.
:
: [condition]
: - Double is defined in group.field
: - Documents without the field which is defined as group.field,
:
: [Sample query and Example]
: -------------------------------
: solr/select?q=*:*&group=true&group.ngroups=true&group.field=Double_Field
:
: * "Double_Field" is defined "solr.TrieDoubleField" type.
: -------------------------------
: When documents with group.field are 4 and documents without group.field are 6,
: then it turns out 10 of group.ngroups as result of the query.
:
: But I think that group.ngroups should be 5 rightly in this case.
:
: [Root Cause]
: It seems there is a bug in the source code of Lucene.
: There is a function that compares a list of whether these groups contain the same group.field,
: It calls MutableValueDouble.compareSameType().
:
: See below the point which seems to be a root cause.
: -----------------------------
:     if (!exists) return -1;
:     if (!b.exists) return 1;
: -----------------------------
: If "exists" is false, it return -1.
:
: But I think it should return 0, when "exists" and "b.exists" are equal.
:
: [Similar problem]
: There is a similar problem to MutableValueBool.compareSameType().
: Therefore, when you grouping the field of type Boolean (solr.BoolField),
: value of group.ngroups is always 0 or 1 .
:
: [Solution]
: I propose the following modifications: MutableValueDouble.compareSameType()
:
: ===================================================================
: --- MutableValueDouble.java
: +++ MutableValueDouble.java
: @@ -54,9 +54,8 @@
:      MutableValueDouble b = (MutableValueDouble)other;
:      int c = Double.compare(value, b.value);
:      if (c != 0) return c;
: -    if (!exists) return -1;
: -    if (!b.exists) return 1;
: -    return 0;
: +    if (exists == b.exists) return 0;
: +    return exists ? 1 : -1;
:    }
: ===================================================================
:
: I propose the following modifications: MutableValueBool.compareSameType()
:
: ===================================================================
: --- MutableValueBool.java
: +++ MutableValueBool.java
: @@ -52,7 +52,7 @@
:    @Override
:    public int compareSameType(Object other) {
:      MutableValueBool b = (MutableValueBool)other;
: -    if (value != b.value) return value ? 1 : 0;
: +    if (value != b.value) return value ? 1 : -1;
:      if (exists == b.exists) return 0;
:      return exists ? 1 : -1;
:    }
: ===================================================================
:
:
: Thanks,
:
: Ebisawa
:
:

-Hoss
http://www.lucidworks.com/