NPE during spell checking when result collapsing is activated and local parameters are used

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

NPE during spell checking when result collapsing is activated and local parameters are used

Stefan Walter
Hi!

I have an issue with Solr 7.3.1 in the spell checking component:

java.lang.NullPointerException at
org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1021)
at
org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1081)
at
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:230)
at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1602)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1419)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:584)
...

I have found an issue that addresses a similiar problem:
https://issues.apache.org/jira/browse/SOLR-8807

The fix, which was introduced with this issue seems to miss our situation,
though. The relevant part of the query is this:

&fq={!tag=collapser}{!collapse field=productId sort='merchantOrder asc,
price asc, id asc'}

When I remove the local parameter {!tag=collapser} the collation works
fine. Looking at the diff of the commit of the issue mentioned above, it
seems that the "startsWith" could be the problem:

+        // Collate testing does not support the Collapse QParser (See
SOLR-8807)
+        params.remove("expand");
+        String[] filters = params.getParams(CommonParams.FQ);
+        if (filters != null) {
+          List<String> filtersToApply = new ArrayList<>(filters.length);
+          for (String fq : filters) {
+            if (!fq.startsWith("{!collapse")) {
+              filtersToApply.add(fq);
+            }
+          }
+          params.set("fq", filtersToApply.toArray(new
String[filtersToApply.size()]));
+        }

Can someone confirm this? I would open a bug ticket then. (Since the code
is unchanged in the latest version.)

Thanks,
Stefan
Reply | Threaded
Open this post in threaded view
|

Re: NPE during spell checking when result collapsing is activated and local parameters are used

Tomás Fernández Löbbe
I believe your syntax is incorrect. I believe local params must all be
included in between the same {!...}, and "{!" can only be at the beginning

have you tried:

&fq={!collapse tag=collapser field=productId sort='merchantOrder asc,
price asc, id asc'}



On Thu, Nov 14, 2019 at 4:54 AM Stefan Walter <[hidden email]> wrote:

> Hi!
>
> I have an issue with Solr 7.3.1 in the spell checking component:
>
> java.lang.NullPointerException at
>
> org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1021)
> at
>
> org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1081)
> at
>
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:230)
> at
>
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1602)
> at
>
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1419)
> at
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:584)
> ...
>
> I have found an issue that addresses a similiar problem:
> https://issues.apache.org/jira/browse/SOLR-8807
>
> The fix, which was introduced with this issue seems to miss our situation,
> though. The relevant part of the query is this:
>
> &fq={!tag=collapser}{!collapse field=productId sort='merchantOrder asc,
> price asc, id asc'}
>
> When I remove the local parameter {!tag=collapser} the collation works
> fine. Looking at the diff of the commit of the issue mentioned above, it
> seems that the "startsWith" could be the problem:
>
> +        // Collate testing does not support the Collapse QParser (See
> SOLR-8807)
> +        params.remove("expand");
> +        String[] filters = params.getParams(CommonParams.FQ);
> +        if (filters != null) {
> +          List<String> filtersToApply = new ArrayList<>(filters.length);
> +          for (String fq : filters) {
> +            if (!fq.startsWith("{!collapse")) {
> +              filtersToApply.add(fq);
> +            }
> +          }
> +          params.set("fq", filtersToApply.toArray(new
> String[filtersToApply.size()]));
> +        }
>
> Can someone confirm this? I would open a bug ticket then. (Since the code
> is unchanged in the latest version.)
>
> Thanks,
> Stefan
>
Reply | Threaded
Open this post in threaded view
|

Re: NPE during spell checking when result collapsing is activated and local parameters are used

Stefan Walter
Indeed, you are right. Interestingly, it generally worked with the two {!
..} in the filter query - besides the problem with the collations, of
course. Therefore I never questioned it...

Thank you!
Stefan


Am 15. November 2019 um 00:01:52, Tomás Fernández Löbbe (
[hidden email]) schrieb:

I believe your syntax is incorrect. I believe local params must all be
included in between the same {!...}, and "{!" can only be at the beginning

have you tried:

&fq={!collapse tag=collapser field=productId sort='merchantOrder asc,
price asc, id asc'}



On Thu, Nov 14, 2019 at 4:54 AM Stefan Walter <[hidden email]> wrote:

> Hi!
>
> I have an issue with Solr 7.3.1 in the spell checking component:
>
> java.lang.NullPointerException at
>
>
org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1021)

> at
>
>
org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1081)

> at
>
>
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:230)

> at
>
>
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1602)

> at
>
>
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1419)

> at
>
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:584)
> ...
>
> I have found an issue that addresses a similiar problem:
> https://issues.apache.org/jira/browse/SOLR-8807
>
> The fix, which was introduced with this issue seems to miss our
situation,

> though. The relevant part of the query is this:
>
> &fq={!tag=collapser}{!collapse field=productId sort='merchantOrder asc,
> price asc, id asc'}
>
> When I remove the local parameter {!tag=collapser} the collation works
> fine. Looking at the diff of the commit of the issue mentioned above, it
> seems that the "startsWith" could be the problem:
>
> + // Collate testing does not support the Collapse QParser (See
> SOLR-8807)
> + params.remove("expand");
> + String[] filters = params.getParams(CommonParams.FQ);
> + if (filters != null) {
> + List<String> filtersToApply = new ArrayList<>(filters.length);
> + for (String fq : filters) {
> + if (!fq.startsWith("{!collapse")) {
> + filtersToApply.add(fq);
> + }
> + }
> + params.set("fq", filtersToApply.toArray(new
> String[filtersToApply.size()]));
> + }
>
> Can someone confirm this? I would open a bug ticket then. (Since the code
> is unchanged in the latest version.)
>
> Thanks,
> Stefan
>
Reply | Threaded
Open this post in threaded view
|

Re: NPE during spell checking when result collapsing is activated and local parameters are used

Tomás Fernández Löbbe
Would you create a Jira issue anyway tu fix the fact that it NPE instead of
throwing a bad request?

On Fri, Nov 15, 2019 at 2:31 AM Stefan Walter <[hidden email]> wrote:

> Indeed, you are right. Interestingly, it generally worked with the two {!
> ..} in the filter query - besides the problem with the collations, of
> course. Therefore I never questioned it...
>
> Thank you!
> Stefan
>
>
> Am 15. November 2019 um 00:01:52, Tomás Fernández Löbbe (
> [hidden email]) schrieb:
>
> I believe your syntax is incorrect. I believe local params must all be
> included in between the same {!...}, and "{!" can only be at the beginning
>
> have you tried:
>
> &fq={!collapse tag=collapser field=productId sort='merchantOrder asc,
> price asc, id asc'}
>
>
>
> On Thu, Nov 14, 2019 at 4:54 AM Stefan Walter <[hidden email]> wrote:
>
> > Hi!
> >
> > I have an issue with Solr 7.3.1 in the spell checking component:
> >
> > java.lang.NullPointerException at
> >
> >
>
> org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1021)
>
> > at
> >
> >
>
> org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1081)
>
> > at
> >
> >
>
> org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:230)
>
> > at
> >
> >
>
> org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1602)
>
> > at
> >
> >
>
> org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1419)
>
> > at
> >
> org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:584)
> > ...
> >
> > I have found an issue that addresses a similiar problem:
> > https://issues.apache.org/jira/browse/SOLR-8807
> >
> > The fix, which was introduced with this issue seems to miss our
> situation,
> > though. The relevant part of the query is this:
> >
> > &fq={!tag=collapser}{!collapse field=productId sort='merchantOrder asc,
> > price asc, id asc'}
> >
> > When I remove the local parameter {!tag=collapser} the collation works
> > fine. Looking at the diff of the commit of the issue mentioned above, it
> > seems that the "startsWith" could be the problem:
> >
> > + // Collate testing does not support the Collapse QParser (See
> > SOLR-8807)
> > + params.remove("expand");
> > + String[] filters = params.getParams(CommonParams.FQ);
> > + if (filters != null) {
> > + List<String> filtersToApply = new ArrayList<>(filters.length);
> > + for (String fq : filters) {
> > + if (!fq.startsWith("{!collapse")) {
> > + filtersToApply.add(fq);
> > + }
> > + }
> > + params.set("fq", filtersToApply.toArray(new
> > String[filtersToApply.size()]));
> > + }
> >
> > Can someone confirm this? I would open a bug ticket then. (Since the code
> > is unchanged in the latest version.)
> >
> > Thanks,
> > Stefan
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: NPE during spell checking when result collapsing is activated and local parameters are used

Stefan Walter
I created https://issues.apache.org/jira/browse/SOLR-13944 (sorry, it took
a while). Thanks again!


Am 15. November 2019 um 16:10:55, Tomás Fernández Löbbe (
[hidden email]) schrieb:

Would you create a Jira issue anyway tu fix the fact that it NPE instead of
throwing a bad request?

On Fri, Nov 15, 2019 at 2:31 AM Stefan Walter <[hidden email]> wrote:

> Indeed, you are right. Interestingly, it generally worked with the two {!
> ..} in the filter query - besides the problem with the collations, of
> course. Therefore I never questioned it...
>
> Thank you!
> Stefan
>
>
> Am 15. November 2019 um 00:01:52, Tomás Fernández Löbbe (
> [hidden email]) schrieb:
>
> I believe your syntax is incorrect. I believe local params must all be
> included in between the same {!...}, and "{!" can only be at the
beginning

>
> have you tried:
>
> &fq={!collapse tag=collapser field=productId sort='merchantOrder asc,
> price asc, id asc'}
>
>
>
> On Thu, Nov 14, 2019 at 4:54 AM Stefan Walter <[hidden email]> wrote:
>
> > Hi!
> >
> > I have an issue with Solr 7.3.1 in the spell checking component:
> >
> > java.lang.NullPointerException at
> >
> >
>
>
org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1021)

>
> > at
> >
> >
>
>
org.apache.solr.search.CollapsingQParserPlugin$OrdFieldValueCollector.finish(CollapsingQParserPlugin.java:1081)

>
> > at
> >
> >
>
>
org.apache.solr.search.SolrIndexSearcher.buildAndRunCollectorChain(SolrIndexSearcher.java:230)

>
> > at
> >
> >
>
>
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:1602)

>
> > at
> >
> >
>
>
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:1419)

>
> > at
> >
>
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:584)

> > ...
> >
> > I have found an issue that addresses a similiar problem:
> > https://issues.apache.org/jira/browse/SOLR-8807
> >
> > The fix, which was introduced with this issue seems to miss our
> situation,
> > though. The relevant part of the query is this:
> >
> > &fq={!tag=collapser}{!collapse field=productId sort='merchantOrder asc,
> > price asc, id asc'}
> >
> > When I remove the local parameter {!tag=collapser} the collation works
> > fine. Looking at the diff of the commit of the issue mentioned above,
it

> > seems that the "startsWith" could be the problem:
> >
> > + // Collate testing does not support the Collapse QParser (See
> > SOLR-8807)
> > + params.remove("expand");
> > + String[] filters = params.getParams(CommonParams.FQ);
> > + if (filters != null) {
> > + List<String> filtersToApply = new ArrayList<>(filters.length);
> > + for (String fq : filters) {
> > + if (!fq.startsWith("{!collapse")) {
> > + filtersToApply.add(fq);
> > + }
> > + }
> > + params.set("fq", filtersToApply.toArray(new
> > String[filtersToApply.size()]));
> > + }
> >
> > Can someone confirm this? I would open a bug ticket then. (Since the
code
> > is unchanged in the latest version.)
> >
> > Thanks,
> > Stefan
> >
>