CustomQuery.bulkScorer isn't called from BooleanQuery with filter block

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

CustomQuery.bulkScorer isn't called from BooleanQuery with filter block

Vadim Gindin
Hi all!

I have the following BooleanQuery (JSON representation as it is defined in
Elasticsearch):

{
  "query" : {
    "bool" : {
      "must" : {
        "custom_query" : {      // has own bulkScorer() implementation
             ...
         }
      },
      "filter" : {
             ...
      }
    }
  }
}

Here "custom_query" is a custom query I defined with custom Weight and
custom bulkScorer() method.

1. This method isn't called if BooleanQuery contains Filter clause near
CustomQuery.
2. This method is called if BooleanQuery contains only one clause with
CustomQuery.

Why? I need my CustomQuery.CustomWieght.bulkScorer() working.

P.S. As I understand 2 clauses are "required" clauses. CustomQuery having
bulkScorer() doesn't know about Filter, and Filter doesn't know about
CustomQuery, but it seems they should work together. And that is why
bulkScorer isn't called.

Is there a way to integrate CustomQuery.bulkScorer() with possible adjacent
filters?

Regards,
Vadim Gindin
Reply | Threaded
Open this post in threaded view
|

Re: CustomQuery.bulkScorer isn't called from BooleanQuery with filter block

Adrien Grand
Hello Vadim,

It looks like your query only supports bulkScorer() and not scorer()?
Unfortunately this is illegal: queries must implement scorer(). Today,
conjunctions never use the bulkScorer API.

Le mer. 25 juil. 2018 à 18:47, Vadim Gindin <[hidden email]> a écrit :

> Hi all!
>
> I have the following BooleanQuery (JSON representation as it is defined in
> Elasticsearch):
>
> {
>   "query" : {
>     "bool" : {
>       "must" : {
>         "custom_query" : {      // has own bulkScorer() implementation
>              ...
>          }
>       },
>       "filter" : {
>              ...
>       }
>     }
>   }
> }
>
> Here "custom_query" is a custom query I defined with custom Weight and
> custom bulkScorer() method.
>
> 1. This method isn't called if BooleanQuery contains Filter clause near
> CustomQuery.
> 2. This method is called if BooleanQuery contains only one clause with
> CustomQuery.
>
> Why? I need my CustomQuery.CustomWieght.bulkScorer() working.
>
> P.S. As I understand 2 clauses are "required" clauses. CustomQuery having
> bulkScorer() doesn't know about Filter, and Filter doesn't know about
> CustomQuery, but it seems they should work together. And that is why
> bulkScorer isn't called.
>
> Is there a way to integrate CustomQuery.bulkScorer() with possible adjacent
> filters?
>
> Regards,
> Vadim Gindin
>
Reply | Threaded
Open this post in threaded view
|

Re: CustomQuery.bulkScorer isn't called from BooleanQuery with filter block

Vadim Gindin
Hello Adrien.

No, My query supports both scorer() and bulkScorer(). Sorry, I didn't
specified that, because I thought it is not related to my difficulty.

Could you explain what did you mean by " Today, conjunctions never use the
bulkScorer API."?

Thanks.

On Thu, Jul 26, 2018 at 1:11 AM Adrien Grand <[hidden email]> wrote:

> Hello Vadim,
>
> It looks like your query only supports bulkScorer() and not scorer()?
> Unfortunately this is illegal: queries must implement scorer(). Today,
> conjunctions never use the bulkScorer API.
>
> Le mer. 25 juil. 2018 à 18:47, Vadim Gindin <[hidden email]> a
> écrit :
>
> > Hi all!
> >
> > I have the following BooleanQuery (JSON representation as it is defined
> in
> > Elasticsearch):
> >
> > {
> >   "query" : {
> >     "bool" : {
> >       "must" : {
> >         "custom_query" : {      // has own bulkScorer() implementation
> >              ...
> >          }
> >       },
> >       "filter" : {
> >              ...
> >       }
> >     }
> >   }
> > }
> >
> > Here "custom_query" is a custom query I defined with custom Weight and
> > custom bulkScorer() method.
> >
> > 1. This method isn't called if BooleanQuery contains Filter clause near
> > CustomQuery.
> > 2. This method is called if BooleanQuery contains only one clause with
> > CustomQuery.
> >
> > Why? I need my CustomQuery.CustomWieght.bulkScorer() working.
> >
> > P.S. As I understand 2 clauses are "required" clauses. CustomQuery having
> > bulkScorer() doesn't know about Filter, and Filter doesn't know about
> > CustomQuery, but it seems they should work together. And that is why
> > bulkScorer isn't called.
> >
> > Is there a way to integrate CustomQuery.bulkScorer() with possible
> adjacent
> > filters?
> >
> > Regards,
> > Vadim Gindin
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: CustomQuery.bulkScorer isn't called from BooleanQuery with filter block

Adrien Grand
Weight#bulkScorer has a similar purpose as Weight#scorer but is optimized
for the case that all matches need to be visited. Say your query is "a OR
b": all hits of "a" are also hits of "a OR b" so if you run this query,
Lucene will call Weight#bulkScorer on both sub clauses and merge results.
Now if you query is "a AND b" many hits that match "a" may not match "a AND
b" if they don't match "b" as well. So Lucene uses Weight#scorer in that
case and tries to compute the intersection by alternatively advancing each
iterator to the current document of the other iterator.

Le jeu. 26 juil. 2018 à 20:03, Vadim Gindin <[hidden email]> a écrit :

> Hello Adrien.
>
> No, My query supports both scorer() and bulkScorer(). Sorry, I didn't
> specified that, because I thought it is not related to my difficulty.
>
> Could you explain what did you mean by " Today, conjunctions never use the
> bulkScorer API."?
>
> Thanks.
>
> On Thu, Jul 26, 2018 at 1:11 AM Adrien Grand <[hidden email]> wrote:
>
> > Hello Vadim,
> >
> > It looks like your query only supports bulkScorer() and not scorer()?
> > Unfortunately this is illegal: queries must implement scorer(). Today,
> > conjunctions never use the bulkScorer API.
> >
> > Le mer. 25 juil. 2018 à 18:47, Vadim Gindin <[hidden email]> a
> > écrit :
> >
> > > Hi all!
> > >
> > > I have the following BooleanQuery (JSON representation as it is defined
> > in
> > > Elasticsearch):
> > >
> > > {
> > >   "query" : {
> > >     "bool" : {
> > >       "must" : {
> > >         "custom_query" : {      // has own bulkScorer() implementation
> > >              ...
> > >          }
> > >       },
> > >       "filter" : {
> > >              ...
> > >       }
> > >     }
> > >   }
> > > }
> > >
> > > Here "custom_query" is a custom query I defined with custom Weight and
> > > custom bulkScorer() method.
> > >
> > > 1. This method isn't called if BooleanQuery contains Filter clause near
> > > CustomQuery.
> > > 2. This method is called if BooleanQuery contains only one clause with
> > > CustomQuery.
> > >
> > > Why? I need my CustomQuery.CustomWieght.bulkScorer() working.
> > >
> > > P.S. As I understand 2 clauses are "required" clauses. CustomQuery
> having
> > > bulkScorer() doesn't know about Filter, and Filter doesn't know about
> > > CustomQuery, but it seems they should work together. And that is why
> > > bulkScorer isn't called.
> > >
> > > Is there a way to integrate CustomQuery.bulkScorer() with possible
> > adjacent
> > > filters?
> > >
> > > Regards,
> > > Vadim Gindin
> > >
> >
>