Issue: Hit Highlighting Working Inconsistently in Solr 6.6

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Issue: Hit Highlighting Working Inconsistently in Solr 6.6

Vikram Oberoi
Hi there,

I'm seeing inconsistent highlighting behavior using a default, fresh Solr
6.6 install and it's unclear to me why or how to go about debugging it.

Hit highlights either show entirely correct highlights or none at all when
there should be highlights.

   - Some queries show highlights out of the box, some do not.
      - e.g. "hello" yields no highlights, but "goodbye" correctly yields
      highlights
   - Some queries that do not show highlights suddenly work when specifying
   fields
      - e.g. "subject:hello" yields highlights, but "hello" does not
   - When queries that yield highlights and queries that do not are
   combined, only those that work are highlighted.
      - e.g. "hello goodbye" yields highlights correctly for "goodbye", but
      not for "hello"

I've thrown specific details and examples in a Gist here:

Full Gist: https://gist.github.com/voberoi/a7a8a679390fc4f27422e70600cfb338

   - Problem description:
      - https://gist.github.com/voberoi/a7a8a679390fc4f27422e70600cfb3
      38#file-problem-details-md
   - Solr install, my schema, solrconfig details:
      - https://gist.github.com/voberoi/a7a8a679390fc4f27422e70600cfb3
      38#file-solr-details-md

Does anyone here have any hypotheses for why this might be happening?

Thanks!
Vikram
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Issue: Hit Highlighting Working Inconsistently in Solr 6.6

Vikram Oberoi
Hi!

Just wanted to close the loop here.

I'm pretty sure this has something to do with the default _text_ "catchall"
field being a slightly differently type ('text_general') from all my
textual fields ('text_en'). A few things I tried support that hypothesis:

- Specifying fields for terms correctly yields highlights consistently
(e.g. "hello" doesn't work but "subject:hello" always does).
- Creating a different catchall field with same type as all my textual
fields ('text_en') and making that the default field yields highlighting
results that work properly and consistently.
- Finally -- I need to use a friendlier parser anyway. Using edismax for
all my queries -- and eliminating my catchall field -- yields highlighting
results properly and consistently.

I've got this working, but I'm curious to know if this is what's happening
more around precisely why. If anyone more knowledgable has thoughts or
pointers to writing on how highlighting works internally, I'd really
appreciate it!

Cheers,
Vikram

On Thu, Jul 13, 2017 at 5:51 PM, Vikram Oberoi <[hidden email]> wrote:

> Hi there,
>
> I'm seeing inconsistent highlighting behavior using a default, fresh Solr
> 6.6 install and it's unclear to me why or how to go about debugging it.
>
> Hit highlights either show entirely correct highlights or none at all when
> there should be highlights.
>
>    - Some queries show highlights out of the box, some do not.
>       - e.g. "hello" yields no highlights, but "goodbye" correctly yields
>       highlights
>    - Some queries that do not show highlights suddenly work when
>    specifying fields
>       - e.g. "subject:hello" yields highlights, but "hello" does not
>    - When queries that yield highlights and queries that do not are
>    combined, only those that work are highlighted.
>       - e.g. "hello goodbye" yields highlights correctly for "goodbye",
>       but not for "hello"
>
> I've thrown specific details and examples in a Gist here:
>
> Full Gist: https://gist.github.com/voberoi/a7a8a679390fc4f27422e7
> 0600cfb338
>
>    - Problem description:
>       - https://gist.github.com/voberoi/a7a8a679390fc4f27422e70600cf
>       b338#file-problem-details-md
>    - Solr install, my schema, solrconfig details:
>       - https://gist.github.com/voberoi/a7a8a679390fc4f27422e70600cf
>       b338#file-solr-details-md
>
> Does anyone here have any hypotheses for why this might be happening?
>
> Thanks!
> Vikram
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Issue: Hit Highlighting Working Inconsistently in Solr 6.6

david.w.smiley@gmail.com
Does hl.method=unified help any?

Perhaps you need to set hl.fl?  or hl.requireFieldMatch=false? (although it
should default to false already)

On Fri, Jul 14, 2017 at 6:52 PM Vikram Oberoi <[hidden email]> wrote:

> Hi!
>
> Just wanted to close the loop here.
>
> I'm pretty sure this has something to do with the default _text_ "catchall"
> field being a slightly differently type ('text_general') from all my
> textual fields ('text_en'). A few things I tried support that hypothesis:
>
> - Specifying fields for terms correctly yields highlights consistently
> (e.g. "hello" doesn't work but "subject:hello" always does).
> - Creating a different catchall field with same type as all my textual
> fields ('text_en') and making that the default field yields highlighting
> results that work properly and consistently.
> - Finally -- I need to use a friendlier parser anyway. Using edismax for
> all my queries -- and eliminating my catchall field -- yields highlighting
> results properly and consistently.
>
> I've got this working, but I'm curious to know if this is what's happening
> more around precisely why. If anyone more knowledgable has thoughts or
> pointers to writing on how highlighting works internally, I'd really
> appreciate it!
>
> Cheers,
> Vikram
>
> On Thu, Jul 13, 2017 at 5:51 PM, Vikram Oberoi <[hidden email]> wrote:
>
> > Hi there,
> >
> > I'm seeing inconsistent highlighting behavior using a default, fresh Solr
> > 6.6 install and it's unclear to me why or how to go about debugging it.
> >
> > Hit highlights either show entirely correct highlights or none at all
> when
> > there should be highlights.
> >
> >    - Some queries show highlights out of the box, some do not.
> >       - e.g. "hello" yields no highlights, but "goodbye" correctly yields
> >       highlights
> >    - Some queries that do not show highlights suddenly work when
> >    specifying fields
> >       - e.g. "subject:hello" yields highlights, but "hello" does not
> >    - When queries that yield highlights and queries that do not are
> >    combined, only those that work are highlighted.
> >       - e.g. "hello goodbye" yields highlights correctly for "goodbye",
> >       but not for "hello"
> >
> > I've thrown specific details and examples in a Gist here:
> >
> > Full Gist: https://gist.github.com/voberoi/a7a8a679390fc4f27422e7
> > 0600cfb338
> >
> >    - Problem description:
> >       - https://gist.github.com/voberoi/a7a8a679390fc4f27422e70600cf
> >       b338#file-problem-details-md
> >    - Solr install, my schema, solrconfig details:
> >       - https://gist.github.com/voberoi/a7a8a679390fc4f27422e70600cf
> >       b338#file-solr-details-md
> >
> > Does anyone here have any hypotheses for why this might be happening?
> >
> > Thanks!
> > Vikram
> >
>
--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com
Loading...