return matched terms / fuzzy or wildcard searches

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

return matched terms / fuzzy or wildcard searches

Krystian Napiatek
Hi,

is it possible to get a list of all matched terms, when using queries like:
dna~0.7; d?a; dn*;
I need the terms for highlighting them later in the output.

Thank you && greets
Krystian
Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Erik Hatcher

On Mar 23, 2007, at 5:44 AM, Krystian Napiatek wrote:
> is it possible to get a list of all matched terms, when using  
> queries like:
> dna~0.7; d?a; dn*;
> I need the terms for highlighting them later in the output.

Will the built-in highlighting capability help you here?

        <http://wiki.apache.org/solr/HighlightingParameters>


Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Krystian Napiatek
Yes I do:
...&hl=on&hl.fl=figure&hl.fragsize=0&hl.snippets=200&hl.simple.pre=<span
style="background-color: tomato">&hl.simple.post=</span>...

But the response isn't highlighted using fuzzy or wildcard searches...


2007/3/23, Erik Hatcher <[hidden email]>:

>
>
> On Mar 23, 2007, at 5:44 AM, Krystian Napiatek wrote:
> > is it possible to get a list of all matched terms, when using
> > queries like:
> > dna~0.7; d?a; dn*;
> > I need the terms for highlighting them later in the output.
>
> Will the built-in highlighting capability help you here?
>
>        <http://wiki.apache.org/solr/HighlightingParameters>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Chris Hostetter-3

: But the response isn't highlighted using fuzzy or wildcard searches...

Hmmm... this seems like a bug in the highlighting, using the sample schema
this highlights properly...

http://localhost:8983/solr/select/?q=id%3AVA902B&version=2.2&start=0&rows=10&indent=on&fl=id&hl=true&hl.fl=id

...but this does not...

http://localhost:8983/solr/select/?q=id%3AV*&version=2.2&start=0&rows=10&indent=on&fl=id&hl=true&hl.fl=id

perhaps the Solr highlighting code isn't calling rewrite() before using
the Highlighter?



-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Mike Klaas
On 3/23/07, Chris Hostetter <[hidden email]> wrote:

>
> : But the response isn't highlighted using fuzzy or wildcard searches...
>
> Hmmm... this seems like a bug in the highlighting, using the sample schema
> this highlights properly...
>
> http://localhost:8983/solr/select/?q=id%3AVA902B&version=2.2&start=0&rows=10&indent=on&fl=id&hl=true&hl.fl=id
>
> ...but this does not...
>
> http://localhost:8983/solr/select/?q=id%3AV*&version=2.2&start=0&rows=10&indent=on&fl=id&hl=true&hl.fl=id
>
> perhaps the Solr highlighting code isn't calling rewrite() before using
> the Highlighter?

It is, in trunk/:

NamedList sumData = HighlightingUtils.doHighlighting(
        results.docList, query.rewrite(req.getSearcher().getReader()),
req, new String[]{defaultFiel
d});

Definitely a bug somewhere.  Does anyone more familiar with lucene see
why the above wouldn't be sufficient?

-Mike
Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Yonik Seeley-2
On 3/23/07, Mike Klaas <[hidden email]> wrote:

> On 3/23/07, Chris Hostetter <[hidden email]> wrote:
> >
> > : But the response isn't highlighted using fuzzy or wildcard searches...
> >
> > Hmmm... this seems like a bug in the highlighting, using the sample schema
> > this highlights properly...
> >
> > http://localhost:8983/solr/select/?q=id%3AVA902B&version=2.2&start=0&rows=10&indent=on&fl=id&hl=true&hl.fl=id
> >
> > ...but this does not...
> >
> > http://localhost:8983/solr/select/?q=id%3AV*&version=2.2&start=0&rows=10&indent=on&fl=id&hl=true&hl.fl=id
> >
> > perhaps the Solr highlighting code isn't calling rewrite() before using
> > the Highlighter?
>
> It is, in trunk/:
>
> NamedList sumData = HighlightingUtils.doHighlighting(
>         results.docList, query.rewrite(req.getSearcher().getReader()),
> req, new String[]{defaultFiel
> d});
>
> Definitely a bug somewhere.  Does anyone more familiar with lucene see
> why the above wouldn't be sufficient?

Perhaps our use of ConstantScorePrefixQuery by default?

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Mike Klaas
On 3/23/07, Yonik Seeley <[hidden email]> wrote:
> On 3/23/07, Mike Klaas <[hidden email]> wrote:

> > Definitely a bug somewhere.  Does anyone more familiar with lucene see
> > why the above wouldn't be sufficient?
>
> Perhaps our use of ConstantScorePrefixQuery by default?

tracked here: http://issues.apache.org/jira/browse/SOLR-195

-Mike
Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Erik Hatcher
In reply to this post by Yonik Seeley-2

On Mar 23, 2007, at 3:26 PM, Yonik Seeley wrote:

> On 3/23/07, Mike Klaas <[hidden email]> wrote:
>> On 3/23/07, Chris Hostetter <[hidden email]> wrote:
>> >
>> > : But the response isn't highlighted using fuzzy or wildcard  
>> searches...
>> >
>> > Hmmm... this seems like a bug in the highlighting, using the  
>> sample schema
>> > this highlights properly...
>> >
>> > <a href="http://localhost:8983/solr/select/?q=id%">http://localhost:8983/solr/select/?q=id% 
>> 3AVA902B&version=2.2&start=0&rows=10&indent=on&fl=id&hl=true&hl.fl=id
>> >
>> > ...but this does not...
>> >
>> > <a href="http://localhost:8983/solr/select/?q=id%">http://localhost:8983/solr/select/?q=id% 
>> 3AV*&version=2.2&start=0&rows=10&indent=on&fl=id&hl=true&hl.fl=id
>> >
>> > perhaps the Solr highlighting code isn't calling rewrite()  
>> before using
>> > the Highlighter?
>>
>> It is, in trunk/:
>>
>> NamedList sumData = HighlightingUtils.doHighlighting(
>>         results.docList, query.rewrite(req.getSearcher().getReader
>> ()),
>> req, new String[]{defaultFiel
>> d});
>>
>> Definitely a bug somewhere.  Does anyone more familiar with lucene  
>> see
>> why the above wouldn't be sufficient?
>
> Perhaps our use of ConstantScorePrefixQuery by default?

Ah, that would probably explain it!   I had stumbled on this before  
too and went to fix it and saw the rewrite in there and was  
perplexed, but then got distracted by something shiny.

        Erik

Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Chris Hostetter-3

: > Perhaps our use of ConstantScorePrefixQuery by default?
:
: Ah, that would probably explain it!   I had stumbled on this before
: too and went to fix it and saw the rewrite in there and was
: perplexed, but then got distracted by something shiny.

yeah, that makes sense ... a true wildcard query works fine...

http://localhost:8983/solr/select/?q=id:V???B*&fl=id&hl=true&hl.fl=id


To answer your question Krystian: it's suppose to work for you, for
fuzzy queries (like: dna~0.7) and wildcard queries (like: d?a) it
should currently be working fine ... pelase send us an example Solr URL
that doesn't work if it's not what you are observing.

Only a simple prefix query (like: dn*) doesn't work ... and that seems to
be because of the way we optimize a PrefixQuery into a
ConstantScorePrefixQuery .. a workarround is to allways include a "?" in
your query when you want highlighting -- so instead of dn* search for dn?*


-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Krystian Napiatek
My Solr-Server: http://www.captionsearch.de/solr.html
Everytime you make a new search you get the last response file here:
http://www.captionsearch.de/response.xml


2007/3/24, Chris Hostetter <[hidden email]>:

>
>
> : > Perhaps our use of ConstantScorePrefixQuery by default?
> :
> : Ah, that would probably explain it!   I had stumbled on this before
> : too and went to fix it and saw the rewrite in there and was
> : perplexed, but then got distracted by something shiny.
>
> yeah, that makes sense ... a true wildcard query works fine...
>
> http://localhost:8983/solr/select/?q=id:V???B*&fl=id&hl=true&hl.fl=id
>
>
> To answer your question Krystian: it's suppose to work for you, for
> fuzzy queries (like: dna~0.7) and wildcard queries (like: d?a) it
> should currently be working fine ... pelase send us an example Solr URL
> that doesn't work if it's not what you are observing.
>
> Only a simple prefix query (like: dn*) doesn't work ... and that seems to
> be because of the way we optimize a PrefixQuery into a
> ConstantScorePrefixQuery .. a workarround is to allways include a "?" in
> your query when you want highlighting -- so instead of dn* search for dn?*
>
>
> -Hoss
>
>
Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Mike Klaas
In reply to this post by Chris Hostetter-3
On 3/23/07, Chris Hostetter <[hidden email]> wrote:

> Only a simple prefix query (like: dn*) doesn't work ... and that seems to
> be because of the way we optimize a PrefixQuery into a
> ConstantScorePrefixQuery .. a workarround is to allways include a "?" in
> your query when you want highlighting -- so instead of dn* search for dn?*

Note that you need the a recent nightly build for that to work--it
wasn't there for the last release.

-Mike
Reply | Threaded
Open this post in threaded view
|

Re: return matched terms / fuzzy or wildcard searches

Krystian Napiatek
Thank you...it works now fine with the nightly build...Greets Krystian

2007/3/25, Mike Klaas <[hidden email]>:

>
> On 3/23/07, Chris Hostetter <[hidden email]> wrote:
>
> > Only a simple prefix query (like: dn*) doesn't work ... and that seems
> to
> > be because of the way we optimize a PrefixQuery into a
> > ConstantScorePrefixQuery .. a workarround is to allways include a "?" in
> > your query when you want highlighting -- so instead of dn* search for
> dn?*
>
> Note that you need the a recent nightly build for that to work--it
> wasn't there for the last release.
>
> -Mike
>