Some solrconfig.xml attributes being ignored

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Some solrconfig.xml attributes being ignored

Mark Ferguson
Hello,

In my solrconfig.xml file I am setting the attribute hl.snippets to 3. When
I perform a search, it returns only a single snippet for each highlighted
field. However, when I set the hl.snippets field manually as a search
parameter, I get up to 3 highlighted snippets. This is the configuration
that I am using to set the highlighted parameters:

<fragmenter name="regex" class="org.apache.solr.highlight.RegexFragmenter"
default="true">
    <lst name="defaults">
      <str name="hl.snippets">3</str>
      <str name="hl.fragsize">100</str>
      <str name="hl.regex.slop">0.5</str>
      <str name="hl.regex.pattern">\w[-\w ,/\n\"']{50,150}</str>
    </lst>
</fragmenter>

I tried setting hl.fragmenter=regex as a parameter as well, to be sure that
it was using the correct one, and the result set is the same. Any ideas what
could be causing this attribute not to be read? It has me concerned that
other attributes are being ignored as well.

Thanks,

Mark Ferguson
Reply | Threaded
Open this post in threaded view
|

Re: Some solrconfig.xml attributes being ignored

Yonik Seeley
Try adding echoParams=all to your query to verify the params that the
solr request handler is getting.

-Yonik

On Mon, Dec 15, 2008 at 6:10 PM, Mark Ferguson
<[hidden email]> wrote:

> Hello,
>
> In my solrconfig.xml file I am setting the attribute hl.snippets to 3. When
> I perform a search, it returns only a single snippet for each highlighted
> field. However, when I set the hl.snippets field manually as a search
> parameter, I get up to 3 highlighted snippets. This is the configuration
> that I am using to set the highlighted parameters:
>
> <fragmenter name="regex" class="org.apache.solr.highlight.RegexFragmenter"
> default="true">
>    <lst name="defaults">
>      <str name="hl.snippets">3</str>
>      <str name="hl.fragsize">100</str>
>      <str name="hl.regex.slop">0.5</str>
>      <str name="hl.regex.pattern">\w[-\w ,/\n\"']{50,150}</str>
>    </lst>
> </fragmenter>
>
> I tried setting hl.fragmenter=regex as a parameter as well, to be sure that
> it was using the correct one, and the result set is the same. Any ideas what
> could be causing this attribute not to be read? It has me concerned that
> other attributes are being ignored as well.
>
> Thanks,
>
> Mark Ferguson
>
Reply | Threaded
Open this post in threaded view
|

Re: Some solrconfig.xml attributes being ignored

Mark Ferguson
Thanks for this tip, it's very helpful. Indeed, it looks like none of the
highlighting parameters are being included. It's using the correct request
handler and hl is set to true, but none of the highlighting parameters from
solrconfig.xml are in the parameter list.

Here is my query:

http://localhost:8080/solr1/select?rows=50&hl=true&fl=url,urlmd5,page_title,score&echoParams=all&q=java

Here are the settings for the request handler and the highlighter:

<requestHandler name="dismax" class="solr.SearchHandler" default="true">
  <lst name="defaults">
   <str name="defType">dismax</str>
   <float name="tie">0.01</float>
   <str name="qf">body_text^1.0 page_title^1.6 meta_desc^1.3</str>
   <str name="q.alt">*:*</str>
   <str name="hl.fl">body_text page_title meta_desc</str>
   <str name="f.page_title.hl.fragsize">0</str>
   <str name="f.meta_desc.hl.fragsize">0</str>
   <str name="hl.fragmenter">regex</str>
  </lst>
</requestHandler>

<highlighting>
  <fragmenter name="regex" class="org.apache.solr.highlight.RegexFragmenter"
default="true">
    <lst name="defaults">
      <str name="hl.snippets">3</str>
      <str name="hl.fragsize">100</str>
      <str name="hl.regex.slop">0.5</str>
      <str name="hl.regex.pattern">\w[-\w ,/\n\"']{50,150}</str>
    </lst>
  </fragmenter>
</highlighting>

And here is the param list returned to me:

<lst name="params">
<str name="echoParams">all</str>
<str name="tie">0.01</str>
<str name="hl.fragmenter">regex</str>
<str name="f.page_title.hl.fragsize">0</str>
<str name="qf">body_text^1.0 page_title^1.6 meta_desc^1.3</str>
<str name="f.meta_desc.hl.fragsize">0</str>
<str name="q.alt">*:*</str>
<str name="hl.fl">page_title,body_text</str>
<str name="defType">dismax</str>
<str name="echoParams">all</str>
<str name="fl">url,urlmd5,page_title,score</str>
<str name="q">java</str>
<str name="hl">true</str>
<str name="rows">50</str>
</lst>

So it seems like everything is working except for the highlighter. I should
mention that when I enter a bogus fragmenter as a parameter (e.g.
hl.fragmenter=bogus), it returns a 400 error that the fragmenter cannot be
found, so the config file _is_ finding the regex fragmenter. It just doesn't
seem to actually be including its parameters... Any ideas are appreciated,
thanks again for the help.

Mark


On Mon, Dec 15, 2008 at 4:23 PM, Yonik Seeley <[hidden email]> wrote:

> Try adding echoParams=all to your query to verify the params that the
> solr request handler is getting.
>
> -Yonik
>
> On Mon, Dec 15, 2008 at 6:10 PM, Mark Ferguson
> <[hidden email]> wrote:
> > Hello,
> >
> > In my solrconfig.xml file I am setting the attribute hl.snippets to 3.
> When
> > I perform a search, it returns only a single snippet for each highlighted
> > field. However, when I set the hl.snippets field manually as a search
> > parameter, I get up to 3 highlighted snippets. This is the configuration
> > that I am using to set the highlighted parameters:
> >
> > <fragmenter name="regex"
> class="org.apache.solr.highlight.RegexFragmenter"
> > default="true">
> >    <lst name="defaults">
> >      <str name="hl.snippets">3</str>
> >      <str name="hl.fragsize">100</str>
> >      <str name="hl.regex.slop">0.5</str>
> >      <str name="hl.regex.pattern">\w[-\w ,/\n\"']{50,150}</str>
> >    </lst>
> > </fragmenter>
> >
> > I tried setting hl.fragmenter=regex as a parameter as well, to be sure
> that
> > it was using the correct one, and the result set is the same. Any ideas
> what
> > could be causing this attribute not to be read? It has me concerned that
> > other attributes are being ignored as well.
> >
> > Thanks,
> >
> > Mark Ferguson
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Some solrconfig.xml attributes being ignored

Mark Ferguson
It seems like maybe the fragmenter parameters just don't get displayed with
echoParams=all set. It may only display as far as the request handler's
parameters. The reason I think this is because I tried increasing
hl.fragsize to 1000 and the results were returned correctly (much larger
snippets), so I know it was read correctly.

I moved hl.snippets into the requestHandler config instead of the
fragmenter, and this seems to have solved the problem. However, I'm uneasy
with this solution because I don't know why it wasn't being read correctly
when setting it inside the fragmenter.

Mark



On Mon, Dec 15, 2008 at 5:08 PM, Mark Ferguson <[hidden email]>wrote:

> Thanks for this tip, it's very helpful. Indeed, it looks like none of the
> highlighting parameters are being included. It's using the correct request
> handler and hl is set to true, but none of the highlighting parameters from
> solrconfig.xml are in the parameter list.
>
> Here is my query:
>
>
> http://localhost:8080/solr1/select?rows=50&hl=true&fl=url,urlmd5,page_title,score&echoParams=all&q=java
>
> Here are the settings for the request handler and the highlighter:
>
> <requestHandler name="dismax" class="solr.SearchHandler" default="true">
>   <lst name="defaults">
>    <str name="defType">dismax</str>
>    <float name="tie">0.01</float>
>    <str name="qf">body_text^1.0 page_title^1.6 meta_desc^1.3</str>
>    <str name="q.alt">*:*</str>
>    <str name="hl.fl">body_text page_title meta_desc</str>
>    <str name="f.page_title.hl.fragsize">0</str>
>    <str name="f.meta_desc.hl.fragsize">0</str>
>    <str name="hl.fragmenter">regex</str>
>   </lst>
> </requestHandler>
>
> <highlighting>
>   <fragmenter name="regex"
> class="org.apache.solr.highlight.RegexFragmenter" default="true">
>     <lst name="defaults">
>       <str name="hl.snippets">3</str>
>       <str name="hl.fragsize">100</str>
>       <str name="hl.regex.slop">0.5</str>
>       <str name="hl.regex.pattern">\w[-\w ,/\n\"']{50,150}</str>
>     </lst>
>   </fragmenter>
> </highlighting>
>
> And here is the param list returned to me:
>
> <lst name="params">
> <str name="echoParams">all</str>
> <str name="tie">0.01</str>
> <str name="hl.fragmenter">regex</str>
> <str name="f.page_title.hl.fragsize">0</str>
> <str name="qf">body_text^1.0 page_title^1.6 meta_desc^1.3</str>
> <str name="f.meta_desc.hl.fragsize">0</str>
> <str name="q.alt">*:*</str>
> <str name="hl.fl">page_title,body_text</str>
> <str name="defType">dismax</str>
> <str name="echoParams">all</str>
> <str name="fl">url,urlmd5,page_title,score</str>
> <str name="q">java</str>
> <str name="hl">true</str>
> <str name="rows">50</str>
> </lst>
>
> So it seems like everything is working except for the highlighter. I should
> mention that when I enter a bogus fragmenter as a parameter (e.g.
> hl.fragmenter=bogus), it returns a 400 error that the fragmenter cannot be
> found, so the config file _is_ finding the regex fragmenter. It just doesn't
> seem to actually be including its parameters... Any ideas are appreciated,
> thanks again for the help.
>
> Mark
>
>
>
> On Mon, Dec 15, 2008 at 4:23 PM, Yonik Seeley <[hidden email]> wrote:
>
>> Try adding echoParams=all to your query to verify the params that the
>> solr request handler is getting.
>>
>> -Yonik
>>
>> On Mon, Dec 15, 2008 at 6:10 PM, Mark Ferguson
>> <[hidden email]> wrote:
>> > Hello,
>> >
>> > In my solrconfig.xml file I am setting the attribute hl.snippets to 3.
>> When
>> > I perform a search, it returns only a single snippet for each
>> highlighted
>> > field. However, when I set the hl.snippets field manually as a search
>> > parameter, I get up to 3 highlighted snippets. This is the configuration
>> > that I am using to set the highlighted parameters:
>> >
>> > <fragmenter name="regex"
>> class="org.apache.solr.highlight.RegexFragmenter"
>> > default="true">
>> >    <lst name="defaults">
>> >      <str name="hl.snippets">3</str>
>> >      <str name="hl.fragsize">100</str>
>> >      <str name="hl.regex.slop">0.5</str>
>> >      <str name="hl.regex.pattern">\w[-\w ,/\n\"']{50,150}</str>
>> >    </lst>
>> > </fragmenter>
>> >
>> > I tried setting hl.fragmenter=regex as a parameter as well, to be sure
>> that
>> > it was using the correct one, and the result set is the same. Any ideas
>> what
>> > could be causing this attribute not to be read? It has me concerned that
>> > other attributes are being ignored as well.
>> >
>> > Thanks,
>> >
>> > Mark Ferguson
>> >
>>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Some solrconfig.xml attributes being ignored

Erik Hatcher
Mark,

Looked at the code to discern this...

A fragmenter isn't responsible for the number of snippets - the higher  
level SolrHighlighter is the component that uses that parameter.  So  
yes, it must be specified at the request handler level, not the  
fragmenter configuration.

        Erik

On Dec 15, 2008, at 7:35 PM, Mark Ferguson wrote:

> It seems like maybe the fragmenter parameters just don't get  
> displayed with
> echoParams=all set. It may only display as far as the request  
> handler's
> parameters. The reason I think this is because I tried increasing
> hl.fragsize to 1000 and the results were returned correctly (much  
> larger
> snippets), so I know it was read correctly.
>
> I moved hl.snippets into the requestHandler config instead of the
> fragmenter, and this seems to have solved the problem. However, I'm  
> uneasy
> with this solution because I don't know why it wasn't being read  
> correctly
> when setting it inside the fragmenter.
>
> Mark
>
>
>
> On Mon, Dec 15, 2008 at 5:08 PM, Mark Ferguson <[hidden email]
> >wrote:
>
>> Thanks for this tip, it's very helpful. Indeed, it looks like none  
>> of the
>> highlighting parameters are being included. It's using the correct  
>> request
>> handler and hl is set to true, but none of the highlighting  
>> parameters from
>> solrconfig.xml are in the parameter list.
>>
>> Here is my query:
>>
>>
>> http://localhost:8080/solr1/select?rows=50&hl=true&fl=url,urlmd5,page_title,score&echoParams=all&q=java
>>
>> Here are the settings for the request handler and the highlighter:
>>
>> <requestHandler name="dismax" class="solr.SearchHandler"  
>> default="true">
>>  <lst name="defaults">
>>   <str name="defType">dismax</str>
>>   <float name="tie">0.01</float>
>>   <str name="qf">body_text^1.0 page_title^1.6 meta_desc^1.3</str>
>>   <str name="q.alt">*:*</str>
>>   <str name="hl.fl">body_text page_title meta_desc</str>
>>   <str name="f.page_title.hl.fragsize">0</str>
>>   <str name="f.meta_desc.hl.fragsize">0</str>
>>   <str name="hl.fragmenter">regex</str>
>>  </lst>
>> </requestHandler>
>>
>> <highlighting>
>>  <fragmenter name="regex"
>> class="org.apache.solr.highlight.RegexFragmenter" default="true">
>>    <lst name="defaults">
>>      <str name="hl.snippets">3</str>
>>      <str name="hl.fragsize">100</str>
>>      <str name="hl.regex.slop">0.5</str>
>>      <str name="hl.regex.pattern">\w[-\w ,/\n\"']{50,150}</str>
>>    </lst>
>>  </fragmenter>
>> </highlighting>
>>
>> And here is the param list returned to me:
>>
>> <lst name="params">
>> <str name="echoParams">all</str>
>> <str name="tie">0.01</str>
>> <str name="hl.fragmenter">regex</str>
>> <str name="f.page_title.hl.fragsize">0</str>
>> <str name="qf">body_text^1.0 page_title^1.6 meta_desc^1.3</str>
>> <str name="f.meta_desc.hl.fragsize">0</str>
>> <str name="q.alt">*:*</str>
>> <str name="hl.fl">page_title,body_text</str>
>> <str name="defType">dismax</str>
>> <str name="echoParams">all</str>
>> <str name="fl">url,urlmd5,page_title,score</str>
>> <str name="q">java</str>
>> <str name="hl">true</str>
>> <str name="rows">50</str>
>> </lst>
>>
>> So it seems like everything is working except for the highlighter.  
>> I should
>> mention that when I enter a bogus fragmenter as a parameter (e.g.
>> hl.fragmenter=bogus), it returns a 400 error that the fragmenter  
>> cannot be
>> found, so the config file _is_ finding the regex fragmenter. It  
>> just doesn't
>> seem to actually be including its parameters... Any ideas are  
>> appreciated,
>> thanks again for the help.
>>
>> Mark
>>
>>
>>
>> On Mon, Dec 15, 2008 at 4:23 PM, Yonik Seeley <[hidden email]>  
>> wrote:
>>
>>> Try adding echoParams=all to your query to verify the params that  
>>> the
>>> solr request handler is getting.
>>>
>>> -Yonik
>>>
>>> On Mon, Dec 15, 2008 at 6:10 PM, Mark Ferguson
>>> <[hidden email]> wrote:
>>>> Hello,
>>>>
>>>> In my solrconfig.xml file I am setting the attribute hl.snippets  
>>>> to 3.
>>> When
>>>> I perform a search, it returns only a single snippet for each
>>> highlighted
>>>> field. However, when I set the hl.snippets field manually as a  
>>>> search
>>>> parameter, I get up to 3 highlighted snippets. This is the  
>>>> configuration
>>>> that I am using to set the highlighted parameters:
>>>>
>>>> <fragmenter name="regex"
>>> class="org.apache.solr.highlight.RegexFragmenter"
>>>> default="true">
>>>>   <lst name="defaults">
>>>>     <str name="hl.snippets">3</str>
>>>>     <str name="hl.fragsize">100</str>
>>>>     <str name="hl.regex.slop">0.5</str>
>>>>     <str name="hl.regex.pattern">\w[-\w ,/\n\"']{50,150}</str>
>>>>   </lst>
>>>> </fragmenter>
>>>>
>>>> I tried setting hl.fragmenter=regex as a parameter as well, to be  
>>>> sure
>>> that
>>>> it was using the correct one, and the result set is the same. Any  
>>>> ideas
>>> what
>>>> could be causing this attribute not to be read? It has me  
>>>> concerned that
>>>> other attributes are being ignored as well.
>>>>
>>>> Thanks,
>>>>
>>>> Mark Ferguson
>>>>
>>>
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Some solrconfig.xml attributes being ignored

Mark Ferguson
Hi Erik,

Thanks a lot for looking into this, it's greatly appreciated.

Mark


On Tue, Dec 16, 2008 at 2:51 AM, Erik Hatcher <[hidden email]>wrote:

> Mark,
>
> Looked at the code to discern this...
>
> A fragmenter isn't responsible for the number of snippets - the higher
> level SolrHighlighter is the component that uses that parameter.  So yes, it
> must be specified at the request handler level, not the fragmenter
> configuration.
>
>        Erik
>
>
> On Dec 15, 2008, at 7:35 PM, Mark Ferguson wrote:
>
>  It seems like maybe the fragmenter parameters just don't get displayed
>> with
>> echoParams=all set. It may only display as far as the request handler's
>> parameters. The reason I think this is because I tried increasing
>> hl.fragsize to 1000 and the results were returned correctly (much larger
>> snippets), so I know it was read correctly.
>>
>> I moved hl.snippets into the requestHandler config instead of the
>> fragmenter, and this seems to have solved the problem. However, I'm uneasy
>> with this solution because I don't know why it wasn't being read correctly
>> when setting it inside the fragmenter.
>>
>> Mark
>>
>>
>>
>> On Mon, Dec 15, 2008 at 5:08 PM, Mark Ferguson <[hidden email]
>> >wrote:
>>
>>  Thanks for this tip, it's very helpful. Indeed, it looks like none of the
>>> highlighting parameters are being included. It's using the correct
>>> request
>>> handler and hl is set to true, but none of the highlighting parameters
>>> from
>>> solrconfig.xml are in the parameter list.
>>>
>>> Here is my query:
>>>
>>>
>>>
>>> http://localhost:8080/solr1/select?rows=50&hl=true&fl=url,urlmd5,page_title,score&echoParams=all&q=java
>>>
>>> Here are the settings for the request handler and the highlighter:
>>>
>>> <requestHandler name="dismax" class="solr.SearchHandler" default="true">
>>>  <lst name="defaults">
>>>  <str name="defType">dismax</str>
>>>  <float name="tie">0.01</float>
>>>  <str name="qf">body_text^1.0 page_title^1.6 meta_desc^1.3</str>
>>>  <str name="q.alt">*:*</str>
>>>  <str name="hl.fl">body_text page_title meta_desc</str>
>>>  <str name="f.page_title.hl.fragsize">0</str>
>>>  <str name="f.meta_desc.hl.fragsize">0</str>
>>>  <str name="hl.fragmenter">regex</str>
>>>  </lst>
>>> </requestHandler>
>>>
>>> <highlighting>
>>>  <fragmenter name="regex"
>>> class="org.apache.solr.highlight.RegexFragmenter" default="true">
>>>   <lst name="defaults">
>>>     <str name="hl.snippets">3</str>
>>>     <str name="hl.fragsize">100</str>
>>>     <str name="hl.regex.slop">0.5</str>
>>>     <str name="hl.regex.pattern">\w[-\w ,/\n\"']{50,150}</str>
>>>   </lst>
>>>  </fragmenter>
>>> </highlighting>
>>>
>>> And here is the param list returned to me:
>>>
>>> <lst name="params">
>>> <str name="echoParams">all</str>
>>> <str name="tie">0.01</str>
>>> <str name="hl.fragmenter">regex</str>
>>> <str name="f.page_title.hl.fragsize">0</str>
>>> <str name="qf">body_text^1.0 page_title^1.6 meta_desc^1.3</str>
>>> <str name="f.meta_desc.hl.fragsize">0</str>
>>> <str name="q.alt">*:*</str>
>>> <str name="hl.fl">page_title,body_text</str>
>>> <str name="defType">dismax</str>
>>> <str name="echoParams">all</str>
>>> <str name="fl">url,urlmd5,page_title,score</str>
>>> <str name="q">java</str>
>>> <str name="hl">true</str>
>>> <str name="rows">50</str>
>>> </lst>
>>>
>>> So it seems like everything is working except for the highlighter. I
>>> should
>>> mention that when I enter a bogus fragmenter as a parameter (e.g.
>>> hl.fragmenter=bogus), it returns a 400 error that the fragmenter cannot
>>> be
>>> found, so the config file _is_ finding the regex fragmenter. It just
>>> doesn't
>>> seem to actually be including its parameters... Any ideas are
>>> appreciated,
>>> thanks again for the help.
>>>
>>> Mark
>>>
>>>
>>>
>>> On Mon, Dec 15, 2008 at 4:23 PM, Yonik Seeley <[hidden email]> wrote:
>>>
>>>  Try adding echoParams=all to your query to verify the params that the
>>>> solr request handler is getting.
>>>>
>>>> -Yonik
>>>>
>>>> On Mon, Dec 15, 2008 at 6:10 PM, Mark Ferguson
>>>> <[hidden email]> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> In my solrconfig.xml file I am setting the attribute hl.snippets to 3.
>>>>>
>>>> When
>>>>
>>>>> I perform a search, it returns only a single snippet for each
>>>>>
>>>> highlighted
>>>>
>>>>> field. However, when I set the hl.snippets field manually as a search
>>>>> parameter, I get up to 3 highlighted snippets. This is the
>>>>> configuration
>>>>> that I am using to set the highlighted parameters:
>>>>>
>>>>> <fragmenter name="regex"
>>>>>
>>>> class="org.apache.solr.highlight.RegexFragmenter"
>>>>
>>>>> default="true">
>>>>>  <lst name="defaults">
>>>>>    <str name="hl.snippets">3</str>
>>>>>    <str name="hl.fragsize">100</str>
>>>>>    <str name="hl.regex.slop">0.5</str>
>>>>>    <str name="hl.regex.pattern">\w[-\w ,/\n\"']{50,150}</str>
>>>>>  </lst>
>>>>> </fragmenter>
>>>>>
>>>>> I tried setting hl.fragmenter=regex as a parameter as well, to be sure
>>>>>
>>>> that
>>>>
>>>>> it was using the correct one, and the result set is the same. Any ideas
>>>>>
>>>> what
>>>>
>>>>> could be causing this attribute not to be read? It has me concerned
>>>>> that
>>>>> other attributes are being ignored as well.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Mark Ferguson
>>>>>
>>>>>
>>>>
>>>
>>>
>