FastVectorHighlighter ignoring fragmenter parameter . . .

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

FastVectorHighlighter ignoring fragmenter parameter . . .

Christopher Ball-3
Got the FVH to work in Solr 3.1 (or at least I presume I have given I
can see multi-color highlighting in the output.)

But I am not able to get it to recognize the "regex" fragmenter. I get
no change in output if I specify the fragmenter. In fact, I can even
enter bogus names for the fragmenter and get no change in the output.

Grateful for any suggestions.

Settings and output below.

Christopher


*Query*

    http://localhost:8983/solr/10k-Fragments/select?
    q=content%3Aliquidity
    &rows=100
    &fl=id%2Ccontent
    &qt=standard
    &hl.fl=content
    &hl.useFastVectorHighlighter=true
    &hl=true
    &hl.fragmentsBuilder=colored
    &hl.fragmenter=regex

*Response* (Abbreviated)

    <response>
    -
    <lst name="responseHeader">
    <int name="status">0</int>
    <int name="QTime">47</int>
    -
    <lst name="params">
    <str name="fl">id,content</str>
    <str name="hl.useFastVectorHighlighter">true</str>
    <str name="q">content:liquidity</str>
    <str name="hl.fragmenter">regex1text</str>
    <str name="hl.fl">content</str>
    <str name="hl.fragmentsBuilder">colored</str>
    <str name="qt">standard</str>
    <str name="hl">true</str>
    <str name="rows">100</str>
    </lst>
    </lst>
    . . .
    <lst name="highlighting">
    -
    <lst
    name="10K/1997-12-31/1998-04-01/1stBergenBancorp/0001005016/ManagementsDiscussionAndAnalysisOfFinancialConditionAndResultsOfOperations/LiquidityAndCapitalResource/paragraph/1/mh1261">
    -
    <arr name="content">
    -
    <str>
    &#4504; <b style="background:yellow">Liquidity</b> is a measure of a
    bank's ability to fund loans and withdrawals of deposits in a cost-ef
    </str>
    </arr>
    </lst>
    . . .

*Field listing in schema.xml*

    <field name="content" type="text" indexed="true" stored="true"
    termVectors="true" termPositions="true" termOffsets="true"/>

*Highlighter listing in solrconfig.xml*

    <highlighting>

    <fragmenter name="gap"
    class="org.apache.solr.highlight.GapFragmenter" default="true">
    <lst name="defaults">
    <int name="hl.fragsize">100</int>
    </lst>
    </fragmenter>
    <fragmenter name="regex"
    class="org.apache.solr.highlight.RegexFragmenter" >
    <lst name="defaults">
    <int name="hl.fragsize">70</int>
    <float name="hl.regex.slop">0.5</float>
    <str name="hl.regex.pattern">[-\w ,/\n\"']{20,200}</str>
    </lst>
    </fragmenter>

    <formatter name="html"
    class="org.apache.solr.highlight.HtmlFormatter" default="true">
    <lst name="defaults">
    <str name="hl.simple.pre"><![CDATA[<em>]]></str>
    <str name="hl.simple.post"><![CDATA[</em>]]></str>
    </lst>
    </formatter>

    <!-- Configure the standard encoder -->
    <encoder name="html" class="org.apache.solr.highlight.HtmlEncoder"
    default="true"/>

    <!-- Configure the standard fragListBuilder -->
    <fragListBuilder name="simple"
    class="org.apache.solr.highlight.SimpleFragListBuilder" default="true"/>

    <!-- multi-colored tag FragmentsBuilder -->
    <fragmentsBuilder name="colored"
    class="org.apache.solr.highlight.ScoreOrderFragmentsBuilder"
    default="true">
    <lst name="defaults">
    <str name="hl.tag.pre"><![CDATA[
    <b style="background:yellow">,<b style="background:lawgreen">,
    <b style="background:aquamarine">,<b style="background:magenta">,
    <b style="background:palegreen">,<b style="background:coral">,
    <b style="background:wheat">,<b style="background:khaki">,
    <b style="background:lime">,<b style="background:deepskyblue">]]></str>
    <str name="hl.tag.post"><![CDATA[</b>]]></str>
    </lst>
    </fragmentsBuilder>
    </highlighting>

Reply | Threaded
Open this post in threaded view
|

Re: FastVectorHighlighter ignoring fragmenter parameter . . .

Koji Sekiguchi
(10/12/05 5:53), CRB wrote:

> Got the FVH to work in Solr 3.1 (or at least I presume I have given I can see multi-color
> highlighting in the output.)
>
> But I am not able to get it to recognize the "regex" fragmenter. I get no change in output if I
> specify the fragmenter. In fact, I can even enter bogus names for the fragmenter and get no change
> in the output.
>
> Grateful for any suggestions.
>
> Settings and output below.
>
> Christopher
>
>
> *Query*
>
> http://localhost:8983/solr/10k-Fragments/select?
> q=content%3Aliquidity
> &rows=100
> &fl=id%2Ccontent
> &qt=standard
> &hl.fl=content
> &hl.useFastVectorHighlighter=true
> &hl=true
> &hl.fragmentsBuilder=colored
> &hl.fragmenter=regex

Christopher,

Because algorithm of FVH is totally different from (traditional) highlighter,
FVH doesn't see hl.fragmenter and hl.formatter, but see hl.fragListBuilder
and hl.fragmentsBuilder instead. I think your settings and request/response
looks good except hl.fragmenter=regex. FVH simply ignores the parameter.

Koji
--
http://www.rondhuit.com/en/
Reply | Threaded
Open this post in threaded view
|

Re: FastVectorHighlighter ignoring fragmenter parameter . . .

Christopher Ball-3
Koji,

Thank you for the reply.

Being something of a novice with Solr, I would be grateful if you could
clarify my next steps.

I infer from your reply that there is no current implementation yet
contributed for the FVH similar to the regex fragmenter.

Thus I need to write my own custom extensions of *FragmentsBuilder
<http://lucene.apache.org/java/3_0_1/api/contrib-fast-vector-highlighter/org/apache/lucene/search/vectorhighlight/FragmentsBuilder.html>
& **FragListBuilder
<http://lucene.apache.org/java/3_0_1/api/contrib-fast-vector-highlighter/org/apache/lucene/search/vectorhighlight/FragListBuilder.html>
*interfaces to take in and apply the regex.

I would be happy to contribute back what I create.

Appreciate whatever guidance you can offer,

Christopher

On 2:59 PM, Koji Sekiguchi wrote:

> (10/12/05 5:53), CRB wrote:
>> Got the FVH to work in Solr 3.1 (or at least I presume I have given I
>> can see multi-color
>> highlighting in the output.)
>>
>> But I am not able to get it to recognize the "regex" fragmenter. I
>> get no change in output if I
>> specify the fragmenter. In fact, I can even enter bogus names for the
>> fragmenter and get no change
>> in the output.
>>
>> Grateful for any suggestions.
>>
>> Settings and output below.
>>
>> Christopher
>>
>>
>> *Query*
>>
>> http://localhost:8983/solr/10k-Fragments/select?
>> q=content%3Aliquidity
>> &rows=100
>> &fl=id%2Ccontent
>> &qt=standard
>> &hl.fl=content
>> &hl.useFastVectorHighlighter=true
>> &hl=true
>> &hl.fragmentsBuilder=colored
>> &hl.fragmenter=regex
>
> Christopher,
>
> Because algorithm of FVH is totally different from (traditional)
> highlighter,
> FVH doesn't see hl.fragmenter and hl.formatter, but see
> hl.fragListBuilder
> and hl.fragmentsBuilder instead. I think your settings and
> request/response
> looks good except hl.fragmenter=regex. FVH simply ignores the parameter.
>
> Koji

Reply | Threaded
Open this post in threaded view
|

Re: FastVectorHighlighter ignoring fragmenter parameter . . .

Christopher Ball-3
Koji,

Thank you for the reply.

Being something of a novice with Solr, I would be grateful if you could
clarify my next steps.

I infer from your reply that there is no current implementation yet
contributed for the FVH similar to the regex fragmenter.

Thus I need to write my own custom extensions of FragmentsBuilder &
FragListBuilder interfaces to take in and apply the regex.

I would be happy to contribute back what I create.

Appreciate whatever guidance you can offer,

Christopher
Reply | Threaded
Open this post in threaded view
|

Re: FastVectorHighlighter ignoring fragmenter parameter . . .

Koji Sekiguchi
In reply to this post by Christopher Ball-3
(10/12/06 23:52), CRB wrote:

> Koji,
>
> Thank you for the reply.
>
> Being something of a novice with Solr, I would be grateful if you could clarify my next steps.
>
> I infer from your reply that there is no current implementation yet contributed for the FVH similar
> to the regex fragmenter.
>
> Thus I need to write my own custom extensions of *FragmentsBuilder
> <http://lucene.apache.org/java/3_0_1/api/contrib-fast-vector-highlighter/org/apache/lucene/search/vectorhighlight/FragmentsBuilder.html>
> & **FragListBuilder
> <http://lucene.apache.org/java/3_0_1/api/contrib-fast-vector-highlighter/org/apache/lucene/search/vectorhighlight/FragListBuilder.html>
> *interfaces to take in and apply the regex.
>
> I would be happy to contribute back what I create.
>
> Appreciate whatever guidance you can offer,
>
> Christopher

Christopher,

Thank you for being interested in FVH!

As I'm not sure a regex-fragmenter-like-function can be implemented for FVH,
I cannot give any advise to you. Sorry about that.
Basically, contribution back is always welcome!

Thank you,

Koji
--
http://www.rondhuit.com/en/