How can i Tokenize money values?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

How can i Tokenize money values?

Gustavo Scrigna
Hello all!,
    How can i tokenize money values? 
    Example: $25000, u$s45000, etc, so that i can search for "$25000" or "$250*"
    I think de "StandardTokenizer" class is the responsible for tokenize the content of the field based on the grammar generated by javaCC, the question is: I have to override the StandardTokenizer or i can use Filter's to solve this problem?
 
Thank's in advance!
 
    Gustavo //
 

No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.405 / Virus Database: 268.10.8/415 - Release Date: 09/08/2006


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: How can i Tokenize money values?

Erick Erickson
I'd do neither <G> You can look at other analyzers, WhitespaceAnalyzer comes
to mind, breaks on whitespace and leavs all special characters in. There are
several to choose from.

And, if you are indexing other fields and want them handled differently, use
a PerFieldAnalyzerWrapper.

Finally, you might consider indexing the same value in more than one field
for different purposes (e.g. searching/displaying).

Best
Erick

On 8/11/06, Gustavo Scrigna <[hidden email]> wrote:

>
>  Hello all!,
>     How can i tokenize money values?
>     Example: $25000, u$s45000, etc, so that i can search for "$25000" or
> "$250*"
>     I think de "StandardTokenizer"* *class is the responsible for tokenize
> the content of the field based on the grammar generated by javaCC, the
> question is: I have to override the StandardTokenizer or i can use Filter's
> to solve this problem?
>
> Thank's in advance!
>
>     Gustavo //
>
>
> No virus found in this outgoing message.
> Checked by AVG Free Edition.
> Version: 7.1.405 / Virus Database: 268.10.8/415 - Release Date: 09/08/2006
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Highlighter

Mark Miller-3
Am I the only one that gets back a string missing the final character
when using the highlighter and the null fragmenter? I always have to add
the last character of what I have asked to be highlighted to what the
highlighter returns when trying to hit highlight an entire
document...anyone else every run into this?

cheers ( don't I wish i was British),

- mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Highlighter

Ronnie Kolehmainen
There is an issue in JIRA, see http://issues.apache.org/jira/browse/LUCENE-645

So I guess you're not the only one.

/Ronnie

Citerar Mark Miller <[hidden email]>:

> Am I the only one that gets back a string missing the final character
> when using the highlighter and the null fragmenter? I always have to add
> the last character of what I have asked to be highlighted to what the
> highlighter returns when trying to hit highlight an entire
> document...anyone else every run into this?
>
> cheers ( don't I wish i was British),
>
> - mark
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Highlighter

Ramesh Salla
which version of Lucene and which version of Highlighter, do you use.
I dont see any such issues?
I think,  I can resolve the issue,  if you can pass on a few info on you are trying to get the data and highlight things.




On Sat, 2006-08-12 at 00:05 +0000, Ronnie Kolehmainen wrote:
There is an issue in JIRA, see http://issues.apache.org/jira/browse/LUCENE-645

So I guess you're not the only one.

/Ronnie

Citerar Mark Miller <[hidden email]>:

> Am I the only one that gets back a string missing the final character 
> when using the highlighter and the null fragmenter? I always have to add 
> the last character of what I have asked to be highlighted to what the 
> highlighter returns when trying to hit highlight an entire 
> document...anyone else every run into this?
> 
> cheers ( don't I wish i was British),
> 
> - mark
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
> 
> 




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


Reply | Threaded
Open this post in threaded view
|

Re: Highlighter

Bill Taylor-2
[hidden email] told me that the highlighter ALWAYS does this
under certain conditions.  In my case, it is when the string ends with
<BR>.  He knew why but I did not.  I just fixed it in my code by
putting things back.

On Aug 16, 2006, at 3:17 AM, Ramesh Salla wrote:

>  which version of Lucene and which version of Highlighter, do you use.
>  I dont see any such issues?
>  I think,  I can resolve the issue,  if you can pass on a few info on
> you are trying to get the data and highlight things.
>
> <image.tiff>
>
>
>  On Sat, 2006-08-12 at 00:05 +0000, Ronnie Kolehmainen wrote:
>> There is an issue in JIRA, see
>> http://issues.apache.org/jira/browse/LUCENE-645
>>
>> So I guess you're not the only one.
>>
>> /Ronnie
>>
>> Citerar Mark Miller <[hidden email]>:
>>
>> > Am I the only one that gets back a string missing the final
>> character
>> > when using the highlighter and the null fragmenter? I always have
>> to add
>> > the last character of what I have asked to be highlighted to what
>> the
>> > highlighter returns when trying to hit highlight an entire
>> > document...anyone else every run into this?
>> >
>> > cheers ( don't I wish i was British),
>> >
>> > - mark
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: [hidden email]
>> > For additional commands, e-mail: [hidden email]
>> >
>> >
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>>
Reply | Threaded
Open this post in threaded view
|

Re: Highlighter

Mark Miller-3
The reason has already been posted in response to my initial inquiry.
This problem bugged me last month. I did not know the particulars but I
assumed it was a bug.

I inquired on the mailing list and someone responded with the following
link:

Highligter fails to include non-token at end of string to be highlighted
http://issues.apache.org/jira/browse/LUCENE-645

That seemed to resolve the issue for me. It's a bug and it has not been
fixed and may be slightly complicated. Thanks to that guy who gave the
link, I don't recall who it was.

Case closed on this one from my end.

- Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]