Sending text into a number field

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Sending text into a number field

John Blythe
hey all,

i'm tossing a lot of mud against the wall and am wanting to see what
sticks. part of that includes throwing item descriptions against some
fields i've set up as doubles. the imported data is a double and some of
the descriptions will have the related data within it (product sizes, e.g.
"Super awesome product 10 x 20cm"). is there a way to throw text at a
number field (tdouble) and it only analyze the numbers instead of throwing
an error?

thanks for any info!
Reply | Threaded
Open this post in threaded view
|

Re: Sending text into a number field

Malcolm Upayavira Holmes


On Fri, 11 Mar 2016, at 03:19 PM, John Blythe wrote:

> hey all,
>
> i'm tossing a lot of mud against the wall and am wanting to see what
> sticks. part of that includes throwing item descriptions against some
> fields i've set up as doubles. the imported data is a double and some of
> the descriptions will have the related data within it (product sizes,
> e.g.
> "Super awesome product 10 x 20cm"). is there a way to throw text at a
> number field (tdouble) and it only analyze the numbers instead of
> throwing
> an error?
>
> thanks for any info!

If you really must do that on the Solr side, I'd suggest you try doing
it in an UpdateProcessor. You can either code these in Java, or in a
scripting language with the StatelessScriptUpdateProcessor. You could
strip out all of the non-numeric characters before they get to the
index.

Upayavira
Reply | Threaded
Open this post in threaded view
|

Re: Sending text into a number field

Alessandro Benedetti
I agree with Upayavira,
this is an information extraction task, you need to implement your logic to
extract the proper numeric values from the textual field.
Your update request processor could be as simple as you want in extracting
the number and setting them in numeric fields.
So this task is responsibility of a component that process the original
field value.
It is not responsibility of the tdouble field type.

Cheers

On 11 March 2016 at 15:29, Upayavira <[hidden email]> wrote:

>
>
> On Fri, 11 Mar 2016, at 03:19 PM, John Blythe wrote:
> > hey all,
> >
> > i'm tossing a lot of mud against the wall and am wanting to see what
> > sticks. part of that includes throwing item descriptions against some
> > fields i've set up as doubles. the imported data is a double and some of
> > the descriptions will have the related data within it (product sizes,
> > e.g.
> > "Super awesome product 10 x 20cm"). is there a way to throw text at a
> > number field (tdouble) and it only analyze the numbers instead of
> > throwing
> > an error?
> >
> > thanks for any info!
>
> If you really must do that on the Solr side, I'd suggest you try doing
> it in an UpdateProcessor. You can either code these in Java, or in a
> scripting language with the StatelessScriptUpdateProcessor. You could
> strip out all of the non-numeric characters before they get to the
> index.
>
> Upayavira
>



--
--------------------------

Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
Reply | Threaded
Open this post in threaded view
|

Re: Sending text into a number field

Jack Krupansky-3
It might be nice to have a specialized update processor for this common
case of wanting to specify two separate but related numeric fields using
one string. IOW, parse out two numbers and then send them to two separate
fields. Seems doable, either as a script or in Java. The script/processor
could take three field names and a flag: the raw input source, the two
destination fields, and whether the raw source field should be removed or
passed through (presumably into a text/string field.)

(If I was still updating my old Solr 4.x Deep Dive book I'd be adding that
script right now, but... that's not happening.)

-- Jack Krupansky

On Fri, Mar 11, 2016 at 11:03 AM, Alessandro Benedetti <
[hidden email]> wrote:

> I agree with Upayavira,
> this is an information extraction task, you need to implement your logic to
> extract the proper numeric values from the textual field.
> Your update request processor could be as simple as you want in extracting
> the number and setting them in numeric fields.
> So this task is responsibility of a component that process the original
> field value.
> It is not responsibility of the tdouble field type.
>
> Cheers
>
> On 11 March 2016 at 15:29, Upayavira <[hidden email]> wrote:
>
> >
> >
> > On Fri, 11 Mar 2016, at 03:19 PM, John Blythe wrote:
> > > hey all,
> > >
> > > i'm tossing a lot of mud against the wall and am wanting to see what
> > > sticks. part of that includes throwing item descriptions against some
> > > fields i've set up as doubles. the imported data is a double and some
> of
> > > the descriptions will have the related data within it (product sizes,
> > > e.g.
> > > "Super awesome product 10 x 20cm"). is there a way to throw text at a
> > > number field (tdouble) and it only analyze the numbers instead of
> > > throwing
> > > an error?
> > >
> > > thanks for any info!
> >
> > If you really must do that on the Solr side, I'd suggest you try doing
> > it in an UpdateProcessor. You can either code these in Java, or in a
> > scripting language with the StatelessScriptUpdateProcessor. You could
> > strip out all of the non-numeric characters before they get to the
> > index.
> >
> > Upayavira
> >
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>
Reply | Threaded
Open this post in threaded view
|

Re: Sending text into a number field

John Blythe
In reply to this post by Alessandro Benedetti
makes sense. could i set up a simple regex filter in a placeholder field of
sorts and then copy that field into my tdouble field?

--
*John Blythe*
Product Manager & Lead Developer

251.605.3071 | [hidden email]
www.curvolabs.com

58 Adams Ave
Evansville, IN 47713

On Fri, Mar 11, 2016 at 11:03 AM, Alessandro Benedetti <
[hidden email]> wrote:

> I agree with Upayavira,
> this is an information extraction task, you need to implement your logic to
> extract the proper numeric values from the textual field.
> Your update request processor could be as simple as you want in extracting
> the number and setting them in numeric fields.
> So this task is responsibility of a component that process the original
> field value.
> It is not responsibility of the tdouble field type.
>
> Cheers
>
> On 11 March 2016 at 15:29, Upayavira <[hidden email]> wrote:
>
> >
> >
> > On Fri, 11 Mar 2016, at 03:19 PM, John Blythe wrote:
> > > hey all,
> > >
> > > i'm tossing a lot of mud against the wall and am wanting to see what
> > > sticks. part of that includes throwing item descriptions against some
> > > fields i've set up as doubles. the imported data is a double and some
> of
> > > the descriptions will have the related data within it (product sizes,
> > > e.g.
> > > "Super awesome product 10 x 20cm"). is there a way to throw text at a
> > > number field (tdouble) and it only analyze the numbers instead of
> > > throwing
> > > an error?
> > >
> > > thanks for any info!
> >
> > If you really must do that on the Solr side, I'd suggest you try doing
> > it in an UpdateProcessor. You can either code these in Java, or in a
> > scripting language with the StatelessScriptUpdateProcessor. You could
> > strip out all of the non-numeric characters before they get to the
> > index.
> >
> > Upayavira
> >
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>
Reply | Threaded
Open this post in threaded view
|

Re: Sending text into a number field

Alessandro Benedetti
Copyfield won't work if this is what you meant. The source content of the
field ( prior any analysis) is extracted and then sent to the copy field
where it is analysed.

You need to simulate that approach in the process method of the update
request processor.
But i assume that that was what you thought altready.

To answer Jack, the lat lon field type, does already a similar thing under
the hood ( and store the lat in one dynamic field and the lon in another
dynamic field ) .
A similar ( but parametric approach) could be used !

Cheers

On 11 March 2016 at 16:17, John Blythe <[hidden email]> wrote:

> makes sense. could i set up a simple regex filter in a placeholder field of
> sorts and then copy that field into my tdouble field?
>
> --
> *John Blythe*
> Product Manager & Lead Developer
>
> 251.605.3071 | [hidden email]
> www.curvolabs.com
>
> 58 Adams Ave
> Evansville, IN 47713
>
> On Fri, Mar 11, 2016 at 11:03 AM, Alessandro Benedetti <
> [hidden email]> wrote:
>
> > I agree with Upayavira,
> > this is an information extraction task, you need to implement your logic
> to
> > extract the proper numeric values from the textual field.
> > Your update request processor could be as simple as you want in
> extracting
> > the number and setting them in numeric fields.
> > So this task is responsibility of a component that process the original
> > field value.
> > It is not responsibility of the tdouble field type.
> >
> > Cheers
> >
> > On 11 March 2016 at 15:29, Upayavira <[hidden email]> wrote:
> >
> > >
> > >
> > > On Fri, 11 Mar 2016, at 03:19 PM, John Blythe wrote:
> > > > hey all,
> > > >
> > > > i'm tossing a lot of mud against the wall and am wanting to see what
> > > > sticks. part of that includes throwing item descriptions against some
> > > > fields i've set up as doubles. the imported data is a double and some
> > of
> > > > the descriptions will have the related data within it (product sizes,
> > > > e.g.
> > > > "Super awesome product 10 x 20cm"). is there a way to throw text at a
> > > > number field (tdouble) and it only analyze the numbers instead of
> > > > throwing
> > > > an error?
> > > >
> > > > thanks for any info!
> > >
> > > If you really must do that on the Solr side, I'd suggest you try doing
> > > it in an UpdateProcessor. You can either code these in Java, or in a
> > > scripting language with the StatelessScriptUpdateProcessor. You could
> > > strip out all of the non-numeric characters before they get to the
> > > index.
> > >
> > > Upayavira
> > >
> >
> >
> >
> > --
> > --------------------------
> >
> > Benedetti Alessandro
> > Visiting card : http://about.me/alessandro_benedetti
> >
> > "Tyger, tyger burning bright
> > In the forests of the night,
> > What immortal hand or eye
> > Could frame thy fearful symmetry?"
> >
> > William Blake - Songs of Experience -1794 England
> >
>



--
--------------------------

Benedetti Alessandro
Visiting card : http://about.me/alessandro_benedetti

"Tyger, tyger burning bright
In the forests of the night,
What immortal hand or eye
Could frame thy fearful symmetry?"

William Blake - Songs of Experience -1794 England
---------------
Alessandro Benedetti
Search Consultant, R&D Software Engineer, Director
Sease Ltd. - www.sease.io
Reply | Threaded
Open this post in threaded view
|

Re: Sending text into a number field

John Blythe
gotcha. thanks for the tips guys

best,

--
*John Blythe*
Product Manager & Lead Developer

251.605.3071 | [hidden email]
www.curvolabs.com

58 Adams Ave
Evansville, IN 47713

On Fri, Mar 11, 2016 at 11:25 AM, Alessandro Benedetti <
[hidden email]> wrote:

> Copyfield won't work if this is what you meant. The source content of the
> field ( prior any analysis) is extracted and then sent to the copy field
> where it is analysed.
>
> You need to simulate that approach in the process method of the update
> request processor.
> But i assume that that was what you thought altready.
>
> To answer Jack, the lat lon field type, does already a similar thing under
> the hood ( and store the lat in one dynamic field and the lon in another
> dynamic field ) .
> A similar ( but parametric approach) could be used !
>
> Cheers
>
> On 11 March 2016 at 16:17, John Blythe <[hidden email]> wrote:
>
> > makes sense. could i set up a simple regex filter in a placeholder field
> of
> > sorts and then copy that field into my tdouble field?
> >
> > --
> > *John Blythe*
> > Product Manager & Lead Developer
> >
> > 251.605.3071 | [hidden email]
> > www.curvolabs.com
> >
> > 58 Adams Ave
> > Evansville, IN 47713
> >
> > On Fri, Mar 11, 2016 at 11:03 AM, Alessandro Benedetti <
> > [hidden email]> wrote:
> >
> > > I agree with Upayavira,
> > > this is an information extraction task, you need to implement your
> logic
> > to
> > > extract the proper numeric values from the textual field.
> > > Your update request processor could be as simple as you want in
> > extracting
> > > the number and setting them in numeric fields.
> > > So this task is responsibility of a component that process the original
> > > field value.
> > > It is not responsibility of the tdouble field type.
> > >
> > > Cheers
> > >
> > > On 11 March 2016 at 15:29, Upayavira <[hidden email]> wrote:
> > >
> > > >
> > > >
> > > > On Fri, 11 Mar 2016, at 03:19 PM, John Blythe wrote:
> > > > > hey all,
> > > > >
> > > > > i'm tossing a lot of mud against the wall and am wanting to see
> what
> > > > > sticks. part of that includes throwing item descriptions against
> some
> > > > > fields i've set up as doubles. the imported data is a double and
> some
> > > of
> > > > > the descriptions will have the related data within it (product
> sizes,
> > > > > e.g.
> > > > > "Super awesome product 10 x 20cm"). is there a way to throw text
> at a
> > > > > number field (tdouble) and it only analyze the numbers instead of
> > > > > throwing
> > > > > an error?
> > > > >
> > > > > thanks for any info!
> > > >
> > > > If you really must do that on the Solr side, I'd suggest you try
> doing
> > > > it in an UpdateProcessor. You can either code these in Java, or in a
> > > > scripting language with the StatelessScriptUpdateProcessor. You could
> > > > strip out all of the non-numeric characters before they get to the
> > > > index.
> > > >
> > > > Upayavira
> > > >
> > >
> > >
> > >
> > > --
> > > --------------------------
> > >
> > > Benedetti Alessandro
> > > Visiting card : http://about.me/alessandro_benedetti
> > >
> > > "Tyger, tyger burning bright
> > > In the forests of the night,
> > > What immortal hand or eye
> > > Could frame thy fearful symmetry?"
> > >
> > > William Blake - Songs of Experience -1794 England
> > >
> >
>
>
>
> --
> --------------------------
>
> Benedetti Alessandro
> Visiting card : http://about.me/alessandro_benedetti
>
> "Tyger, tyger burning bright
> In the forests of the night,
> What immortal hand or eye
> Could frame thy fearful symmetry?"
>
> William Blake - Songs of Experience -1794 England
>