Solr Payload example

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Solr Payload example

Vincenzo D'Amore
Hi all,

I'm trying to understand what I did wrong with a payload query that returns

error: {
metadata: [ "error-class", "org.apache.solr.common.SolrException",
"root-error-class", "org.apache.solr.common.SolrException" ],
msg: "No payload decoder found for field: colorCode",
code: 400
}

I have reduced my problem in a little sample to show what happens to me.
Basically I have a document with a couple of payload fields one
delimited_payloads_string and one delimited_payloads_integer

{
field_dps: "key|data",
field_dpi: "key|1",
}

When I execute this query solr returns as expected the payload for the key

q=*:*&fl=payload(field_dpi,key)

{
payload(field_dpi,key): 1
}

But for the strings there have to be something of different to do, because
I'm unable receive the payload value back. Executing this query, as in the
short introduction of this post, I receive an error.

?q=*:*&fl=payload(field_dps,key)

error: {
metadata: [ "error-class", "org.apache.solr.common.SolrException",
"root-error-class", "org.apache.solr.common.SolrException" ],
msg: "No payload decoder found for field: colorCode",
code: 400
}

Am I doing something wrong? How can I read strings payload data?

Thanks in advance for your time,
Vincenzo

--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: Solr Payload example

Vincenzo D'Amore
Sorry, I just realized that I was wrong in how I'm using the payload
function.
Give that the payload function only handles a numeric (integer or float)
payload, could you suggest me an alternative function that handles strings?
If not, should I write one?

On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <[hidden email]>
wrote:

> Hi all,
>
> I'm trying to understand what I did wrong with a payload query that
> returns
>
> error: {
> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> "root-error-class", "org.apache.solr.common.SolrException" ],
> msg: "No payload decoder found for field: colorCode",
> code: 400
> }
>
> I have reduced my problem in a little sample to show what happens to me.
> Basically I have a document with a couple of payload fields one
> delimited_payloads_string and one delimited_payloads_integer
>
> {
> field_dps: "key|data",
> field_dpi: "key|1",
> }
>
> When I execute this query solr returns as expected the payload for the key
>
> q=*:*&fl=payload(field_dpi,key)
>
> {
> payload(field_dpi,key): 1
> }
>
> But for the strings there have to be something of different to do, because
> I'm unable receive the payload value back. Executing this query, as in the
> short introduction of this post, I receive an error.
>
> ?q=*:*&fl=payload(field_dps,key)
>
> error: {
> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> "root-error-class", "org.apache.solr.common.SolrException" ],
> msg: "No payload decoder found for field: colorCode",
> code: 400
> }
>
> Am I doing something wrong? How can I read strings payload data?
>
> Thanks in advance for your time,
> Vincenzo
>
> --
> Vincenzo D'Amore
>
>

--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Erick Erickson
You’d need to write one. Payloads are generally intended to hold numerics you can then use in a function query to factor into the score…

Best,
Erick

> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]> wrote:
>
> Sorry, I just realized that I was wrong in how I'm using the payload
> function.
> Give that the payload function only handles a numeric (integer or float)
> payload, could you suggest me an alternative function that handles strings?
> If not, should I write one?
>
> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <[hidden email]>
> wrote:
>
>> Hi all,
>>
>> I'm trying to understand what I did wrong with a payload query that
>> returns
>>
>> error: {
>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>> "root-error-class", "org.apache.solr.common.SolrException" ],
>> msg: "No payload decoder found for field: colorCode",
>> code: 400
>> }
>>
>> I have reduced my problem in a little sample to show what happens to me.
>> Basically I have a document with a couple of payload fields one
>> delimited_payloads_string and one delimited_payloads_integer
>>
>> {
>> field_dps: "key|data",
>> field_dpi: "key|1",
>> }
>>
>> When I execute this query solr returns as expected the payload for the key
>>
>> q=*:*&fl=payload(field_dpi,key)
>>
>> {
>> payload(field_dpi,key): 1
>> }
>>
>> But for the strings there have to be something of different to do, because
>> I'm unable receive the payload value back. Executing this query, as in the
>> short introduction of this post, I receive an error.
>>
>> ?q=*:*&fl=payload(field_dps,key)
>>
>> error: {
>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>> "root-error-class", "org.apache.solr.common.SolrException" ],
>> msg: "No payload decoder found for field: colorCode",
>> code: 400
>> }
>>
>> Am I doing something wrong? How can I read strings payload data?
>>
>> Thanks in advance for your time,
>> Vincenzo
>>
>> --
>> Vincenzo D'Amore
>>
>>
>
> --
> Vincenzo D'Amore

Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Vincenzo D'Amore
Thanks Eric,

please, do you have a clue or a suggestion to give me in order to write
such function? Or should I write a payload decoder for the
delimited_payloads_string.
And again, if I have to write a decoder, would be easier if I found an
example to start with.

Best regards,
Vincenzo

On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <[hidden email]>
wrote:

> You’d need to write one. Payloads are generally intended to hold numerics
> you can then use in a function query to factor into the score…
>
> Best,
> Erick
>
> > On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
> wrote:
> >
> > Sorry, I just realized that I was wrong in how I'm using the payload
> > function.
> > Give that the payload function only handles a numeric (integer or float)
> > payload, could you suggest me an alternative function that handles
> strings?
> > If not, should I write one?
> >
> > On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <[hidden email]>
> > wrote:
> >
> >> Hi all,
> >>
> >> I'm trying to understand what I did wrong with a payload query that
> >> returns
> >>
> >> error: {
> >> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >> "root-error-class", "org.apache.solr.common.SolrException" ],
> >> msg: "No payload decoder found for field: colorCode",
> >> code: 400
> >> }
> >>
> >> I have reduced my problem in a little sample to show what happens to me.
> >> Basically I have a document with a couple of payload fields one
> >> delimited_payloads_string and one delimited_payloads_integer
> >>
> >> {
> >> field_dps: "key|data",
> >> field_dpi: "key|1",
> >> }
> >>
> >> When I execute this query solr returns as expected the payload for the
> key
> >>
> >> q=*:*&fl=payload(field_dpi,key)
> >>
> >> {
> >> payload(field_dpi,key): 1
> >> }
> >>
> >> But for the strings there have to be something of different to do,
> because
> >> I'm unable receive the payload value back. Executing this query, as in
> the
> >> short introduction of this post, I receive an error.
> >>
> >> ?q=*:*&fl=payload(field_dps,key)
> >>
> >> error: {
> >> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >> "root-error-class", "org.apache.solr.common.SolrException" ],
> >> msg: "No payload decoder found for field: colorCode",
> >> code: 400
> >> }
> >>
> >> Am I doing something wrong? How can I read strings payload data?
> >>
> >> Thanks in advance for your time,
> >> Vincenzo
> >>
> >> --
> >> Vincenzo D'Amore
> >>
> >>
> >
> > --
> > Vincenzo D'Amore
>
>

--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Vincenzo D'Amore
In reply to this post by Erick Erickson
Hi Erick,

It seems I've reached a dead-point, or at least it seems looking at the
code, it seems I can't  easily add a custom decoder:

Looking at PayloadUtils class there is getPayloadDecoder method invoked to
return the PayloadDecoder :

  public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
    PayloadDecoder decoder = null;

    String encoder = getPayloadEncoder(fieldType);

    if ("integer".equals(encoder)) {
      decoder = (BytesRef payload) -> payload == null ? 1 :
PayloadHelper.decodeInt(payload.bytes, payload.offset);
    }
    if ("float".equals(encoder)) {
      decoder = (BytesRef payload) -> payload == null ? 1 :
PayloadHelper.decodeFloat(payload.bytes, payload.offset);
    }
    // encoder could be "identity" at this point, in the case of
DelimitedTokenFilterFactory encoder="identity"

    // TODO: support pluggable payload decoders?

    return decoder;
  }

Any advice to work around this situation?


On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <[hidden email]>
wrote:

> You’d need to write one. Payloads are generally intended to hold numerics
> you can then use in a function query to factor into the score…
>
> Best,
> Erick
>
> > On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
> wrote:
> >
> > Sorry, I just realized that I was wrong in how I'm using the payload
> > function.
> > Give that the payload function only handles a numeric (integer or float)
> > payload, could you suggest me an alternative function that handles
> strings?
> > If not, should I write one?
> >
> > On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <[hidden email]>
> > wrote:
> >
> >> Hi all,
> >>
> >> I'm trying to understand what I did wrong with a payload query that
> >> returns
> >>
> >> error: {
> >> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >> "root-error-class", "org.apache.solr.common.SolrException" ],
> >> msg: "No payload decoder found for field: colorCode",
> >> code: 400
> >> }
> >>
> >> I have reduced my problem in a little sample to show what happens to me.
> >> Basically I have a document with a couple of payload fields one
> >> delimited_payloads_string and one delimited_payloads_integer
> >>
> >> {
> >> field_dps: "key|data",
> >> field_dpi: "key|1",
> >> }
> >>
> >> When I execute this query solr returns as expected the payload for the
> key
> >>
> >> q=*:*&fl=payload(field_dpi,key)
> >>
> >> {
> >> payload(field_dpi,key): 1
> >> }
> >>
> >> But for the strings there have to be something of different to do,
> because
> >> I'm unable receive the payload value back. Executing this query, as in
> the
> >> short introduction of this post, I receive an error.
> >>
> >> ?q=*:*&fl=payload(field_dps,key)
> >>
> >> error: {
> >> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >> "root-error-class", "org.apache.solr.common.SolrException" ],
> >> msg: "No payload decoder found for field: colorCode",
> >> code: 400
> >> }
> >>
> >> Am I doing something wrong? How can I read strings payload data?
> >>
> >> Thanks in advance for your time,
> >> Vincenzo
> >>
> >> --
> >> Vincenzo D'Amore
> >>
> >>
> >
> > --
> > Vincenzo D'Amore
>
>

--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Erick Erickson
This is one of those situations where I know a client did it, but didn’t see the code myself.

So I can’t help much.

Perhaps a good question at this point, though, is “why do you want to add string payloads anyway”?

This isn’t the client, but it might give you some pointers:

https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java

Best,
Erick

> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]> wrote:
>
> Hi Erick,
>
> It seems I've reached a dead-point, or at least it seems looking at the
> code, it seems I can't  easily add a custom decoder:
>
> Looking at PayloadUtils class there is getPayloadDecoder method invoked to
> return the PayloadDecoder :
>
>  public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
>    PayloadDecoder decoder = null;
>
>    String encoder = getPayloadEncoder(fieldType);
>
>    if ("integer".equals(encoder)) {
>      decoder = (BytesRef payload) -> payload == null ? 1 :
> PayloadHelper.decodeInt(payload.bytes, payload.offset);
>    }
>    if ("float".equals(encoder)) {
>      decoder = (BytesRef payload) -> payload == null ? 1 :
> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
>    }
>    // encoder could be "identity" at this point, in the case of
> DelimitedTokenFilterFactory encoder="identity"
>
>    // TODO: support pluggable payload decoders?
>
>    return decoder;
>  }
>
> Any advice to work around this situation?
>
>
> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <[hidden email]>
> wrote:
>
>> You’d need to write one. Payloads are generally intended to hold numerics
>> you can then use in a function query to factor into the score…
>>
>> Best,
>> Erick
>>
>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
>> wrote:
>>>
>>> Sorry, I just realized that I was wrong in how I'm using the payload
>>> function.
>>> Give that the payload function only handles a numeric (integer or float)
>>> payload, could you suggest me an alternative function that handles
>> strings?
>>> If not, should I write one?
>>>
>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <[hidden email]>
>>> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I'm trying to understand what I did wrong with a payload query that
>>>> returns
>>>>
>>>> error: {
>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>> msg: "No payload decoder found for field: colorCode",
>>>> code: 400
>>>> }
>>>>
>>>> I have reduced my problem in a little sample to show what happens to me.
>>>> Basically I have a document with a couple of payload fields one
>>>> delimited_payloads_string and one delimited_payloads_integer
>>>>
>>>> {
>>>> field_dps: "key|data",
>>>> field_dpi: "key|1",
>>>> }
>>>>
>>>> When I execute this query solr returns as expected the payload for the
>> key
>>>>
>>>> q=*:*&fl=payload(field_dpi,key)
>>>>
>>>> {
>>>> payload(field_dpi,key): 1
>>>> }
>>>>
>>>> But for the strings there have to be something of different to do,
>> because
>>>> I'm unable receive the payload value back. Executing this query, as in
>> the
>>>> short introduction of this post, I receive an error.
>>>>
>>>> ?q=*:*&fl=payload(field_dps,key)
>>>>
>>>> error: {
>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>> msg: "No payload decoder found for field: colorCode",
>>>> code: 400
>>>> }
>>>>
>>>> Am I doing something wrong? How can I read strings payload data?
>>>>
>>>> Thanks in advance for your time,
>>>> Vincenzo
>>>>
>>>> --
>>>> Vincenzo D'Amore
>>>>
>>>>
>>>
>>> --
>>> Vincenzo D'Amore
>>
>>
>
> --
> Vincenzo D'Amore

Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Vincenzo D'Amore
Hi Erick,

thanks for getting back to me. We started to use payloads because we have
the classical per-store pricing problem.
Thousands of stores across and different prices.
Then we found the payloads very useful started to use it for many reasons,
like enabling/disabling the product for such store, save the stock
availability, or save the other info like buy/sell price, discount rates,
and so on.
All those information are numbers, but stores can also be in different
countries, I mean would be useful also have the currency and other
attributes related to the store.

Thinking about an alternative for payloads maybe I could use the dynamic
fields, well, I know it is ugly.

Consider this hypothetical case where I have two field payload :

payloadPrice: [
"store1|125.0",
"store2|220.0",
"store3|225.0"
]

payloadCurrency: [
"store1|USD",
"store2|EUR",
"store3|GBP"
]

with dynamic fields I could have different fields for each document.

currency_store1_s: "USD"
currency_store2_s: "EUR"
currency_store3_s: "GBP"

But how many dynamic fields like this can I have? more than thousands?

Again, I've just started to look at solr-ocrhighlighting github project you
suggested.
Those seems have written their own payload object type where store ocr
highlighting information.
It seems interesting, I'll take a look immediately.

Thanks again for your time.

Best regards,
Vincenzo


On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <[hidden email]>
wrote:

> This is one of those situations where I know a client did it, but didn’t
> see the code myself.
>
> So I can’t help much.
>
> Perhaps a good question at this point, though, is “why do you want to add
> string payloads anyway”?
>
> This isn’t the client, but it might give you some pointers:
>
>
> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
>
> Best,
> Erick
>
> > On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
> wrote:
> >
> > Hi Erick,
> >
> > It seems I've reached a dead-point, or at least it seems looking at the
> > code, it seems I can't  easily add a custom decoder:
> >
> > Looking at PayloadUtils class there is getPayloadDecoder method invoked
> to
> > return the PayloadDecoder :
> >
> >  public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
> >    PayloadDecoder decoder = null;
> >
> >    String encoder = getPayloadEncoder(fieldType);
> >
> >    if ("integer".equals(encoder)) {
> >      decoder = (BytesRef payload) -> payload == null ? 1 :
> > PayloadHelper.decodeInt(payload.bytes, payload.offset);
> >    }
> >    if ("float".equals(encoder)) {
> >      decoder = (BytesRef payload) -> payload == null ? 1 :
> > PayloadHelper.decodeFloat(payload.bytes, payload.offset);
> >    }
> >    // encoder could be "identity" at this point, in the case of
> > DelimitedTokenFilterFactory encoder="identity"
> >
> >    // TODO: support pluggable payload decoders?
> >
> >    return decoder;
> >  }
> >
> > Any advice to work around this situation?
> >
> >
> > On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <[hidden email]>
> > wrote:
> >
> >> You’d need to write one. Payloads are generally intended to hold
> numerics
> >> you can then use in a function query to factor into the score…
> >>
> >> Best,
> >> Erick
> >>
> >>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
> >> wrote:
> >>>
> >>> Sorry, I just realized that I was wrong in how I'm using the payload
> >>> function.
> >>> Give that the payload function only handles a numeric (integer or
> float)
> >>> payload, could you suggest me an alternative function that handles
> >> strings?
> >>> If not, should I write one?
> >>>
> >>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <[hidden email]>
> >>> wrote:
> >>>
> >>>> Hi all,
> >>>>
> >>>> I'm trying to understand what I did wrong with a payload query that
> >>>> returns
> >>>>
> >>>> error: {
> >>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> >>>> msg: "No payload decoder found for field: colorCode",
> >>>> code: 400
> >>>> }
> >>>>
> >>>> I have reduced my problem in a little sample to show what happens to
> me.
> >>>> Basically I have a document with a couple of payload fields one
> >>>> delimited_payloads_string and one delimited_payloads_integer
> >>>>
> >>>> {
> >>>> field_dps: "key|data",
> >>>> field_dpi: "key|1",
> >>>> }
> >>>>
> >>>> When I execute this query solr returns as expected the payload for the
> >> key
> >>>>
> >>>> q=*:*&fl=payload(field_dpi,key)
> >>>>
> >>>> {
> >>>> payload(field_dpi,key): 1
> >>>> }
> >>>>
> >>>> But for the strings there have to be something of different to do,
> >> because
> >>>> I'm unable receive the payload value back. Executing this query, as in
> >> the
> >>>> short introduction of this post, I receive an error.
> >>>>
> >>>> ?q=*:*&fl=payload(field_dps,key)
> >>>>
> >>>> error: {
> >>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> >>>> msg: "No payload decoder found for field: colorCode",
> >>>> code: 400
> >>>> }
> >>>>
> >>>> Am I doing something wrong? How can I read strings payload data?
> >>>>
> >>>> Thanks in advance for your time,
> >>>> Vincenzo
> >>>>
> >>>> --
> >>>> Vincenzo D'Amore
> >>>>
> >>>>
> >>>
> >>> --
> >>> Vincenzo D'Amore
> >>
> >>
> >
> > --
> > Vincenzo D'Amore
>
>

--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Alexandre Rafalovitch
I remember several years ago a discussion/blog post about a similar
problem. The author went through a lot of thinking and decided that
the best way to deal with a similar problem was to have Solr documents
represent different level of abstraction, more granular.

IIRC, the equivalent for your example would be to represent 'pricing'
(or even pricing/availability) as the document, not a 'product'. You
may need to duplicate product field values for that to work. But that
- apparently - allowed them to represent a lot of concepts (like
upcoming discounts, etc) easier by just adding availability dates, etc
into that final lower-level record. And, of course, allowed to update
individual store/product price by changing one record per time without
having to invalidate the cache for all other stores.

Regards,
   Alex.


On Mon, 21 Oct 2019 at 09:59, Vincenzo D'Amore <[hidden email]> wrote:

>
> Hi Erick,
>
> thanks for getting back to me. We started to use payloads because we have
> the classical per-store pricing problem.
> Thousands of stores across and different prices.
> Then we found the payloads very useful started to use it for many reasons,
> like enabling/disabling the product for such store, save the stock
> availability, or save the other info like buy/sell price, discount rates,
> and so on.
> All those information are numbers, but stores can also be in different
> countries, I mean would be useful also have the currency and other
> attributes related to the store.
>
> Thinking about an alternative for payloads maybe I could use the dynamic
> fields, well, I know it is ugly.
>
> Consider this hypothetical case where I have two field payload :
>
> payloadPrice: [
> "store1|125.0",
> "store2|220.0",
> "store3|225.0"
> ]
>
> payloadCurrency: [
> "store1|USD",
> "store2|EUR",
> "store3|GBP"
> ]
>
> with dynamic fields I could have different fields for each document.
>
> currency_store1_s: "USD"
> currency_store2_s: "EUR"
> currency_store3_s: "GBP"
>
> But how many dynamic fields like this can I have? more than thousands?
>
> Again, I've just started to look at solr-ocrhighlighting github project you
> suggested.
> Those seems have written their own payload object type where store ocr
> highlighting information.
> It seems interesting, I'll take a look immediately.
>
> Thanks again for your time.
>
> Best regards,
> Vincenzo
>
>
> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <[hidden email]>
> wrote:
>
> > This is one of those situations where I know a client did it, but didn’t
> > see the code myself.
> >
> > So I can’t help much.
> >
> > Perhaps a good question at this point, though, is “why do you want to add
> > string payloads anyway”?
> >
> > This isn’t the client, but it might give you some pointers:
> >
> >
> > https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
> >
> > Best,
> > Erick
> >
> > > On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
> > wrote:
> > >
> > > Hi Erick,
> > >
> > > It seems I've reached a dead-point, or at least it seems looking at the
> > > code, it seems I can't  easily add a custom decoder:
> > >
> > > Looking at PayloadUtils class there is getPayloadDecoder method invoked
> > to
> > > return the PayloadDecoder :
> > >
> > >  public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
> > >    PayloadDecoder decoder = null;
> > >
> > >    String encoder = getPayloadEncoder(fieldType);
> > >
> > >    if ("integer".equals(encoder)) {
> > >      decoder = (BytesRef payload) -> payload == null ? 1 :
> > > PayloadHelper.decodeInt(payload.bytes, payload.offset);
> > >    }
> > >    if ("float".equals(encoder)) {
> > >      decoder = (BytesRef payload) -> payload == null ? 1 :
> > > PayloadHelper.decodeFloat(payload.bytes, payload.offset);
> > >    }
> > >    // encoder could be "identity" at this point, in the case of
> > > DelimitedTokenFilterFactory encoder="identity"
> > >
> > >    // TODO: support pluggable payload decoders?
> > >
> > >    return decoder;
> > >  }
> > >
> > > Any advice to work around this situation?
> > >
> > >
> > > On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <[hidden email]>
> > > wrote:
> > >
> > >> You’d need to write one. Payloads are generally intended to hold
> > numerics
> > >> you can then use in a function query to factor into the score…
> > >>
> > >> Best,
> > >> Erick
> > >>
> > >>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
> > >> wrote:
> > >>>
> > >>> Sorry, I just realized that I was wrong in how I'm using the payload
> > >>> function.
> > >>> Give that the payload function only handles a numeric (integer or
> > float)
> > >>> payload, could you suggest me an alternative function that handles
> > >> strings?
> > >>> If not, should I write one?
> > >>>
> > >>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <[hidden email]>
> > >>> wrote:
> > >>>
> > >>>> Hi all,
> > >>>>
> > >>>> I'm trying to understand what I did wrong with a payload query that
> > >>>> returns
> > >>>>
> > >>>> error: {
> > >>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> > >>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> > >>>> msg: "No payload decoder found for field: colorCode",
> > >>>> code: 400
> > >>>> }
> > >>>>
> > >>>> I have reduced my problem in a little sample to show what happens to
> > me.
> > >>>> Basically I have a document with a couple of payload fields one
> > >>>> delimited_payloads_string and one delimited_payloads_integer
> > >>>>
> > >>>> {
> > >>>> field_dps: "key|data",
> > >>>> field_dpi: "key|1",
> > >>>> }
> > >>>>
> > >>>> When I execute this query solr returns as expected the payload for the
> > >> key
> > >>>>
> > >>>> q=*:*&fl=payload(field_dpi,key)
> > >>>>
> > >>>> {
> > >>>> payload(field_dpi,key): 1
> > >>>> }
> > >>>>
> > >>>> But for the strings there have to be something of different to do,
> > >> because
> > >>>> I'm unable receive the payload value back. Executing this query, as in
> > >> the
> > >>>> short introduction of this post, I receive an error.
> > >>>>
> > >>>> ?q=*:*&fl=payload(field_dps,key)
> > >>>>
> > >>>> error: {
> > >>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> > >>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> > >>>> msg: "No payload decoder found for field: colorCode",
> > >>>> code: 400
> > >>>> }
> > >>>>
> > >>>> Am I doing something wrong? How can I read strings payload data?
> > >>>>
> > >>>> Thanks in advance for your time,
> > >>>> Vincenzo
> > >>>>
> > >>>> --
> > >>>> Vincenzo D'Amore
> > >>>>
> > >>>>
> > >>>
> > >>> --
> > >>> Vincenzo D'Amore
> > >>
> > >>
> > >
> > > --
> > > Vincenzo D'Amore
> >
> >
>
> --
> Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Erick Erickson
Don’t go to thousands of fields, it’s usually a bad idea for that many fields.

If all you’re doing is returning the data for display, have you considered indexing, perhaps in a separate field, a string token for each? Something like
store1_USD_125.00
?

Then have the UI split that apart in a pleasing manner.

Doesn’t work if you want to, say, sort by price though.

Erick

> On Oct 21, 2019, at 10:19 AM, Alexandre Rafalovitch <[hidden email]> wrote:
>
> I remember several years ago a discussion/blog post about a similar
> problem. The author went through a lot of thinking and decided that
> the best way to deal with a similar problem was to have Solr documents
> represent different level of abstraction, more granular.
>
> IIRC, the equivalent for your example would be to represent 'pricing'
> (or even pricing/availability) as the document, not a 'product'. You
> may need to duplicate product field values for that to work. But that
> - apparently - allowed them to represent a lot of concepts (like
> upcoming discounts, etc) easier by just adding availability dates, etc
> into that final lower-level record. And, of course, allowed to update
> individual store/product price by changing one record per time without
> having to invalidate the cache for all other stores.
>
> Regards,
>   Alex.
>
>
> On Mon, 21 Oct 2019 at 09:59, Vincenzo D'Amore <[hidden email]> wrote:
>>
>> Hi Erick,
>>
>> thanks for getting back to me. We started to use payloads because we have
>> the classical per-store pricing problem.
>> Thousands of stores across and different prices.
>> Then we found the payloads very useful started to use it for many reasons,
>> like enabling/disabling the product for such store, save the stock
>> availability, or save the other info like buy/sell price, discount rates,
>> and so on.
>> All those information are numbers, but stores can also be in different
>> countries, I mean would be useful also have the currency and other
>> attributes related to the store.
>>
>> Thinking about an alternative for payloads maybe I could use the dynamic
>> fields, well, I know it is ugly.
>>
>> Consider this hypothetical case where I have two field payload :
>>
>> payloadPrice: [
>> "store1|125.0",
>> "store2|220.0",
>> "store3|225.0"
>> ]
>>
>> payloadCurrency: [
>> "store1|USD",
>> "store2|EUR",
>> "store3|GBP"
>> ]
>>
>> with dynamic fields I could have different fields for each document.
>>
>> currency_store1_s: "USD"
>> currency_store2_s: "EUR"
>> currency_store3_s: "GBP"
>>
>> But how many dynamic fields like this can I have? more than thousands?
>>
>> Again, I've just started to look at solr-ocrhighlighting github project you
>> suggested.
>> Those seems have written their own payload object type where store ocr
>> highlighting information.
>> It seems interesting, I'll take a look immediately.
>>
>> Thanks again for your time.
>>
>> Best regards,
>> Vincenzo
>>
>>
>> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <[hidden email]>
>> wrote:
>>
>>> This is one of those situations where I know a client did it, but didn’t
>>> see the code myself.
>>>
>>> So I can’t help much.
>>>
>>> Perhaps a good question at this point, though, is “why do you want to add
>>> string payloads anyway”?
>>>
>>> This isn’t the client, but it might give you some pointers:
>>>
>>>
>>> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
>>>
>>> Best,
>>> Erick
>>>
>>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
>>> wrote:
>>>>
>>>> Hi Erick,
>>>>
>>>> It seems I've reached a dead-point, or at least it seems looking at the
>>>> code, it seems I can't  easily add a custom decoder:
>>>>
>>>> Looking at PayloadUtils class there is getPayloadDecoder method invoked
>>> to
>>>> return the PayloadDecoder :
>>>>
>>>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
>>>>   PayloadDecoder decoder = null;
>>>>
>>>>   String encoder = getPayloadEncoder(fieldType);
>>>>
>>>>   if ("integer".equals(encoder)) {
>>>>     decoder = (BytesRef payload) -> payload == null ? 1 :
>>>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
>>>>   }
>>>>   if ("float".equals(encoder)) {
>>>>     decoder = (BytesRef payload) -> payload == null ? 1 :
>>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
>>>>   }
>>>>   // encoder could be "identity" at this point, in the case of
>>>> DelimitedTokenFilterFactory encoder="identity"
>>>>
>>>>   // TODO: support pluggable payload decoders?
>>>>
>>>>   return decoder;
>>>> }
>>>>
>>>> Any advice to work around this situation?
>>>>
>>>>
>>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <[hidden email]>
>>>> wrote:
>>>>
>>>>> You’d need to write one. Payloads are generally intended to hold
>>> numerics
>>>>> you can then use in a function query to factor into the score…
>>>>>
>>>>> Best,
>>>>> Erick
>>>>>
>>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>> Sorry, I just realized that I was wrong in how I'm using the payload
>>>>>> function.
>>>>>> Give that the payload function only handles a numeric (integer or
>>> float)
>>>>>> payload, could you suggest me an alternative function that handles
>>>>> strings?
>>>>>> If not, should I write one?
>>>>>>
>>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <[hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I'm trying to understand what I did wrong with a payload query that
>>>>>>> returns
>>>>>>>
>>>>>>> error: {
>>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>>>>> msg: "No payload decoder found for field: colorCode",
>>>>>>> code: 400
>>>>>>> }
>>>>>>>
>>>>>>> I have reduced my problem in a little sample to show what happens to
>>> me.
>>>>>>> Basically I have a document with a couple of payload fields one
>>>>>>> delimited_payloads_string and one delimited_payloads_integer
>>>>>>>
>>>>>>> {
>>>>>>> field_dps: "key|data",
>>>>>>> field_dpi: "key|1",
>>>>>>> }
>>>>>>>
>>>>>>> When I execute this query solr returns as expected the payload for the
>>>>> key
>>>>>>>
>>>>>>> q=*:*&fl=payload(field_dpi,key)
>>>>>>>
>>>>>>> {
>>>>>>> payload(field_dpi,key): 1
>>>>>>> }
>>>>>>>
>>>>>>> But for the strings there have to be something of different to do,
>>>>> because
>>>>>>> I'm unable receive the payload value back. Executing this query, as in
>>>>> the
>>>>>>> short introduction of this post, I receive an error.
>>>>>>>
>>>>>>> ?q=*:*&fl=payload(field_dps,key)
>>>>>>>
>>>>>>> error: {
>>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>>>>> msg: "No payload decoder found for field: colorCode",
>>>>>>> code: 400
>>>>>>> }
>>>>>>>
>>>>>>> Am I doing something wrong? How can I read strings payload data?
>>>>>>>
>>>>>>> Thanks in advance for your time,
>>>>>>> Vincenzo
>>>>>>>
>>>>>>> --
>>>>>>> Vincenzo D'Amore
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Vincenzo D'Amore
>>>>>
>>>>>
>>>>
>>>> --
>>>> Vincenzo D'Amore
>>>
>>>
>>
>> --
>> Vincenzo D'Amore

Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Shawn Heisey-2
In reply to this post by Vincenzo D'Amore
On 10/21/2019 7:59 AM, Vincenzo D'Amore wrote:
> But how many dynamic fields like this can I have? more than thousands?

There are no actual limits on the number of fields, dynamic or not.  You
can have as many fields as you want per document, and as many fields as
you want across the whole index.

But as soon as you start wondering whether Solr can handle thousands of
anything, chances are good that there will be a performance wall.
Unless it's total document count in the index, which readily scales to
billions if the hardware is sized approriately.  It takes time to
iterate through parts of the index that are large, and not all aspects
can be accelerated by clever programming.

Thousands of fields per document would be harder for Solr to handle than
thousands of total fields across the whole index, but I would expect the
latter to eventually be a problem for some aspect of Lucene or Solr.

It is also possible that you can use thousands of fields and never see a
problem.  But we have seen it cause issues, so it's generally not
recommended.

Thanks,
Shawn
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Vincenzo D'Amore
In reply to this post by Erick Erickson
Hi all,

at last I've tried to write my own payload function, and it has worked
(pretty well) immediately!
Well, to be honest I'm a little puzzled, so I have applied immediately the
first rule: always beware when your code works at first run.

Now I'm testing it.

On Mon, Oct 21, 2019 at 4:37 PM Erick Erickson <[hidden email]>
wrote:

> Don’t go to thousands of fields, it’s usually a bad idea for that many
> fields.
>
> If all you’re doing is returning the data for display, have you considered
> indexing, perhaps in a separate field, a string token for each? Something
> like
> store1_USD_125.00
> ?
>
> Then have the UI split that apart in a pleasing manner.
>
> Doesn’t work if you want to, say, sort by price though.
>
> Erick
>
> > On Oct 21, 2019, at 10:19 AM, Alexandre Rafalovitch <[hidden email]>
> wrote:
> >
> > I remember several years ago a discussion/blog post about a similar
> > problem. The author went through a lot of thinking and decided that
> > the best way to deal with a similar problem was to have Solr documents
> > represent different level of abstraction, more granular.
> >
> > IIRC, the equivalent for your example would be to represent 'pricing'
> > (or even pricing/availability) as the document, not a 'product'. You
> > may need to duplicate product field values for that to work. But that
> > - apparently - allowed them to represent a lot of concepts (like
> > upcoming discounts, etc) easier by just adding availability dates, etc
> > into that final lower-level record. And, of course, allowed to update
> > individual store/product price by changing one record per time without
> > having to invalidate the cache for all other stores.
> >
> > Regards,
> >   Alex.
> >
> >
> > On Mon, 21 Oct 2019 at 09:59, Vincenzo D'Amore <[hidden email]>
> wrote:
> >>
> >> Hi Erick,
> >>
> >> thanks for getting back to me. We started to use payloads because we
> have
> >> the classical per-store pricing problem.
> >> Thousands of stores across and different prices.
> >> Then we found the payloads very useful started to use it for many
> reasons,
> >> like enabling/disabling the product for such store, save the stock
> >> availability, or save the other info like buy/sell price, discount
> rates,
> >> and so on.
> >> All those information are numbers, but stores can also be in different
> >> countries, I mean would be useful also have the currency and other
> >> attributes related to the store.
> >>
> >> Thinking about an alternative for payloads maybe I could use the dynamic
> >> fields, well, I know it is ugly.
> >>
> >> Consider this hypothetical case where I have two field payload :
> >>
> >> payloadPrice: [
> >> "store1|125.0",
> >> "store2|220.0",
> >> "store3|225.0"
> >> ]
> >>
> >> payloadCurrency: [
> >> "store1|USD",
> >> "store2|EUR",
> >> "store3|GBP"
> >> ]
> >>
> >> with dynamic fields I could have different fields for each document.
> >>
> >> currency_store1_s: "USD"
> >> currency_store2_s: "EUR"
> >> currency_store3_s: "GBP"
> >>
> >> But how many dynamic fields like this can I have? more than thousands?
> >>
> >> Again, I've just started to look at solr-ocrhighlighting github project
> you
> >> suggested.
> >> Those seems have written their own payload object type where store ocr
> >> highlighting information.
> >> It seems interesting, I'll take a look immediately.
> >>
> >> Thanks again for your time.
> >>
> >> Best regards,
> >> Vincenzo
> >>
> >>
> >> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <[hidden email]
> >
> >> wrote:
> >>
> >>> This is one of those situations where I know a client did it, but
> didn’t
> >>> see the code myself.
> >>>
> >>> So I can’t help much.
> >>>
> >>> Perhaps a good question at this point, though, is “why do you want to
> add
> >>> string payloads anyway”?
> >>>
> >>> This isn’t the client, but it might give you some pointers:
> >>>
> >>>
> >>>
> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
> >>>
> >>> Best,
> >>> Erick
> >>>
> >>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
> >>> wrote:
> >>>>
> >>>> Hi Erick,
> >>>>
> >>>> It seems I've reached a dead-point, or at least it seems looking at
> the
> >>>> code, it seems I can't  easily add a custom decoder:
> >>>>
> >>>> Looking at PayloadUtils class there is getPayloadDecoder method
> invoked
> >>> to
> >>>> return the PayloadDecoder :
> >>>>
> >>>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
> >>>>   PayloadDecoder decoder = null;
> >>>>
> >>>>   String encoder = getPayloadEncoder(fieldType);
> >>>>
> >>>>   if ("integer".equals(encoder)) {
> >>>>     decoder = (BytesRef payload) -> payload == null ? 1 :
> >>>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
> >>>>   }
> >>>>   if ("float".equals(encoder)) {
> >>>>     decoder = (BytesRef payload) -> payload == null ? 1 :
> >>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
> >>>>   }
> >>>>   // encoder could be "identity" at this point, in the case of
> >>>> DelimitedTokenFilterFactory encoder="identity"
> >>>>
> >>>>   // TODO: support pluggable payload decoders?
> >>>>
> >>>>   return decoder;
> >>>> }
> >>>>
> >>>> Any advice to work around this situation?
> >>>>
> >>>>
> >>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <
> [hidden email]>
> >>>> wrote:
> >>>>
> >>>>> You’d need to write one. Payloads are generally intended to hold
> >>> numerics
> >>>>> you can then use in a function query to factor into the score…
> >>>>>
> >>>>> Best,
> >>>>> Erick
> >>>>>
> >>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
> >>>>> wrote:
> >>>>>>
> >>>>>> Sorry, I just realized that I was wrong in how I'm using the payload
> >>>>>> function.
> >>>>>> Give that the payload function only handles a numeric (integer or
> >>> float)
> >>>>>> payload, could you suggest me an alternative function that handles
> >>>>> strings?
> >>>>>> If not, should I write one?
> >>>>>>
> >>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <
> [hidden email]>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi all,
> >>>>>>>
> >>>>>>> I'm trying to understand what I did wrong with a payload query that
> >>>>>>> returns
> >>>>>>>
> >>>>>>> error: {
> >>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> >>>>>>> msg: "No payload decoder found for field: colorCode",
> >>>>>>> code: 400
> >>>>>>> }
> >>>>>>>
> >>>>>>> I have reduced my problem in a little sample to show what happens
> to
> >>> me.
> >>>>>>> Basically I have a document with a couple of payload fields one
> >>>>>>> delimited_payloads_string and one delimited_payloads_integer
> >>>>>>>
> >>>>>>> {
> >>>>>>> field_dps: "key|data",
> >>>>>>> field_dpi: "key|1",
> >>>>>>> }
> >>>>>>>
> >>>>>>> When I execute this query solr returns as expected the payload for
> the
> >>>>> key
> >>>>>>>
> >>>>>>> q=*:*&fl=payload(field_dpi,key)
> >>>>>>>
> >>>>>>> {
> >>>>>>> payload(field_dpi,key): 1
> >>>>>>> }
> >>>>>>>
> >>>>>>> But for the strings there have to be something of different to do,
> >>>>> because
> >>>>>>> I'm unable receive the payload value back. Executing this query,
> as in
> >>>>> the
> >>>>>>> short introduction of this post, I receive an error.
> >>>>>>>
> >>>>>>> ?q=*:*&fl=payload(field_dps,key)
> >>>>>>>
> >>>>>>> error: {
> >>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> >>>>>>> msg: "No payload decoder found for field: colorCode",
> >>>>>>> code: 400
> >>>>>>> }
> >>>>>>>
> >>>>>>> Am I doing something wrong? How can I read strings payload data?
> >>>>>>>
> >>>>>>> Thanks in advance for your time,
> >>>>>>> Vincenzo
> >>>>>>>
> >>>>>>> --
> >>>>>>> Vincenzo D'Amore
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Vincenzo D'Amore
> >>>>>
> >>>>>
> >>>>
> >>>> --
> >>>> Vincenzo D'Amore
> >>>
> >>>
> >>
> >> --
> >> Vincenzo D'Amore
>
>

--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Erick Erickson
My tongue-in-cheek comment about how to tell veteran programmers from novices:

Novice: “My code compiled fine, it should run perfectly”

Veteran: “That can’t be right, my code appeared to succeed the first time I ran it.”...

> On Oct 21, 2019, at 12:54 PM, Vincenzo D'Amore <[hidden email]> wrote:
>
> Hi all,
>
> at last I've tried to write my own payload function, and it has worked
> (pretty well) immediately!
> Well, to be honest I'm a little puzzled, so I have applied immediately the
> first rule: always beware when your code works at first run.
>
> Now I'm testing it.
>
> On Mon, Oct 21, 2019 at 4:37 PM Erick Erickson <[hidden email]>
> wrote:
>
>> Don’t go to thousands of fields, it’s usually a bad idea for that many
>> fields.
>>
>> If all you’re doing is returning the data for display, have you considered
>> indexing, perhaps in a separate field, a string token for each? Something
>> like
>> store1_USD_125.00
>> ?
>>
>> Then have the UI split that apart in a pleasing manner.
>>
>> Doesn’t work if you want to, say, sort by price though.
>>
>> Erick
>>
>>> On Oct 21, 2019, at 10:19 AM, Alexandre Rafalovitch <[hidden email]>
>> wrote:
>>>
>>> I remember several years ago a discussion/blog post about a similar
>>> problem. The author went through a lot of thinking and decided that
>>> the best way to deal with a similar problem was to have Solr documents
>>> represent different level of abstraction, more granular.
>>>
>>> IIRC, the equivalent for your example would be to represent 'pricing'
>>> (or even pricing/availability) as the document, not a 'product'. You
>>> may need to duplicate product field values for that to work. But that
>>> - apparently - allowed them to represent a lot of concepts (like
>>> upcoming discounts, etc) easier by just adding availability dates, etc
>>> into that final lower-level record. And, of course, allowed to update
>>> individual store/product price by changing one record per time without
>>> having to invalidate the cache for all other stores.
>>>
>>> Regards,
>>>  Alex.
>>>
>>>
>>> On Mon, 21 Oct 2019 at 09:59, Vincenzo D'Amore <[hidden email]>
>> wrote:
>>>>
>>>> Hi Erick,
>>>>
>>>> thanks for getting back to me. We started to use payloads because we
>> have
>>>> the classical per-store pricing problem.
>>>> Thousands of stores across and different prices.
>>>> Then we found the payloads very useful started to use it for many
>> reasons,
>>>> like enabling/disabling the product for such store, save the stock
>>>> availability, or save the other info like buy/sell price, discount
>> rates,
>>>> and so on.
>>>> All those information are numbers, but stores can also be in different
>>>> countries, I mean would be useful also have the currency and other
>>>> attributes related to the store.
>>>>
>>>> Thinking about an alternative for payloads maybe I could use the dynamic
>>>> fields, well, I know it is ugly.
>>>>
>>>> Consider this hypothetical case where I have two field payload :
>>>>
>>>> payloadPrice: [
>>>> "store1|125.0",
>>>> "store2|220.0",
>>>> "store3|225.0"
>>>> ]
>>>>
>>>> payloadCurrency: [
>>>> "store1|USD",
>>>> "store2|EUR",
>>>> "store3|GBP"
>>>> ]
>>>>
>>>> with dynamic fields I could have different fields for each document.
>>>>
>>>> currency_store1_s: "USD"
>>>> currency_store2_s: "EUR"
>>>> currency_store3_s: "GBP"
>>>>
>>>> But how many dynamic fields like this can I have? more than thousands?
>>>>
>>>> Again, I've just started to look at solr-ocrhighlighting github project
>> you
>>>> suggested.
>>>> Those seems have written their own payload object type where store ocr
>>>> highlighting information.
>>>> It seems interesting, I'll take a look immediately.
>>>>
>>>> Thanks again for your time.
>>>>
>>>> Best regards,
>>>> Vincenzo
>>>>
>>>>
>>>> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <[hidden email]
>>>
>>>> wrote:
>>>>
>>>>> This is one of those situations where I know a client did it, but
>> didn’t
>>>>> see the code myself.
>>>>>
>>>>> So I can’t help much.
>>>>>
>>>>> Perhaps a good question at this point, though, is “why do you want to
>> add
>>>>> string payloads anyway”?
>>>>>
>>>>> This isn’t the client, but it might give you some pointers:
>>>>>
>>>>>
>>>>>
>> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
>>>>>
>>>>> Best,
>>>>> Erick
>>>>>
>>>>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>> Hi Erick,
>>>>>>
>>>>>> It seems I've reached a dead-point, or at least it seems looking at
>> the
>>>>>> code, it seems I can't  easily add a custom decoder:
>>>>>>
>>>>>> Looking at PayloadUtils class there is getPayloadDecoder method
>> invoked
>>>>> to
>>>>>> return the PayloadDecoder :
>>>>>>
>>>>>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
>>>>>>  PayloadDecoder decoder = null;
>>>>>>
>>>>>>  String encoder = getPayloadEncoder(fieldType);
>>>>>>
>>>>>>  if ("integer".equals(encoder)) {
>>>>>>    decoder = (BytesRef payload) -> payload == null ? 1 :
>>>>>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
>>>>>>  }
>>>>>>  if ("float".equals(encoder)) {
>>>>>>    decoder = (BytesRef payload) -> payload == null ? 1 :
>>>>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
>>>>>>  }
>>>>>>  // encoder could be "identity" at this point, in the case of
>>>>>> DelimitedTokenFilterFactory encoder="identity"
>>>>>>
>>>>>>  // TODO: support pluggable payload decoders?
>>>>>>
>>>>>>  return decoder;
>>>>>> }
>>>>>>
>>>>>> Any advice to work around this situation?
>>>>>>
>>>>>>
>>>>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <
>> [hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> You’d need to write one. Payloads are generally intended to hold
>>>>> numerics
>>>>>>> you can then use in a function query to factor into the score…
>>>>>>>
>>>>>>> Best,
>>>>>>> Erick
>>>>>>>
>>>>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Sorry, I just realized that I was wrong in how I'm using the payload
>>>>>>>> function.
>>>>>>>> Give that the payload function only handles a numeric (integer or
>>>>> float)
>>>>>>>> payload, could you suggest me an alternative function that handles
>>>>>>> strings?
>>>>>>>> If not, should I write one?
>>>>>>>>
>>>>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <
>> [hidden email]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi all,
>>>>>>>>>
>>>>>>>>> I'm trying to understand what I did wrong with a payload query that
>>>>>>>>> returns
>>>>>>>>>
>>>>>>>>> error: {
>>>>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>>>>>>> msg: "No payload decoder found for field: colorCode",
>>>>>>>>> code: 400
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> I have reduced my problem in a little sample to show what happens
>> to
>>>>> me.
>>>>>>>>> Basically I have a document with a couple of payload fields one
>>>>>>>>> delimited_payloads_string and one delimited_payloads_integer
>>>>>>>>>
>>>>>>>>> {
>>>>>>>>> field_dps: "key|data",
>>>>>>>>> field_dpi: "key|1",
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> When I execute this query solr returns as expected the payload for
>> the
>>>>>>> key
>>>>>>>>>
>>>>>>>>> q=*:*&fl=payload(field_dpi,key)
>>>>>>>>>
>>>>>>>>> {
>>>>>>>>> payload(field_dpi,key): 1
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> But for the strings there have to be something of different to do,
>>>>>>> because
>>>>>>>>> I'm unable receive the payload value back. Executing this query,
>> as in
>>>>>>> the
>>>>>>>>> short introduction of this post, I receive an error.
>>>>>>>>>
>>>>>>>>> ?q=*:*&fl=payload(field_dps,key)
>>>>>>>>>
>>>>>>>>> error: {
>>>>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>>>>>>> msg: "No payload decoder found for field: colorCode",
>>>>>>>>> code: 400
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> Am I doing something wrong? How can I read strings payload data?
>>>>>>>>>
>>>>>>>>> Thanks in advance for your time,
>>>>>>>>> Vincenzo
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Vincenzo D'Amore
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Vincenzo D'Amore
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Vincenzo D'Amore
>>>>>
>>>>>
>>>>
>>>> --
>>>> Vincenzo D'Amore
>>
>>
>
> --
> Vincenzo D'Amore

Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Erik Hatcher-4
In reply to this post by Vincenzo D'Amore
How about a single field, with terms like:

    store1_USD|125.0 store2_EUR|220.0 store3_GBP|225.0

Would that do the trick?    

And yeah, payload decoding is currently limited to float and int with the built-in payload() function.   We'd need a new way to pull out textual/bytes payloads - like maybe a DocTransformer?

        Erik


> On Oct 21, 2019, at 9:59 AM, Vincenzo D'Amore <[hidden email]> wrote:
>
> Hi Erick,
>
> thanks for getting back to me. We started to use payloads because we have
> the classical per-store pricing problem.
> Thousands of stores across and different prices.
> Then we found the payloads very useful started to use it for many reasons,
> like enabling/disabling the product for such store, save the stock
> availability, or save the other info like buy/sell price, discount rates,
> and so on.
> All those information are numbers, but stores can also be in different
> countries, I mean would be useful also have the currency and other
> attributes related to the store.
>
> Thinking about an alternative for payloads maybe I could use the dynamic
> fields, well, I know it is ugly.
>
> Consider this hypothetical case where I have two field payload :
>
> payloadPrice: [
> "store1|125.0",
> "store2|220.0",
> "store3|225.0"
> ]
>
> payloadCurrency: [
> "store1|USD",
> "store2|EUR",
> "store3|GBP"
> ]
>
> with dynamic fields I could have different fields for each document.
>
> currency_store1_s: "USD"
> currency_store2_s: "EUR"
> currency_store3_s: "GBP"
>
> But how many dynamic fields like this can I have? more than thousands?
>
> Again, I've just started to look at solr-ocrhighlighting github project you
> suggested.
> Those seems have written their own payload object type where store ocr
> highlighting information.
> It seems interesting, I'll take a look immediately.
>
> Thanks again for your time.
>
> Best regards,
> Vincenzo
>
>
> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <[hidden email]>
> wrote:
>
>> This is one of those situations where I know a client did it, but didn’t
>> see the code myself.
>>
>> So I can’t help much.
>>
>> Perhaps a good question at this point, though, is “why do you want to add
>> string payloads anyway”?
>>
>> This isn’t the client, but it might give you some pointers:
>>
>>
>> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
>>
>> Best,
>> Erick
>>
>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
>> wrote:
>>>
>>> Hi Erick,
>>>
>>> It seems I've reached a dead-point, or at least it seems looking at the
>>> code, it seems I can't  easily add a custom decoder:
>>>
>>> Looking at PayloadUtils class there is getPayloadDecoder method invoked
>> to
>>> return the PayloadDecoder :
>>>
>>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
>>>   PayloadDecoder decoder = null;
>>>
>>>   String encoder = getPayloadEncoder(fieldType);
>>>
>>>   if ("integer".equals(encoder)) {
>>>     decoder = (BytesRef payload) -> payload == null ? 1 :
>>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
>>>   }
>>>   if ("float".equals(encoder)) {
>>>     decoder = (BytesRef payload) -> payload == null ? 1 :
>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
>>>   }
>>>   // encoder could be "identity" at this point, in the case of
>>> DelimitedTokenFilterFactory encoder="identity"
>>>
>>>   // TODO: support pluggable payload decoders?
>>>
>>>   return decoder;
>>> }
>>>
>>> Any advice to work around this situation?
>>>
>>>
>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <[hidden email]>
>>> wrote:
>>>
>>>> You’d need to write one. Payloads are generally intended to hold
>> numerics
>>>> you can then use in a function query to factor into the score…
>>>>
>>>> Best,
>>>> Erick
>>>>
>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
>>>> wrote:
>>>>>
>>>>> Sorry, I just realized that I was wrong in how I'm using the payload
>>>>> function.
>>>>> Give that the payload function only handles a numeric (integer or
>> float)
>>>>> payload, could you suggest me an alternative function that handles
>>>> strings?
>>>>> If not, should I write one?
>>>>>
>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <[hidden email]>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> I'm trying to understand what I did wrong with a payload query that
>>>>>> returns
>>>>>>
>>>>>> error: {
>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>>>> msg: "No payload decoder found for field: colorCode",
>>>>>> code: 400
>>>>>> }
>>>>>>
>>>>>> I have reduced my problem in a little sample to show what happens to
>> me.
>>>>>> Basically I have a document with a couple of payload fields one
>>>>>> delimited_payloads_string and one delimited_payloads_integer
>>>>>>
>>>>>> {
>>>>>> field_dps: "key|data",
>>>>>> field_dpi: "key|1",
>>>>>> }
>>>>>>
>>>>>> When I execute this query solr returns as expected the payload for the
>>>> key
>>>>>>
>>>>>> q=*:*&fl=payload(field_dpi,key)
>>>>>>
>>>>>> {
>>>>>> payload(field_dpi,key): 1
>>>>>> }
>>>>>>
>>>>>> But for the strings there have to be something of different to do,
>>>> because
>>>>>> I'm unable receive the payload value back. Executing this query, as in
>>>> the
>>>>>> short introduction of this post, I receive an error.
>>>>>>
>>>>>> ?q=*:*&fl=payload(field_dps,key)
>>>>>>
>>>>>> error: {
>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>>>> msg: "No payload decoder found for field: colorCode",
>>>>>> code: 400
>>>>>> }
>>>>>>
>>>>>> Am I doing something wrong? How can I read strings payload data?
>>>>>>
>>>>>> Thanks in advance for your time,
>>>>>> Vincenzo
>>>>>>
>>>>>> --
>>>>>> Vincenzo D'Amore
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Vincenzo D'Amore
>>>>
>>>>
>>>
>>> --
>>> Vincenzo D'Amore
>>
>>
>
> --
> Vincenzo D'Amore

Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Eric Pugh-4
Have you checked out
https://github.com/o19s/payload-component

On Mon, Oct 21, 2019 at 2:47 PM Erik Hatcher <[hidden email]> wrote:

> How about a single field, with terms like:
>
>     store1_USD|125.0 store2_EUR|220.0 store3_GBP|225.0
>
> Would that do the trick?
>
> And yeah, payload decoding is currently limited to float and int with the
> built-in payload() function.   We'd need a new way to pull out
> textual/bytes payloads - like maybe a DocTransformer?
>
>         Erik
>
>
> > On Oct 21, 2019, at 9:59 AM, Vincenzo D'Amore <[hidden email]>
> wrote:
> >
> > Hi Erick,
> >
> > thanks for getting back to me. We started to use payloads because we have
> > the classical per-store pricing problem.
> > Thousands of stores across and different prices.
> > Then we found the payloads very useful started to use it for many
> reasons,
> > like enabling/disabling the product for such store, save the stock
> > availability, or save the other info like buy/sell price, discount rates,
> > and so on.
> > All those information are numbers, but stores can also be in different
> > countries, I mean would be useful also have the currency and other
> > attributes related to the store.
> >
> > Thinking about an alternative for payloads maybe I could use the dynamic
> > fields, well, I know it is ugly.
> >
> > Consider this hypothetical case where I have two field payload :
> >
> > payloadPrice: [
> > "store1|125.0",
> > "store2|220.0",
> > "store3|225.0"
> > ]
> >
> > payloadCurrency: [
> > "store1|USD",
> > "store2|EUR",
> > "store3|GBP"
> > ]
> >
> > with dynamic fields I could have different fields for each document.
> >
> > currency_store1_s: "USD"
> > currency_store2_s: "EUR"
> > currency_store3_s: "GBP"
> >
> > But how many dynamic fields like this can I have? more than thousands?
> >
> > Again, I've just started to look at solr-ocrhighlighting github project
> you
> > suggested.
> > Those seems have written their own payload object type where store ocr
> > highlighting information.
> > It seems interesting, I'll take a look immediately.
> >
> > Thanks again for your time.
> >
> > Best regards,
> > Vincenzo
> >
> >
> > On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <[hidden email]>
> > wrote:
> >
> >> This is one of those situations where I know a client did it, but didn’t
> >> see the code myself.
> >>
> >> So I can’t help much.
> >>
> >> Perhaps a good question at this point, though, is “why do you want to
> add
> >> string payloads anyway”?
> >>
> >> This isn’t the client, but it might give you some pointers:
> >>
> >>
> >>
> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
> >>
> >> Best,
> >> Erick
> >>
> >>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
> >> wrote:
> >>>
> >>> Hi Erick,
> >>>
> >>> It seems I've reached a dead-point, or at least it seems looking at the
> >>> code, it seems I can't  easily add a custom decoder:
> >>>
> >>> Looking at PayloadUtils class there is getPayloadDecoder method invoked
> >> to
> >>> return the PayloadDecoder :
> >>>
> >>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
> >>>   PayloadDecoder decoder = null;
> >>>
> >>>   String encoder = getPayloadEncoder(fieldType);
> >>>
> >>>   if ("integer".equals(encoder)) {
> >>>     decoder = (BytesRef payload) -> payload == null ? 1 :
> >>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
> >>>   }
> >>>   if ("float".equals(encoder)) {
> >>>     decoder = (BytesRef payload) -> payload == null ? 1 :
> >>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
> >>>   }
> >>>   // encoder could be "identity" at this point, in the case of
> >>> DelimitedTokenFilterFactory encoder="identity"
> >>>
> >>>   // TODO: support pluggable payload decoders?
> >>>
> >>>   return decoder;
> >>> }
> >>>
> >>> Any advice to work around this situation?
> >>>
> >>>
> >>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <
> [hidden email]>
> >>> wrote:
> >>>
> >>>> You’d need to write one. Payloads are generally intended to hold
> >> numerics
> >>>> you can then use in a function query to factor into the score…
> >>>>
> >>>> Best,
> >>>> Erick
> >>>>
> >>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
> >>>> wrote:
> >>>>>
> >>>>> Sorry, I just realized that I was wrong in how I'm using the payload
> >>>>> function.
> >>>>> Give that the payload function only handles a numeric (integer or
> >> float)
> >>>>> payload, could you suggest me an alternative function that handles
> >>>> strings?
> >>>>> If not, should I write one?
> >>>>>
> >>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <
> [hidden email]>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> I'm trying to understand what I did wrong with a payload query that
> >>>>>> returns
> >>>>>>
> >>>>>> error: {
> >>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> >>>>>> msg: "No payload decoder found for field: colorCode",
> >>>>>> code: 400
> >>>>>> }
> >>>>>>
> >>>>>> I have reduced my problem in a little sample to show what happens to
> >> me.
> >>>>>> Basically I have a document with a couple of payload fields one
> >>>>>> delimited_payloads_string and one delimited_payloads_integer
> >>>>>>
> >>>>>> {
> >>>>>> field_dps: "key|data",
> >>>>>> field_dpi: "key|1",
> >>>>>> }
> >>>>>>
> >>>>>> When I execute this query solr returns as expected the payload for
> the
> >>>> key
> >>>>>>
> >>>>>> q=*:*&fl=payload(field_dpi,key)
> >>>>>>
> >>>>>> {
> >>>>>> payload(field_dpi,key): 1
> >>>>>> }
> >>>>>>
> >>>>>> But for the strings there have to be something of different to do,
> >>>> because
> >>>>>> I'm unable receive the payload value back. Executing this query, as
> in
> >>>> the
> >>>>>> short introduction of this post, I receive an error.
> >>>>>>
> >>>>>> ?q=*:*&fl=payload(field_dps,key)
> >>>>>>
> >>>>>> error: {
> >>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> >>>>>> msg: "No payload decoder found for field: colorCode",
> >>>>>> code: 400
> >>>>>> }
> >>>>>>
> >>>>>> Am I doing something wrong? How can I read strings payload data?
> >>>>>>
> >>>>>> Thanks in advance for your time,
> >>>>>> Vincenzo
> >>>>>>
> >>>>>> --
> >>>>>> Vincenzo D'Amore
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> --
> >>>>> Vincenzo D'Amore
> >>>>
> >>>>
> >>>
> >>> --
> >>> Vincenzo D'Amore
> >>
> >>
> >
> > --
> > Vincenzo D'Amore
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Erik Hatcher-4
Yes.   The decoding of a payload based on its schema type is what the payload() function does.   Your Payloader won't currently work well/legibly for fields encoded numerically:

    https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130 <https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130>

I think that code could probably be slightly enhanced to leverage PayloadUtils.getPayloadDecoder(fieldType) and use bytes if the field type doesn't have a better decoder.

        Erik


> On Oct 21, 2019, at 2:55 PM, Eric Pugh <[hidden email]> wrote:
>
> Have you checked out
> https://github.com/o19s/payload-component
>
> On Mon, Oct 21, 2019 at 2:47 PM Erik Hatcher <[hidden email]> wrote:
>
>> How about a single field, with terms like:
>>
>>    store1_USD|125.0 store2_EUR|220.0 store3_GBP|225.0
>>
>> Would that do the trick?
>>
>> And yeah, payload decoding is currently limited to float and int with the
>> built-in payload() function.   We'd need a new way to pull out
>> textual/bytes payloads - like maybe a DocTransformer?
>>
>>        Erik
>>
>>
>>> On Oct 21, 2019, at 9:59 AM, Vincenzo D'Amore <[hidden email]>
>> wrote:
>>>
>>> Hi Erick,
>>>
>>> thanks for getting back to me. We started to use payloads because we have
>>> the classical per-store pricing problem.
>>> Thousands of stores across and different prices.
>>> Then we found the payloads very useful started to use it for many
>> reasons,
>>> like enabling/disabling the product for such store, save the stock
>>> availability, or save the other info like buy/sell price, discount rates,
>>> and so on.
>>> All those information are numbers, but stores can also be in different
>>> countries, I mean would be useful also have the currency and other
>>> attributes related to the store.
>>>
>>> Thinking about an alternative for payloads maybe I could use the dynamic
>>> fields, well, I know it is ugly.
>>>
>>> Consider this hypothetical case where I have two field payload :
>>>
>>> payloadPrice: [
>>> "store1|125.0",
>>> "store2|220.0",
>>> "store3|225.0"
>>> ]
>>>
>>> payloadCurrency: [
>>> "store1|USD",
>>> "store2|EUR",
>>> "store3|GBP"
>>> ]
>>>
>>> with dynamic fields I could have different fields for each document.
>>>
>>> currency_store1_s: "USD"
>>> currency_store2_s: "EUR"
>>> currency_store3_s: "GBP"
>>>
>>> But how many dynamic fields like this can I have? more than thousands?
>>>
>>> Again, I've just started to look at solr-ocrhighlighting github project
>> you
>>> suggested.
>>> Those seems have written their own payload object type where store ocr
>>> highlighting information.
>>> It seems interesting, I'll take a look immediately.
>>>
>>> Thanks again for your time.
>>>
>>> Best regards,
>>> Vincenzo
>>>
>>>
>>> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <[hidden email]>
>>> wrote:
>>>
>>>> This is one of those situations where I know a client did it, but didn’t
>>>> see the code myself.
>>>>
>>>> So I can’t help much.
>>>>
>>>> Perhaps a good question at this point, though, is “why do you want to
>> add
>>>> string payloads anyway”?
>>>>
>>>> This isn’t the client, but it might give you some pointers:
>>>>
>>>>
>>>>
>> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
>>>>
>>>> Best,
>>>> Erick
>>>>
>>>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
>>>> wrote:
>>>>>
>>>>> Hi Erick,
>>>>>
>>>>> It seems I've reached a dead-point, or at least it seems looking at the
>>>>> code, it seems I can't  easily add a custom decoder:
>>>>>
>>>>> Looking at PayloadUtils class there is getPayloadDecoder method invoked
>>>> to
>>>>> return the PayloadDecoder :
>>>>>
>>>>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
>>>>>  PayloadDecoder decoder = null;
>>>>>
>>>>>  String encoder = getPayloadEncoder(fieldType);
>>>>>
>>>>>  if ("integer".equals(encoder)) {
>>>>>    decoder = (BytesRef payload) -> payload == null ? 1 :
>>>>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
>>>>>  }
>>>>>  if ("float".equals(encoder)) {
>>>>>    decoder = (BytesRef payload) -> payload == null ? 1 :
>>>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
>>>>>  }
>>>>>  // encoder could be "identity" at this point, in the case of
>>>>> DelimitedTokenFilterFactory encoder="identity"
>>>>>
>>>>>  // TODO: support pluggable payload decoders?
>>>>>
>>>>>  return decoder;
>>>>> }
>>>>>
>>>>> Any advice to work around this situation?
>>>>>
>>>>>
>>>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <
>> [hidden email]>
>>>>> wrote:
>>>>>
>>>>>> You’d need to write one. Payloads are generally intended to hold
>>>> numerics
>>>>>> you can then use in a function query to factor into the score…
>>>>>>
>>>>>> Best,
>>>>>> Erick
>>>>>>
>>>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
>>>>>> wrote:
>>>>>>>
>>>>>>> Sorry, I just realized that I was wrong in how I'm using the payload
>>>>>>> function.
>>>>>>> Give that the payload function only handles a numeric (integer or
>>>> float)
>>>>>>> payload, could you suggest me an alternative function that handles
>>>>>> strings?
>>>>>>> If not, should I write one?
>>>>>>>
>>>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <
>> [hidden email]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I'm trying to understand what I did wrong with a payload query that
>>>>>>>> returns
>>>>>>>>
>>>>>>>> error: {
>>>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>>>>>> msg: "No payload decoder found for field: colorCode",
>>>>>>>> code: 400
>>>>>>>> }
>>>>>>>>
>>>>>>>> I have reduced my problem in a little sample to show what happens to
>>>> me.
>>>>>>>> Basically I have a document with a couple of payload fields one
>>>>>>>> delimited_payloads_string and one delimited_payloads_integer
>>>>>>>>
>>>>>>>> {
>>>>>>>> field_dps: "key|data",
>>>>>>>> field_dpi: "key|1",
>>>>>>>> }
>>>>>>>>
>>>>>>>> When I execute this query solr returns as expected the payload for
>> the
>>>>>> key
>>>>>>>>
>>>>>>>> q=*:*&fl=payload(field_dpi,key)
>>>>>>>>
>>>>>>>> {
>>>>>>>> payload(field_dpi,key): 1
>>>>>>>> }
>>>>>>>>
>>>>>>>> But for the strings there have to be something of different to do,
>>>>>> because
>>>>>>>> I'm unable receive the payload value back. Executing this query, as
>> in
>>>>>> the
>>>>>>>> short introduction of this post, I receive an error.
>>>>>>>>
>>>>>>>> ?q=*:*&fl=payload(field_dps,key)
>>>>>>>>
>>>>>>>> error: {
>>>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>>>>>> msg: "No payload decoder found for field: colorCode",
>>>>>>>> code: 400
>>>>>>>> }
>>>>>>>>
>>>>>>>> Am I doing something wrong? How can I read strings payload data?
>>>>>>>>
>>>>>>>> Thanks in advance for your time,
>>>>>>>> Vincenzo
>>>>>>>>
>>>>>>>> --
>>>>>>>> Vincenzo D'Amore
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Vincenzo D'Amore
>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Vincenzo D'Amore
>>>>
>>>>
>>>
>>> --
>>> Vincenzo D'Amore
>>
>>

Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Vincenzo D'Amore
Hi all,

thanks for supporting. And many thanks whom have implemented
the integration of the github Solr repository with the intellij IDE.
To configure the environment and run the debugger I spent less than one
hour, (and most of the time I had to wait the compilation).
Solr and you guys really rocks together.

What I've done:

I was looking at the original payload function is defined into
the ValueSourceParser, this function uses a FloatPayloadValueSource to
return the value found.

As said I wrote a new version of payload function that handles strings, I
named it spayload, and basically is able to extract the string value from
the payload.

Given the former example where I have a multivalue field payloadCurrency

payloadCurrency: [
"store1|USD",
"store2|EUR",
"store3|GBP"
]

executing spayload(payloadCurrency,store2) returns "EUR", and so on for the
remaining key/value in the field.

To implement the spayload function, I've added a new ValueSourceParser
instance to the list of defined functions and which returns
a StringPayloadValueSource with the value inside (does the same thing of
former FloatPayloadValueSource).

That's all. As said, always beware of your code when works at first run.
And really there was something wrong, initially I messed up in the
conversion of the payload into String (bytes, offset, etc).
Now it is fixed, or at least it seems to me.
I see this function cannot be used in the sort, very likely the simple
implementation of the StringPayloadValueSource miss something.

As far as I understand I'm scratching the surface of this solution, there
are few things I'm worried about. I have a bunch of questions, please be
patient.
This function returns an empty string "" when does not match any key, or
should return an empty value? not sure about, what's the correct way to
return an empty value?
I wasn't able to find a test unit for the payload function in the tests.
Could you give me few suggestion in order to test properly the
implementation?
In case the spayload is used on a different field type (i.e. the use
spayload on a float payload) the behaviour is not handled. Can this
function check the type of the payload content?
And at last, what do you think, can this simple fix be interesting for the
Solr community, may I try to submit a pull request or add a feature to JIRA?

Best regards,
Vincenzo


On Mon, Oct 21, 2019 at 9:12 PM Erik Hatcher <[hidden email]> wrote:

> Yes.   The decoding of a payload based on its schema type is what the
> payload() function does.   Your Payloader won't currently work well/legibly
> for fields encoded numerically:
>
>
> https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130
> <
> https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130
> >
>
> I think that code could probably be slightly enhanced to leverage
> PayloadUtils.getPayloadDecoder(fieldType) and use bytes if the field type
> doesn't have a better decoder.
>
>         Erik
>
>
> > On Oct 21, 2019, at 2:55 PM, Eric Pugh <[hidden email]>
> wrote:
> >
> > Have you checked out
> > https://github.com/o19s/payload-component
> >
> > On Mon, Oct 21, 2019 at 2:47 PM Erik Hatcher <[hidden email]>
> wrote:
> >
> >> How about a single field, with terms like:
> >>
> >>    store1_USD|125.0 store2_EUR|220.0 store3_GBP|225.0
> >>
> >> Would that do the trick?
> >>
> >> And yeah, payload decoding is currently limited to float and int with
> the
> >> built-in payload() function.   We'd need a new way to pull out
> >> textual/bytes payloads - like maybe a DocTransformer?
> >>
> >>        Erik
> >>
> >>
> >>> On Oct 21, 2019, at 9:59 AM, Vincenzo D'Amore <[hidden email]>
> >> wrote:
> >>>
> >>> Hi Erick,
> >>>
> >>> thanks for getting back to me. We started to use payloads because we
> have
> >>> the classical per-store pricing problem.
> >>> Thousands of stores across and different prices.
> >>> Then we found the payloads very useful started to use it for many
> >> reasons,
> >>> like enabling/disabling the product for such store, save the stock
> >>> availability, or save the other info like buy/sell price, discount
> rates,
> >>> and so on.
> >>> All those information are numbers, but stores can also be in different
> >>> countries, I mean would be useful also have the currency and other
> >>> attributes related to the store.
> >>>
> >>> Thinking about an alternative for payloads maybe I could use the
> dynamic
> >>> fields, well, I know it is ugly.
> >>>
> >>> Consider this hypothetical case where I have two field payload :
> >>>
> >>> payloadPrice: [
> >>> "store1|125.0",
> >>> "store2|220.0",
> >>> "store3|225.0"
> >>> ]
> >>>
> >>> payloadCurrency: [
> >>> "store1|USD",
> >>> "store2|EUR",
> >>> "store3|GBP"
> >>> ]
> >>>
> >>> with dynamic fields I could have different fields for each document.
> >>>
> >>> currency_store1_s: "USD"
> >>> currency_store2_s: "EUR"
> >>> currency_store3_s: "GBP"
> >>>
> >>> But how many dynamic fields like this can I have? more than thousands?
> >>>
> >>> Again, I've just started to look at solr-ocrhighlighting github project
> >> you
> >>> suggested.
> >>> Those seems have written their own payload object type where store ocr
> >>> highlighting information.
> >>> It seems interesting, I'll take a look immediately.
> >>>
> >>> Thanks again for your time.
> >>>
> >>> Best regards,
> >>> Vincenzo
> >>>
> >>>
> >>> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <
> [hidden email]>
> >>> wrote:
> >>>
> >>>> This is one of those situations where I know a client did it, but
> didn’t
> >>>> see the code myself.
> >>>>
> >>>> So I can’t help much.
> >>>>
> >>>> Perhaps a good question at this point, though, is “why do you want to
> >> add
> >>>> string payloads anyway”?
> >>>>
> >>>> This isn’t the client, but it might give you some pointers:
> >>>>
> >>>>
> >>>>
> >>
> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
> >>>>
> >>>> Best,
> >>>> Erick
> >>>>
> >>>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
> >>>> wrote:
> >>>>>
> >>>>> Hi Erick,
> >>>>>
> >>>>> It seems I've reached a dead-point, or at least it seems looking at
> the
> >>>>> code, it seems I can't  easily add a custom decoder:
> >>>>>
> >>>>> Looking at PayloadUtils class there is getPayloadDecoder method
> invoked
> >>>> to
> >>>>> return the PayloadDecoder :
> >>>>>
> >>>>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType) {
> >>>>>  PayloadDecoder decoder = null;
> >>>>>
> >>>>>  String encoder = getPayloadEncoder(fieldType);
> >>>>>
> >>>>>  if ("integer".equals(encoder)) {
> >>>>>    decoder = (BytesRef payload) -> payload == null ? 1 :
> >>>>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
> >>>>>  }
> >>>>>  if ("float".equals(encoder)) {
> >>>>>    decoder = (BytesRef payload) -> payload == null ? 1 :
> >>>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
> >>>>>  }
> >>>>>  // encoder could be "identity" at this point, in the case of
> >>>>> DelimitedTokenFilterFactory encoder="identity"
> >>>>>
> >>>>>  // TODO: support pluggable payload decoders?
> >>>>>
> >>>>>  return decoder;
> >>>>> }
> >>>>>
> >>>>> Any advice to work around this situation?
> >>>>>
> >>>>>
> >>>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <
> >> [hidden email]>
> >>>>> wrote:
> >>>>>
> >>>>>> You’d need to write one. Payloads are generally intended to hold
> >>>> numerics
> >>>>>> you can then use in a function query to factor into the score…
> >>>>>>
> >>>>>> Best,
> >>>>>> Erick
> >>>>>>
> >>>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Sorry, I just realized that I was wrong in how I'm using the
> payload
> >>>>>>> function.
> >>>>>>> Give that the payload function only handles a numeric (integer or
> >>>> float)
> >>>>>>> payload, could you suggest me an alternative function that handles
> >>>>>> strings?
> >>>>>>> If not, should I write one?
> >>>>>>>
> >>>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <
> >> [hidden email]>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi all,
> >>>>>>>>
> >>>>>>>> I'm trying to understand what I did wrong with a payload query
> that
> >>>>>>>> returns
> >>>>>>>>
> >>>>>>>> error: {
> >>>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> >>>>>>>> msg: "No payload decoder found for field: colorCode",
> >>>>>>>> code: 400
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> I have reduced my problem in a little sample to show what happens
> to
> >>>> me.
> >>>>>>>> Basically I have a document with a couple of payload fields one
> >>>>>>>> delimited_payloads_string and one delimited_payloads_integer
> >>>>>>>>
> >>>>>>>> {
> >>>>>>>> field_dps: "key|data",
> >>>>>>>> field_dpi: "key|1",
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> When I execute this query solr returns as expected the payload for
> >> the
> >>>>>> key
> >>>>>>>>
> >>>>>>>> q=*:*&fl=payload(field_dpi,key)
> >>>>>>>>
> >>>>>>>> {
> >>>>>>>> payload(field_dpi,key): 1
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> But for the strings there have to be something of different to do,
> >>>>>> because
> >>>>>>>> I'm unable receive the payload value back. Executing this query,
> as
> >> in
> >>>>>> the
> >>>>>>>> short introduction of this post, I receive an error.
> >>>>>>>>
> >>>>>>>> ?q=*:*&fl=payload(field_dps,key)
> >>>>>>>>
> >>>>>>>> error: {
> >>>>>>>> metadata: [ "error-class", "org.apache.solr.common.SolrException",
> >>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> >>>>>>>> msg: "No payload decoder found for field: colorCode",
> >>>>>>>> code: 400
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> Am I doing something wrong? How can I read strings payload data?
> >>>>>>>>
> >>>>>>>> Thanks in advance for your time,
> >>>>>>>> Vincenzo
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Vincenzo D'Amore
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Vincenzo D'Amore
> >>>>>>
> >>>>>>
> >>>>>
> >>>>> --
> >>>>> Vincenzo D'Amore
> >>>>
> >>>>
> >>>
> >>> --
> >>> Vincenzo D'Amore
> >>
> >>
>
>

--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Vincenzo D'Amore
Hi all,

this evening I had some spare hour to spend in order to put everything
together in a repository.

https://github.com/freedev/solr-payload-string-function-query



On Tue, Oct 22, 2019 at 5:54 PM Vincenzo D'Amore <[hidden email]> wrote:

> Hi all,
>
> thanks for supporting. And many thanks whom have implemented
> the integration of the github Solr repository with the intellij IDE.
> To configure the environment and run the debugger I spent less than one
> hour, (and most of the time I had to wait the compilation).
> Solr and you guys really rocks together.
>
> What I've done:
>
> I was looking at the original payload function is defined into
> the ValueSourceParser, this function uses a FloatPayloadValueSource to
> return the value found.
>
> As said I wrote a new version of payload function that handles strings, I
> named it spayload, and basically is able to extract the string value from
> the payload.
>
> Given the former example where I have a multivalue field payloadCurrency
>
> payloadCurrency: [
> "store1|USD",
> "store2|EUR",
> "store3|GBP"
> ]
>
> executing spayload(payloadCurrency,store2) returns "EUR", and so on for
> the remaining key/value in the field.
>
> To implement the spayload function, I've added a new ValueSourceParser
> instance to the list of defined functions and which returns
> a StringPayloadValueSource with the value inside (does the same thing of
> former FloatPayloadValueSource).
>
> That's all. As said, always beware of your code when works at first run.
> And really there was something wrong, initially I messed up in the
> conversion of the payload into String (bytes, offset, etc).
> Now it is fixed, or at least it seems to me.
> I see this function cannot be used in the sort, very likely the simple
> implementation of the StringPayloadValueSource miss something.
>
> As far as I understand I'm scratching the surface of this solution, there
> are few things I'm worried about. I have a bunch of questions, please be
> patient.
> This function returns an empty string "" when does not match any key, or
> should return an empty value? not sure about, what's the correct way to
> return an empty value?
> I wasn't able to find a test unit for the payload function in the tests.
> Could you give me few suggestion in order to test properly the
> implementation?
> In case the spayload is used on a different field type (i.e. the use
> spayload on a float payload) the behaviour is not handled. Can this
> function check the type of the payload content?
> And at last, what do you think, can this simple fix be interesting for the
> Solr community, may I try to submit a pull request or add a feature to JIRA?
>
> Best regards,
> Vincenzo
>
>
> On Mon, Oct 21, 2019 at 9:12 PM Erik Hatcher <[hidden email]>
> wrote:
>
>> Yes.   The decoding of a payload based on its schema type is what the
>> payload() function does.   Your Payloader won't currently work well/legibly
>> for fields encoded numerically:
>>
>>
>> https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130
>> <
>> https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130
>> >
>>
>> I think that code could probably be slightly enhanced to leverage
>> PayloadUtils.getPayloadDecoder(fieldType) and use bytes if the field type
>> doesn't have a better decoder.
>>
>>         Erik
>>
>>
>> > On Oct 21, 2019, at 2:55 PM, Eric Pugh <[hidden email]>
>> wrote:
>> >
>> > Have you checked out
>> > https://github.com/o19s/payload-component
>> >
>> > On Mon, Oct 21, 2019 at 2:47 PM Erik Hatcher <[hidden email]>
>> wrote:
>> >
>> >> How about a single field, with terms like:
>> >>
>> >>    store1_USD|125.0 store2_EUR|220.0 store3_GBP|225.0
>> >>
>> >> Would that do the trick?
>> >>
>> >> And yeah, payload decoding is currently limited to float and int with
>> the
>> >> built-in payload() function.   We'd need a new way to pull out
>> >> textual/bytes payloads - like maybe a DocTransformer?
>> >>
>> >>        Erik
>> >>
>> >>
>> >>> On Oct 21, 2019, at 9:59 AM, Vincenzo D'Amore <[hidden email]>
>> >> wrote:
>> >>>
>> >>> Hi Erick,
>> >>>
>> >>> thanks for getting back to me. We started to use payloads because we
>> have
>> >>> the classical per-store pricing problem.
>> >>> Thousands of stores across and different prices.
>> >>> Then we found the payloads very useful started to use it for many
>> >> reasons,
>> >>> like enabling/disabling the product for such store, save the stock
>> >>> availability, or save the other info like buy/sell price, discount
>> rates,
>> >>> and so on.
>> >>> All those information are numbers, but stores can also be in different
>> >>> countries, I mean would be useful also have the currency and other
>> >>> attributes related to the store.
>> >>>
>> >>> Thinking about an alternative for payloads maybe I could use the
>> dynamic
>> >>> fields, well, I know it is ugly.
>> >>>
>> >>> Consider this hypothetical case where I have two field payload :
>> >>>
>> >>> payloadPrice: [
>> >>> "store1|125.0",
>> >>> "store2|220.0",
>> >>> "store3|225.0"
>> >>> ]
>> >>>
>> >>> payloadCurrency: [
>> >>> "store1|USD",
>> >>> "store2|EUR",
>> >>> "store3|GBP"
>> >>> ]
>> >>>
>> >>> with dynamic fields I could have different fields for each document.
>> >>>
>> >>> currency_store1_s: "USD"
>> >>> currency_store2_s: "EUR"
>> >>> currency_store3_s: "GBP"
>> >>>
>> >>> But how many dynamic fields like this can I have? more than thousands?
>> >>>
>> >>> Again, I've just started to look at solr-ocrhighlighting github
>> project
>> >> you
>> >>> suggested.
>> >>> Those seems have written their own payload object type where store ocr
>> >>> highlighting information.
>> >>> It seems interesting, I'll take a look immediately.
>> >>>
>> >>> Thanks again for your time.
>> >>>
>> >>> Best regards,
>> >>> Vincenzo
>> >>>
>> >>>
>> >>> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <
>> [hidden email]>
>> >>> wrote:
>> >>>
>> >>>> This is one of those situations where I know a client did it, but
>> didn’t
>> >>>> see the code myself.
>> >>>>
>> >>>> So I can’t help much.
>> >>>>
>> >>>> Perhaps a good question at this point, though, is “why do you want to
>> >> add
>> >>>> string payloads anyway”?
>> >>>>
>> >>>> This isn’t the client, but it might give you some pointers:
>> >>>>
>> >>>>
>> >>>>
>> >>
>> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
>> >>>>
>> >>>> Best,
>> >>>> Erick
>> >>>>
>> >>>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
>> >>>> wrote:
>> >>>>>
>> >>>>> Hi Erick,
>> >>>>>
>> >>>>> It seems I've reached a dead-point, or at least it seems looking at
>> the
>> >>>>> code, it seems I can't  easily add a custom decoder:
>> >>>>>
>> >>>>> Looking at PayloadUtils class there is getPayloadDecoder method
>> invoked
>> >>>> to
>> >>>>> return the PayloadDecoder :
>> >>>>>
>> >>>>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType)
>> {
>> >>>>>  PayloadDecoder decoder = null;
>> >>>>>
>> >>>>>  String encoder = getPayloadEncoder(fieldType);
>> >>>>>
>> >>>>>  if ("integer".equals(encoder)) {
>> >>>>>    decoder = (BytesRef payload) -> payload == null ? 1 :
>> >>>>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
>> >>>>>  }
>> >>>>>  if ("float".equals(encoder)) {
>> >>>>>    decoder = (BytesRef payload) -> payload == null ? 1 :
>> >>>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
>> >>>>>  }
>> >>>>>  // encoder could be "identity" at this point, in the case of
>> >>>>> DelimitedTokenFilterFactory encoder="identity"
>> >>>>>
>> >>>>>  // TODO: support pluggable payload decoders?
>> >>>>>
>> >>>>>  return decoder;
>> >>>>> }
>> >>>>>
>> >>>>> Any advice to work around this situation?
>> >>>>>
>> >>>>>
>> >>>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <
>> >> [hidden email]>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> You’d need to write one. Payloads are generally intended to hold
>> >>>> numerics
>> >>>>>> you can then use in a function query to factor into the score…
>> >>>>>>
>> >>>>>> Best,
>> >>>>>> Erick
>> >>>>>>
>> >>>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]
>> >
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>> Sorry, I just realized that I was wrong in how I'm using the
>> payload
>> >>>>>>> function.
>> >>>>>>> Give that the payload function only handles a numeric (integer or
>> >>>> float)
>> >>>>>>> payload, could you suggest me an alternative function that handles
>> >>>>>> strings?
>> >>>>>>> If not, should I write one?
>> >>>>>>>
>> >>>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <
>> >> [hidden email]>
>> >>>>>>> wrote:
>> >>>>>>>
>> >>>>>>>> Hi all,
>> >>>>>>>>
>> >>>>>>>> I'm trying to understand what I did wrong with a payload query
>> that
>> >>>>>>>> returns
>> >>>>>>>>
>> >>>>>>>> error: {
>> >>>>>>>> metadata: [ "error-class",
>> "org.apache.solr.common.SolrException",
>> >>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>> >>>>>>>> msg: "No payload decoder found for field: colorCode",
>> >>>>>>>> code: 400
>> >>>>>>>> }
>> >>>>>>>>
>> >>>>>>>> I have reduced my problem in a little sample to show what
>> happens to
>> >>>> me.
>> >>>>>>>> Basically I have a document with a couple of payload fields one
>> >>>>>>>> delimited_payloads_string and one delimited_payloads_integer
>> >>>>>>>>
>> >>>>>>>> {
>> >>>>>>>> field_dps: "key|data",
>> >>>>>>>> field_dpi: "key|1",
>> >>>>>>>> }
>> >>>>>>>>
>> >>>>>>>> When I execute this query solr returns as expected the payload
>> for
>> >> the
>> >>>>>> key
>> >>>>>>>>
>> >>>>>>>> q=*:*&fl=payload(field_dpi,key)
>> >>>>>>>>
>> >>>>>>>> {
>> >>>>>>>> payload(field_dpi,key): 1
>> >>>>>>>> }
>> >>>>>>>>
>> >>>>>>>> But for the strings there have to be something of different to
>> do,
>> >>>>>> because
>> >>>>>>>> I'm unable receive the payload value back. Executing this query,
>> as
>> >> in
>> >>>>>> the
>> >>>>>>>> short introduction of this post, I receive an error.
>> >>>>>>>>
>> >>>>>>>> ?q=*:*&fl=payload(field_dps,key)
>> >>>>>>>>
>> >>>>>>>> error: {
>> >>>>>>>> metadata: [ "error-class",
>> "org.apache.solr.common.SolrException",
>> >>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>> >>>>>>>> msg: "No payload decoder found for field: colorCode",
>> >>>>>>>> code: 400
>> >>>>>>>> }
>> >>>>>>>>
>> >>>>>>>> Am I doing something wrong? How can I read strings payload data?
>> >>>>>>>>
>> >>>>>>>> Thanks in advance for your time,
>> >>>>>>>> Vincenzo
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Vincenzo D'Amore
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> Vincenzo D'Amore
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>> --
>> >>>>> Vincenzo D'Amore
>> >>>>
>> >>>>
>> >>>
>> >>> --
>> >>> Vincenzo D'Amore
>> >>
>> >>
>>
>>
>
> --
> Vincenzo D'Amore
>
>

--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Erick Erickson
Bookmarked. Do you intend that this should be incorporated into Solr? If so, please raise a JIRA and link your PR in….

Thanks!
Erick

> On Oct 22, 2019, at 6:56 PM, Vincenzo D'Amore <[hidden email]> wrote:
>
> Hi all,
>
> this evening I had some spare hour to spend in order to put everything
> together in a repository.
>
> https://github.com/freedev/solr-payload-string-function-query
>
>
>
> On Tue, Oct 22, 2019 at 5:54 PM Vincenzo D'Amore <[hidden email]> wrote:
>
>> Hi all,
>>
>> thanks for supporting. And many thanks whom have implemented
>> the integration of the github Solr repository with the intellij IDE.
>> To configure the environment and run the debugger I spent less than one
>> hour, (and most of the time I had to wait the compilation).
>> Solr and you guys really rocks together.
>>
>> What I've done:
>>
>> I was looking at the original payload function is defined into
>> the ValueSourceParser, this function uses a FloatPayloadValueSource to
>> return the value found.
>>
>> As said I wrote a new version of payload function that handles strings, I
>> named it spayload, and basically is able to extract the string value from
>> the payload.
>>
>> Given the former example where I have a multivalue field payloadCurrency
>>
>> payloadCurrency: [
>> "store1|USD",
>> "store2|EUR",
>> "store3|GBP"
>> ]
>>
>> executing spayload(payloadCurrency,store2) returns "EUR", and so on for
>> the remaining key/value in the field.
>>
>> To implement the spayload function, I've added a new ValueSourceParser
>> instance to the list of defined functions and which returns
>> a StringPayloadValueSource with the value inside (does the same thing of
>> former FloatPayloadValueSource).
>>
>> That's all. As said, always beware of your code when works at first run.
>> And really there was something wrong, initially I messed up in the
>> conversion of the payload into String (bytes, offset, etc).
>> Now it is fixed, or at least it seems to me.
>> I see this function cannot be used in the sort, very likely the simple
>> implementation of the StringPayloadValueSource miss something.
>>
>> As far as I understand I'm scratching the surface of this solution, there
>> are few things I'm worried about. I have a bunch of questions, please be
>> patient.
>> This function returns an empty string "" when does not match any key, or
>> should return an empty value? not sure about, what's the correct way to
>> return an empty value?
>> I wasn't able to find a test unit for the payload function in the tests.
>> Could you give me few suggestion in order to test properly the
>> implementation?
>> In case the spayload is used on a different field type (i.e. the use
>> spayload on a float payload) the behaviour is not handled. Can this
>> function check the type of the payload content?
>> And at last, what do you think, can this simple fix be interesting for the
>> Solr community, may I try to submit a pull request or add a feature to JIRA?
>>
>> Best regards,
>> Vincenzo
>>
>>
>> On Mon, Oct 21, 2019 at 9:12 PM Erik Hatcher <[hidden email]>
>> wrote:
>>
>>> Yes.   The decoding of a payload based on its schema type is what the
>>> payload() function does.   Your Payloader won't currently work well/legibly
>>> for fields encoded numerically:
>>>
>>>
>>> https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130
>>> <
>>> https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130
>>>>
>>>
>>> I think that code could probably be slightly enhanced to leverage
>>> PayloadUtils.getPayloadDecoder(fieldType) and use bytes if the field type
>>> doesn't have a better decoder.
>>>
>>>        Erik
>>>
>>>
>>>> On Oct 21, 2019, at 2:55 PM, Eric Pugh <[hidden email]>
>>> wrote:
>>>>
>>>> Have you checked out
>>>> https://github.com/o19s/payload-component
>>>>
>>>> On Mon, Oct 21, 2019 at 2:47 PM Erik Hatcher <[hidden email]>
>>> wrote:
>>>>
>>>>> How about a single field, with terms like:
>>>>>
>>>>>   store1_USD|125.0 store2_EUR|220.0 store3_GBP|225.0
>>>>>
>>>>> Would that do the trick?
>>>>>
>>>>> And yeah, payload decoding is currently limited to float and int with
>>> the
>>>>> built-in payload() function.   We'd need a new way to pull out
>>>>> textual/bytes payloads - like maybe a DocTransformer?
>>>>>
>>>>>       Erik
>>>>>
>>>>>
>>>>>> On Oct 21, 2019, at 9:59 AM, Vincenzo D'Amore <[hidden email]>
>>>>> wrote:
>>>>>>
>>>>>> Hi Erick,
>>>>>>
>>>>>> thanks for getting back to me. We started to use payloads because we
>>> have
>>>>>> the classical per-store pricing problem.
>>>>>> Thousands of stores across and different prices.
>>>>>> Then we found the payloads very useful started to use it for many
>>>>> reasons,
>>>>>> like enabling/disabling the product for such store, save the stock
>>>>>> availability, or save the other info like buy/sell price, discount
>>> rates,
>>>>>> and so on.
>>>>>> All those information are numbers, but stores can also be in different
>>>>>> countries, I mean would be useful also have the currency and other
>>>>>> attributes related to the store.
>>>>>>
>>>>>> Thinking about an alternative for payloads maybe I could use the
>>> dynamic
>>>>>> fields, well, I know it is ugly.
>>>>>>
>>>>>> Consider this hypothetical case where I have two field payload :
>>>>>>
>>>>>> payloadPrice: [
>>>>>> "store1|125.0",
>>>>>> "store2|220.0",
>>>>>> "store3|225.0"
>>>>>> ]
>>>>>>
>>>>>> payloadCurrency: [
>>>>>> "store1|USD",
>>>>>> "store2|EUR",
>>>>>> "store3|GBP"
>>>>>> ]
>>>>>>
>>>>>> with dynamic fields I could have different fields for each document.
>>>>>>
>>>>>> currency_store1_s: "USD"
>>>>>> currency_store2_s: "EUR"
>>>>>> currency_store3_s: "GBP"
>>>>>>
>>>>>> But how many dynamic fields like this can I have? more than thousands?
>>>>>>
>>>>>> Again, I've just started to look at solr-ocrhighlighting github
>>> project
>>>>> you
>>>>>> suggested.
>>>>>> Those seems have written their own payload object type where store ocr
>>>>>> highlighting information.
>>>>>> It seems interesting, I'll take a look immediately.
>>>>>>
>>>>>> Thanks again for your time.
>>>>>>
>>>>>> Best regards,
>>>>>> Vincenzo
>>>>>>
>>>>>>
>>>>>> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <
>>> [hidden email]>
>>>>>> wrote:
>>>>>>
>>>>>>> This is one of those situations where I know a client did it, but
>>> didn’t
>>>>>>> see the code myself.
>>>>>>>
>>>>>>> So I can’t help much.
>>>>>>>
>>>>>>> Perhaps a good question at this point, though, is “why do you want to
>>>>> add
>>>>>>> string payloads anyway”?
>>>>>>>
>>>>>>> This isn’t the client, but it might give you some pointers:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
>>>>>>>
>>>>>>> Best,
>>>>>>> Erick
>>>>>>>
>>>>>>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hi Erick,
>>>>>>>>
>>>>>>>> It seems I've reached a dead-point, or at least it seems looking at
>>> the
>>>>>>>> code, it seems I can't  easily add a custom decoder:
>>>>>>>>
>>>>>>>> Looking at PayloadUtils class there is getPayloadDecoder method
>>> invoked
>>>>>>> to
>>>>>>>> return the PayloadDecoder :
>>>>>>>>
>>>>>>>> public static PayloadDecoder getPayloadDecoder(FieldType fieldType)
>>> {
>>>>>>>> PayloadDecoder decoder = null;
>>>>>>>>
>>>>>>>> String encoder = getPayloadEncoder(fieldType);
>>>>>>>>
>>>>>>>> if ("integer".equals(encoder)) {
>>>>>>>>   decoder = (BytesRef payload) -> payload == null ? 1 :
>>>>>>>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
>>>>>>>> }
>>>>>>>> if ("float".equals(encoder)) {
>>>>>>>>   decoder = (BytesRef payload) -> payload == null ? 1 :
>>>>>>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
>>>>>>>> }
>>>>>>>> // encoder could be "identity" at this point, in the case of
>>>>>>>> DelimitedTokenFilterFactory encoder="identity"
>>>>>>>>
>>>>>>>> // TODO: support pluggable payload decoders?
>>>>>>>>
>>>>>>>> return decoder;
>>>>>>>> }
>>>>>>>>
>>>>>>>> Any advice to work around this situation?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <
>>>>> [hidden email]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> You’d need to write one. Payloads are generally intended to hold
>>>>>>> numerics
>>>>>>>>> you can then use in a function query to factor into the score…
>>>>>>>>>
>>>>>>>>> Best,
>>>>>>>>> Erick
>>>>>>>>>
>>>>>>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <[hidden email]
>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Sorry, I just realized that I was wrong in how I'm using the
>>> payload
>>>>>>>>>> function.
>>>>>>>>>> Give that the payload function only handles a numeric (integer or
>>>>>>> float)
>>>>>>>>>> payload, could you suggest me an alternative function that handles
>>>>>>>>> strings?
>>>>>>>>>> If not, should I write one?
>>>>>>>>>>
>>>>>>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <
>>>>> [hidden email]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> I'm trying to understand what I did wrong with a payload query
>>> that
>>>>>>>>>>> returns
>>>>>>>>>>>
>>>>>>>>>>> error: {
>>>>>>>>>>> metadata: [ "error-class",
>>> "org.apache.solr.common.SolrException",
>>>>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>>>>>>>>> msg: "No payload decoder found for field: colorCode",
>>>>>>>>>>> code: 400
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> I have reduced my problem in a little sample to show what
>>> happens to
>>>>>>> me.
>>>>>>>>>>> Basically I have a document with a couple of payload fields one
>>>>>>>>>>> delimited_payloads_string and one delimited_payloads_integer
>>>>>>>>>>>
>>>>>>>>>>> {
>>>>>>>>>>> field_dps: "key|data",
>>>>>>>>>>> field_dpi: "key|1",
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> When I execute this query solr returns as expected the payload
>>> for
>>>>> the
>>>>>>>>> key
>>>>>>>>>>>
>>>>>>>>>>> q=*:*&fl=payload(field_dpi,key)
>>>>>>>>>>>
>>>>>>>>>>> {
>>>>>>>>>>> payload(field_dpi,key): 1
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> But for the strings there have to be something of different to
>>> do,
>>>>>>>>> because
>>>>>>>>>>> I'm unable receive the payload value back. Executing this query,
>>> as
>>>>> in
>>>>>>>>> the
>>>>>>>>>>> short introduction of this post, I receive an error.
>>>>>>>>>>>
>>>>>>>>>>> ?q=*:*&fl=payload(field_dps,key)
>>>>>>>>>>>
>>>>>>>>>>> error: {
>>>>>>>>>>> metadata: [ "error-class",
>>> "org.apache.solr.common.SolrException",
>>>>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>>>>>>>>>>> msg: "No payload decoder found for field: colorCode",
>>>>>>>>>>> code: 400
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> Am I doing something wrong? How can I read strings payload data?
>>>>>>>>>>>
>>>>>>>>>>> Thanks in advance for your time,
>>>>>>>>>>> Vincenzo
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Vincenzo D'Amore
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Vincenzo D'Amore
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Vincenzo D'Amore
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Vincenzo D'Amore
>>>>>
>>>>>
>>>
>>>
>>
>> --
>> Vincenzo D'Amore
>>
>>
>
> --
> Vincenzo D'Amore

Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Vincenzo D'Amore
Hi Erick, yes, absolutely, it's a great pleasure for me contribute.

On Wed, Oct 23, 2019 at 2:25 PM Erick Erickson <[hidden email]>
wrote:

> Bookmarked. Do you intend that this should be incorporated into Solr? If
> so, please raise a JIRA and link your PR in….
>
> Thanks!
> Erick
>
> > On Oct 22, 2019, at 6:56 PM, Vincenzo D'Amore <[hidden email]>
> wrote:
> >
> > Hi all,
> >
> > this evening I had some spare hour to spend in order to put everything
> > together in a repository.
> >
> > https://github.com/freedev/solr-payload-string-function-query
> >
> >
> >
> > On Tue, Oct 22, 2019 at 5:54 PM Vincenzo D'Amore <[hidden email]>
> wrote:
> >
> >> Hi all,
> >>
> >> thanks for supporting. And many thanks whom have implemented
> >> the integration of the github Solr repository with the intellij IDE.
> >> To configure the environment and run the debugger I spent less than one
> >> hour, (and most of the time I had to wait the compilation).
> >> Solr and you guys really rocks together.
> >>
> >> What I've done:
> >>
> >> I was looking at the original payload function is defined into
> >> the ValueSourceParser, this function uses a FloatPayloadValueSource to
> >> return the value found.
> >>
> >> As said I wrote a new version of payload function that handles strings,
> I
> >> named it spayload, and basically is able to extract the string value
> from
> >> the payload.
> >>
> >> Given the former example where I have a multivalue field payloadCurrency
> >>
> >> payloadCurrency: [
> >> "store1|USD",
> >> "store2|EUR",
> >> "store3|GBP"
> >> ]
> >>
> >> executing spayload(payloadCurrency,store2) returns "EUR", and so on for
> >> the remaining key/value in the field.
> >>
> >> To implement the spayload function, I've added a new ValueSourceParser
> >> instance to the list of defined functions and which returns
> >> a StringPayloadValueSource with the value inside (does the same thing of
> >> former FloatPayloadValueSource).
> >>
> >> That's all. As said, always beware of your code when works at first run.
> >> And really there was something wrong, initially I messed up in the
> >> conversion of the payload into String (bytes, offset, etc).
> >> Now it is fixed, or at least it seems to me.
> >> I see this function cannot be used in the sort, very likely the simple
> >> implementation of the StringPayloadValueSource miss something.
> >>
> >> As far as I understand I'm scratching the surface of this solution,
> there
> >> are few things I'm worried about. I have a bunch of questions, please be
> >> patient.
> >> This function returns an empty string "" when does not match any key, or
> >> should return an empty value? not sure about, what's the correct way to
> >> return an empty value?
> >> I wasn't able to find a test unit for the payload function in the tests.
> >> Could you give me few suggestion in order to test properly the
> >> implementation?
> >> In case the spayload is used on a different field type (i.e. the use
> >> spayload on a float payload) the behaviour is not handled. Can this
> >> function check the type of the payload content?
> >> And at last, what do you think, can this simple fix be interesting for
> the
> >> Solr community, may I try to submit a pull request or add a feature to
> JIRA?
> >>
> >> Best regards,
> >> Vincenzo
> >>
> >>
> >> On Mon, Oct 21, 2019 at 9:12 PM Erik Hatcher <[hidden email]>
> >> wrote:
> >>
> >>> Yes.   The decoding of a payload based on its schema type is what the
> >>> payload() function does.   Your Payloader won't currently work
> well/legibly
> >>> for fields encoded numerically:
> >>>
> >>>
> >>>
> https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130
> >>> <
> >>>
> https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130
> >>>>
> >>>
> >>> I think that code could probably be slightly enhanced to leverage
> >>> PayloadUtils.getPayloadDecoder(fieldType) and use bytes if the field
> type
> >>> doesn't have a better decoder.
> >>>
> >>>        Erik
> >>>
> >>>
> >>>> On Oct 21, 2019, at 2:55 PM, Eric Pugh <
> [hidden email]>
> >>> wrote:
> >>>>
> >>>> Have you checked out
> >>>> https://github.com/o19s/payload-component
> >>>>
> >>>> On Mon, Oct 21, 2019 at 2:47 PM Erik Hatcher <[hidden email]>
> >>> wrote:
> >>>>
> >>>>> How about a single field, with terms like:
> >>>>>
> >>>>>   store1_USD|125.0 store2_EUR|220.0 store3_GBP|225.0
> >>>>>
> >>>>> Would that do the trick?
> >>>>>
> >>>>> And yeah, payload decoding is currently limited to float and int with
> >>> the
> >>>>> built-in payload() function.   We'd need a new way to pull out
> >>>>> textual/bytes payloads - like maybe a DocTransformer?
> >>>>>
> >>>>>       Erik
> >>>>>
> >>>>>
> >>>>>> On Oct 21, 2019, at 9:59 AM, Vincenzo D'Amore <[hidden email]>
> >>>>> wrote:
> >>>>>>
> >>>>>> Hi Erick,
> >>>>>>
> >>>>>> thanks for getting back to me. We started to use payloads because we
> >>> have
> >>>>>> the classical per-store pricing problem.
> >>>>>> Thousands of stores across and different prices.
> >>>>>> Then we found the payloads very useful started to use it for many
> >>>>> reasons,
> >>>>>> like enabling/disabling the product for such store, save the stock
> >>>>>> availability, or save the other info like buy/sell price, discount
> >>> rates,
> >>>>>> and so on.
> >>>>>> All those information are numbers, but stores can also be in
> different
> >>>>>> countries, I mean would be useful also have the currency and other
> >>>>>> attributes related to the store.
> >>>>>>
> >>>>>> Thinking about an alternative for payloads maybe I could use the
> >>> dynamic
> >>>>>> fields, well, I know it is ugly.
> >>>>>>
> >>>>>> Consider this hypothetical case where I have two field payload :
> >>>>>>
> >>>>>> payloadPrice: [
> >>>>>> "store1|125.0",
> >>>>>> "store2|220.0",
> >>>>>> "store3|225.0"
> >>>>>> ]
> >>>>>>
> >>>>>> payloadCurrency: [
> >>>>>> "store1|USD",
> >>>>>> "store2|EUR",
> >>>>>> "store3|GBP"
> >>>>>> ]
> >>>>>>
> >>>>>> with dynamic fields I could have different fields for each document.
> >>>>>>
> >>>>>> currency_store1_s: "USD"
> >>>>>> currency_store2_s: "EUR"
> >>>>>> currency_store3_s: "GBP"
> >>>>>>
> >>>>>> But how many dynamic fields like this can I have? more than
> thousands?
> >>>>>>
> >>>>>> Again, I've just started to look at solr-ocrhighlighting github
> >>> project
> >>>>> you
> >>>>>> suggested.
> >>>>>> Those seems have written their own payload object type where store
> ocr
> >>>>>> highlighting information.
> >>>>>> It seems interesting, I'll take a look immediately.
> >>>>>>
> >>>>>> Thanks again for your time.
> >>>>>>
> >>>>>> Best regards,
> >>>>>> Vincenzo
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <
> >>> [hidden email]>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> This is one of those situations where I know a client did it, but
> >>> didn’t
> >>>>>>> see the code myself.
> >>>>>>>
> >>>>>>> So I can’t help much.
> >>>>>>>
> >>>>>>> Perhaps a good question at this point, though, is “why do you want
> to
> >>>>> add
> >>>>>>> string payloads anyway”?
> >>>>>>>
> >>>>>>> This isn’t the client, but it might give you some pointers:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>
> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
> >>>>>>>
> >>>>>>> Best,
> >>>>>>> Erick
> >>>>>>>
> >>>>>>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <[hidden email]
> >
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Hi Erick,
> >>>>>>>>
> >>>>>>>> It seems I've reached a dead-point, or at least it seems looking
> at
> >>> the
> >>>>>>>> code, it seems I can't  easily add a custom decoder:
> >>>>>>>>
> >>>>>>>> Looking at PayloadUtils class there is getPayloadDecoder method
> >>> invoked
> >>>>>>> to
> >>>>>>>> return the PayloadDecoder :
> >>>>>>>>
> >>>>>>>> public static PayloadDecoder getPayloadDecoder(FieldType
> fieldType)
> >>> {
> >>>>>>>> PayloadDecoder decoder = null;
> >>>>>>>>
> >>>>>>>> String encoder = getPayloadEncoder(fieldType);
> >>>>>>>>
> >>>>>>>> if ("integer".equals(encoder)) {
> >>>>>>>>   decoder = (BytesRef payload) -> payload == null ? 1 :
> >>>>>>>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
> >>>>>>>> }
> >>>>>>>> if ("float".equals(encoder)) {
> >>>>>>>>   decoder = (BytesRef payload) -> payload == null ? 1 :
> >>>>>>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
> >>>>>>>> }
> >>>>>>>> // encoder could be "identity" at this point, in the case of
> >>>>>>>> DelimitedTokenFilterFactory encoder="identity"
> >>>>>>>>
> >>>>>>>> // TODO: support pluggable payload decoders?
> >>>>>>>>
> >>>>>>>> return decoder;
> >>>>>>>> }
> >>>>>>>>
> >>>>>>>> Any advice to work around this situation?
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <
> >>>>> [hidden email]>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> You’d need to write one. Payloads are generally intended to hold
> >>>>>>> numerics
> >>>>>>>>> you can then use in a function query to factor into the score…
> >>>>>>>>>
> >>>>>>>>> Best,
> >>>>>>>>> Erick
> >>>>>>>>>
> >>>>>>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <
> [hidden email]
> >>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Sorry, I just realized that I was wrong in how I'm using the
> >>> payload
> >>>>>>>>>> function.
> >>>>>>>>>> Give that the payload function only handles a numeric (integer
> or
> >>>>>>> float)
> >>>>>>>>>> payload, could you suggest me an alternative function that
> handles
> >>>>>>>>> strings?
> >>>>>>>>>> If not, should I write one?
> >>>>>>>>>>
> >>>>>>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <
> >>>>> [hidden email]>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi all,
> >>>>>>>>>>>
> >>>>>>>>>>> I'm trying to understand what I did wrong with a payload query
> >>> that
> >>>>>>>>>>> returns
> >>>>>>>>>>>
> >>>>>>>>>>> error: {
> >>>>>>>>>>> metadata: [ "error-class",
> >>> "org.apache.solr.common.SolrException",
> >>>>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> >>>>>>>>>>> msg: "No payload decoder found for field: colorCode",
> >>>>>>>>>>> code: 400
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>> I have reduced my problem in a little sample to show what
> >>> happens to
> >>>>>>> me.
> >>>>>>>>>>> Basically I have a document with a couple of payload fields one
> >>>>>>>>>>> delimited_payloads_string and one delimited_payloads_integer
> >>>>>>>>>>>
> >>>>>>>>>>> {
> >>>>>>>>>>> field_dps: "key|data",
> >>>>>>>>>>> field_dpi: "key|1",
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>> When I execute this query solr returns as expected the payload
> >>> for
> >>>>> the
> >>>>>>>>> key
> >>>>>>>>>>>
> >>>>>>>>>>> q=*:*&fl=payload(field_dpi,key)
> >>>>>>>>>>>
> >>>>>>>>>>> {
> >>>>>>>>>>> payload(field_dpi,key): 1
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>> But for the strings there have to be something of different to
> >>> do,
> >>>>>>>>> because
> >>>>>>>>>>> I'm unable receive the payload value back. Executing this
> query,
> >>> as
> >>>>> in
> >>>>>>>>> the
> >>>>>>>>>>> short introduction of this post, I receive an error.
> >>>>>>>>>>>
> >>>>>>>>>>> ?q=*:*&fl=payload(field_dps,key)
> >>>>>>>>>>>
> >>>>>>>>>>> error: {
> >>>>>>>>>>> metadata: [ "error-class",
> >>> "org.apache.solr.common.SolrException",
> >>>>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
> >>>>>>>>>>> msg: "No payload decoder found for field: colorCode",
> >>>>>>>>>>> code: 400
> >>>>>>>>>>> }
> >>>>>>>>>>>
> >>>>>>>>>>> Am I doing something wrong? How can I read strings payload
> data?
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks in advance for your time,
> >>>>>>>>>>> Vincenzo
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Vincenzo D'Amore
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Vincenzo D'Amore
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Vincenzo D'Amore
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Vincenzo D'Amore
> >>>>>
> >>>>>
> >>>
> >>>
> >>
> >> --
> >> Vincenzo D'Amore
> >>
> >>
> >
> > --
> > Vincenzo D'Amore
>
>

--
Vincenzo D'Amore
Reply | Threaded
Open this post in threaded view
|

Re: Solr Paryload example

Vincenzo D'Amore
Hi all,

just to let you know that we started using spayload function in our quality
environment for testing.
Within a couple of weeks the feature will be deployed in production.

Best regards,
Vincenzo

On Wed, Oct 23, 2019 at 4:31 PM Vincenzo D'Amore <[hidden email]> wrote:

> Hi Erick, yes, absolutely, it's a great pleasure for me contribute.
>
> On Wed, Oct 23, 2019 at 2:25 PM Erick Erickson <[hidden email]>
> wrote:
>
>> Bookmarked. Do you intend that this should be incorporated into Solr? If
>> so, please raise a JIRA and link your PR in….
>>
>> Thanks!
>> Erick
>>
>> > On Oct 22, 2019, at 6:56 PM, Vincenzo D'Amore <[hidden email]>
>> wrote:
>> >
>> > Hi all,
>> >
>> > this evening I had some spare hour to spend in order to put everything
>> > together in a repository.
>> >
>> > https://github.com/freedev/solr-payload-string-function-query
>> >
>> >
>> >
>> > On Tue, Oct 22, 2019 at 5:54 PM Vincenzo D'Amore <[hidden email]>
>> wrote:
>> >
>> >> Hi all,
>> >>
>> >> thanks for supporting. And many thanks whom have implemented
>> >> the integration of the github Solr repository with the intellij IDE.
>> >> To configure the environment and run the debugger I spent less than one
>> >> hour, (and most of the time I had to wait the compilation).
>> >> Solr and you guys really rocks together.
>> >>
>> >> What I've done:
>> >>
>> >> I was looking at the original payload function is defined into
>> >> the ValueSourceParser, this function uses a FloatPayloadValueSource to
>> >> return the value found.
>> >>
>> >> As said I wrote a new version of payload function that handles
>> strings, I
>> >> named it spayload, and basically is able to extract the string value
>> from
>> >> the payload.
>> >>
>> >> Given the former example where I have a multivalue field
>> payloadCurrency
>> >>
>> >> payloadCurrency: [
>> >> "store1|USD",
>> >> "store2|EUR",
>> >> "store3|GBP"
>> >> ]
>> >>
>> >> executing spayload(payloadCurrency,store2) returns "EUR", and so on for
>> >> the remaining key/value in the field.
>> >>
>> >> To implement the spayload function, I've added a new ValueSourceParser
>> >> instance to the list of defined functions and which returns
>> >> a StringPayloadValueSource with the value inside (does the same thing
>> of
>> >> former FloatPayloadValueSource).
>> >>
>> >> That's all. As said, always beware of your code when works at first
>> run.
>> >> And really there was something wrong, initially I messed up in the
>> >> conversion of the payload into String (bytes, offset, etc).
>> >> Now it is fixed, or at least it seems to me.
>> >> I see this function cannot be used in the sort, very likely the simple
>> >> implementation of the StringPayloadValueSource miss something.
>> >>
>> >> As far as I understand I'm scratching the surface of this solution,
>> there
>> >> are few things I'm worried about. I have a bunch of questions, please
>> be
>> >> patient.
>> >> This function returns an empty string "" when does not match any key,
>> or
>> >> should return an empty value? not sure about, what's the correct way to
>> >> return an empty value?
>> >> I wasn't able to find a test unit for the payload function in the
>> tests.
>> >> Could you give me few suggestion in order to test properly the
>> >> implementation?
>> >> In case the spayload is used on a different field type (i.e. the use
>> >> spayload on a float payload) the behaviour is not handled. Can this
>> >> function check the type of the payload content?
>> >> And at last, what do you think, can this simple fix be interesting for
>> the
>> >> Solr community, may I try to submit a pull request or add a feature to
>> JIRA?
>> >>
>> >> Best regards,
>> >> Vincenzo
>> >>
>> >>
>> >> On Mon, Oct 21, 2019 at 9:12 PM Erik Hatcher <[hidden email]>
>> >> wrote:
>> >>
>> >>> Yes.   The decoding of a payload based on its schema type is what the
>> >>> payload() function does.   Your Payloader won't currently work
>> well/legibly
>> >>> for fields encoded numerically:
>> >>>
>> >>>
>> >>>
>> https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130
>> >>> <
>> >>>
>> https://github.com/o19s/payload-component/blob/master/src/main/java/com/o19s/payloads/Payloader.java#L130
>> >>>>
>> >>>
>> >>> I think that code could probably be slightly enhanced to leverage
>> >>> PayloadUtils.getPayloadDecoder(fieldType) and use bytes if the field
>> type
>> >>> doesn't have a better decoder.
>> >>>
>> >>>        Erik
>> >>>
>> >>>
>> >>>> On Oct 21, 2019, at 2:55 PM, Eric Pugh <
>> [hidden email]>
>> >>> wrote:
>> >>>>
>> >>>> Have you checked out
>> >>>> https://github.com/o19s/payload-component
>> >>>>
>> >>>> On Mon, Oct 21, 2019 at 2:47 PM Erik Hatcher <[hidden email]
>> >
>> >>> wrote:
>> >>>>
>> >>>>> How about a single field, with terms like:
>> >>>>>
>> >>>>>   store1_USD|125.0 store2_EUR|220.0 store3_GBP|225.0
>> >>>>>
>> >>>>> Would that do the trick?
>> >>>>>
>> >>>>> And yeah, payload decoding is currently limited to float and int
>> with
>> >>> the
>> >>>>> built-in payload() function.   We'd need a new way to pull out
>> >>>>> textual/bytes payloads - like maybe a DocTransformer?
>> >>>>>
>> >>>>>       Erik
>> >>>>>
>> >>>>>
>> >>>>>> On Oct 21, 2019, at 9:59 AM, Vincenzo D'Amore <[hidden email]>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>> Hi Erick,
>> >>>>>>
>> >>>>>> thanks for getting back to me. We started to use payloads because
>> we
>> >>> have
>> >>>>>> the classical per-store pricing problem.
>> >>>>>> Thousands of stores across and different prices.
>> >>>>>> Then we found the payloads very useful started to use it for many
>> >>>>> reasons,
>> >>>>>> like enabling/disabling the product for such store, save the stock
>> >>>>>> availability, or save the other info like buy/sell price, discount
>> >>> rates,
>> >>>>>> and so on.
>> >>>>>> All those information are numbers, but stores can also be in
>> different
>> >>>>>> countries, I mean would be useful also have the currency and other
>> >>>>>> attributes related to the store.
>> >>>>>>
>> >>>>>> Thinking about an alternative for payloads maybe I could use the
>> >>> dynamic
>> >>>>>> fields, well, I know it is ugly.
>> >>>>>>
>> >>>>>> Consider this hypothetical case where I have two field payload :
>> >>>>>>
>> >>>>>> payloadPrice: [
>> >>>>>> "store1|125.0",
>> >>>>>> "store2|220.0",
>> >>>>>> "store3|225.0"
>> >>>>>> ]
>> >>>>>>
>> >>>>>> payloadCurrency: [
>> >>>>>> "store1|USD",
>> >>>>>> "store2|EUR",
>> >>>>>> "store3|GBP"
>> >>>>>> ]
>> >>>>>>
>> >>>>>> with dynamic fields I could have different fields for each
>> document.
>> >>>>>>
>> >>>>>> currency_store1_s: "USD"
>> >>>>>> currency_store2_s: "EUR"
>> >>>>>> currency_store3_s: "GBP"
>> >>>>>>
>> >>>>>> But how many dynamic fields like this can I have? more than
>> thousands?
>> >>>>>>
>> >>>>>> Again, I've just started to look at solr-ocrhighlighting github
>> >>> project
>> >>>>> you
>> >>>>>> suggested.
>> >>>>>> Those seems have written their own payload object type where store
>> ocr
>> >>>>>> highlighting information.
>> >>>>>> It seems interesting, I'll take a look immediately.
>> >>>>>>
>> >>>>>> Thanks again for your time.
>> >>>>>>
>> >>>>>> Best regards,
>> >>>>>> Vincenzo
>> >>>>>>
>> >>>>>>
>> >>>>>> On Mon, Oct 21, 2019 at 2:55 PM Erick Erickson <
>> >>> [hidden email]>
>> >>>>>> wrote:
>> >>>>>>
>> >>>>>>> This is one of those situations where I know a client did it, but
>> >>> didn’t
>> >>>>>>> see the code myself.
>> >>>>>>>
>> >>>>>>> So I can’t help much.
>> >>>>>>>
>> >>>>>>> Perhaps a good question at this point, though, is “why do you
>> want to
>> >>>>> add
>> >>>>>>> string payloads anyway”?
>> >>>>>>>
>> >>>>>>> This isn’t the client, but it might give you some pointers:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>
>> https://github.com/dbmdz/solr-ocrpayload-plugin/blob/master/src/main/java/de/digitalcollections/solr/plugin/components/ocrhighlighting/OcrHighlighting.java
>> >>>>>>>
>> >>>>>>> Best,
>> >>>>>>> Erick
>> >>>>>>>
>> >>>>>>>> On Oct 21, 2019, at 6:37 AM, Vincenzo D'Amore <
>> [hidden email]>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Hi Erick,
>> >>>>>>>>
>> >>>>>>>> It seems I've reached a dead-point, or at least it seems looking
>> at
>> >>> the
>> >>>>>>>> code, it seems I can't  easily add a custom decoder:
>> >>>>>>>>
>> >>>>>>>> Looking at PayloadUtils class there is getPayloadDecoder method
>> >>> invoked
>> >>>>>>> to
>> >>>>>>>> return the PayloadDecoder :
>> >>>>>>>>
>> >>>>>>>> public static PayloadDecoder getPayloadDecoder(FieldType
>> fieldType)
>> >>> {
>> >>>>>>>> PayloadDecoder decoder = null;
>> >>>>>>>>
>> >>>>>>>> String encoder = getPayloadEncoder(fieldType);
>> >>>>>>>>
>> >>>>>>>> if ("integer".equals(encoder)) {
>> >>>>>>>>   decoder = (BytesRef payload) -> payload == null ? 1 :
>> >>>>>>>> PayloadHelper.decodeInt(payload.bytes, payload.offset);
>> >>>>>>>> }
>> >>>>>>>> if ("float".equals(encoder)) {
>> >>>>>>>>   decoder = (BytesRef payload) -> payload == null ? 1 :
>> >>>>>>>> PayloadHelper.decodeFloat(payload.bytes, payload.offset);
>> >>>>>>>> }
>> >>>>>>>> // encoder could be "identity" at this point, in the case of
>> >>>>>>>> DelimitedTokenFilterFactory encoder="identity"
>> >>>>>>>>
>> >>>>>>>> // TODO: support pluggable payload decoders?
>> >>>>>>>>
>> >>>>>>>> return decoder;
>> >>>>>>>> }
>> >>>>>>>>
>> >>>>>>>> Any advice to work around this situation?
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On Mon, Oct 21, 2019 at 1:51 AM Erick Erickson <
>> >>>>> [hidden email]>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> You’d need to write one. Payloads are generally intended to hold
>> >>>>>>> numerics
>> >>>>>>>>> you can then use in a function query to factor into the score…
>> >>>>>>>>>
>> >>>>>>>>> Best,
>> >>>>>>>>> Erick
>> >>>>>>>>>
>> >>>>>>>>>> On Oct 20, 2019, at 4:57 PM, Vincenzo D'Amore <
>> [hidden email]
>> >>>>
>> >>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Sorry, I just realized that I was wrong in how I'm using the
>> >>> payload
>> >>>>>>>>>> function.
>> >>>>>>>>>> Give that the payload function only handles a numeric (integer
>> or
>> >>>>>>> float)
>> >>>>>>>>>> payload, could you suggest me an alternative function that
>> handles
>> >>>>>>>>> strings?
>> >>>>>>>>>> If not, should I write one?
>> >>>>>>>>>>
>> >>>>>>>>>> On Sun, Oct 20, 2019 at 10:43 PM Vincenzo D'Amore <
>> >>>>> [hidden email]>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Hi all,
>> >>>>>>>>>>>
>> >>>>>>>>>>> I'm trying to understand what I did wrong with a payload query
>> >>> that
>> >>>>>>>>>>> returns
>> >>>>>>>>>>>
>> >>>>>>>>>>> error: {
>> >>>>>>>>>>> metadata: [ "error-class",
>> >>> "org.apache.solr.common.SolrException",
>> >>>>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>> >>>>>>>>>>> msg: "No payload decoder found for field: colorCode",
>> >>>>>>>>>>> code: 400
>> >>>>>>>>>>> }
>> >>>>>>>>>>>
>> >>>>>>>>>>> I have reduced my problem in a little sample to show what
>> >>> happens to
>> >>>>>>> me.
>> >>>>>>>>>>> Basically I have a document with a couple of payload fields
>> one
>> >>>>>>>>>>> delimited_payloads_string and one delimited_payloads_integer
>> >>>>>>>>>>>
>> >>>>>>>>>>> {
>> >>>>>>>>>>> field_dps: "key|data",
>> >>>>>>>>>>> field_dpi: "key|1",
>> >>>>>>>>>>> }
>> >>>>>>>>>>>
>> >>>>>>>>>>> When I execute this query solr returns as expected the payload
>> >>> for
>> >>>>> the
>> >>>>>>>>> key
>> >>>>>>>>>>>
>> >>>>>>>>>>> q=*:*&fl=payload(field_dpi,key)
>> >>>>>>>>>>>
>> >>>>>>>>>>> {
>> >>>>>>>>>>> payload(field_dpi,key): 1
>> >>>>>>>>>>> }
>> >>>>>>>>>>>
>> >>>>>>>>>>> But for the strings there have to be something of different to
>> >>> do,
>> >>>>>>>>> because
>> >>>>>>>>>>> I'm unable receive the payload value back. Executing this
>> query,
>> >>> as
>> >>>>> in
>> >>>>>>>>> the
>> >>>>>>>>>>> short introduction of this post, I receive an error.
>> >>>>>>>>>>>
>> >>>>>>>>>>> ?q=*:*&fl=payload(field_dps,key)
>> >>>>>>>>>>>
>> >>>>>>>>>>> error: {
>> >>>>>>>>>>> metadata: [ "error-class",
>> >>> "org.apache.solr.common.SolrException",
>> >>>>>>>>>>> "root-error-class", "org.apache.solr.common.SolrException" ],
>> >>>>>>>>>>> msg: "No payload decoder found for field: colorCode",
>> >>>>>>>>>>> code: 400
>> >>>>>>>>>>> }
>> >>>>>>>>>>>
>> >>>>>>>>>>> Am I doing something wrong? How can I read strings payload
>> data?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks in advance for your time,
>> >>>>>>>>>>> Vincenzo
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> Vincenzo D'Amore
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> Vincenzo D'Amore
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Vincenzo D'Amore
>> >>>>>>>
>> >>>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Vincenzo D'Amore
>> >>>>>
>> >>>>>
>> >>>
>> >>>
>> >>
>> >> --
>> >> Vincenzo D'Amore
>> >>
>> >>
>> >
>> > --
>> > Vincenzo D'Amore
>>
>>
>
> --
> Vincenzo D'Amore
>
>

--
Vincenzo D'Amore
12