Re:Facing issue ith MinMaxNormalizer

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re:Facing issue ith MinMaxNormalizer

Christine Poerschke (BLOOMBERG/ LONDON)
Hello Kamal Kishore,

Thanks for including the Solr version alongside your question! What you describe sounds like the https://issues.apache.org/jira/browse/SOLR-11163 issue which is fixed in 7.0.0 but not 6.6.2 release. The fix is a simple two line change to MinMaxNormalizer and perhaps one workaround could be for you to build a custom MinMaxNormalizer locally for use with your 6.6.2 setup.

Hope that helps.

Regards,

Christine

From: [hidden email] At: 06/12/19 12:41:51To:  [hidden email]
Subject: Facing issue ith MinMaxNormalizer

Hi All,

Appreciate if someone can help.
I am using LTR with MinMaxNormalizer in solr 6.6.2.

Model.json

 "class": "org.apache.solr.ltr.model.MultipleAdditiveTreesModel",
    "name": "XGBOOST-BBB-LTR-Model",
    "store":"BBB-Feature-Model",
    "features": [
        {
            "name": "TFIDF",
         "norm":{
                 "class" : "org.apache.solr.ltr.norm.MinMaxNormalizer",
                    "params" : { "min":"0.0", "max":"1.0"}
         }
        },
        {
            "name": "p_ratings_f",
         "norm":{
                 "class" : "org.apache.solr.ltr.norm.MinMaxNormalizer",
                    "params" : { "min":"1.0", "max":"5.0"}
         }
        },
        {
            "name": "p_instore_trans_cnt_f",
             "norm":{
                 "class" : "org.apache.solr.ltr.norm.MinMaxNormalizer",
                    "params" : { "min":"1.0", "max":"209561.0" }
         }
        },
        {
            "name": "p_reviews_f",
             "norm":{
                 "class" : "org.apache.solr.ltr.norm.MinMaxNormalizer",
                    "params" : { "min":"0.0", "max":"58375.0"}
         }
        }

 The model got uploaded successfully, but when I reloaded the collection, it failed and below error is observed:

Caused by: java.lang.RuntimeException: Error invoking setter setMin on class : org.apache.solr.ltr.norm.MinMaxNormalizer
        at org.apache.solr.util.SolrPluginUtils.invokeSetters(SolrPluginUtils.java:1084)
        at org.apache.solr.ltr.norm.Normalizer.getInstance(Normalizer.java:49)
        at org.apache.solr.ltr.store.rest.ManagedModelStore.fromNormalizerMap(ManagedModelStore.java:293)
        at org.apache.solr.ltr.store.rest.ManagedModelStore.createNormalizerFromFeatureMap(ManagedModelStore.java:276)
        at org.apache.solr.ltr.store.rest.ManagedModelStore.fromLTRScoringModelMap(ManagedModelStore.java:230)
        at org.apache.solr.ltr.store.rest.ManagedModelStore.addModelFromMap(ManagedModelStore.java:133)
        at org.apache.solr.ltr.store.rest.ManagedModelStore.loadStoredModels(ManagedModelStore.java:126)
        at org.apache.solr.ltr.search.LTRQParserPlugin.onManagedResourceInitialized(LTRQParserPlugin.java:133)
        at org.apache.solr.rest.ManagedResource.notifyObserversDuringInit(ManagedResource.java:115)
        at org.apache.solr.rest.ManagedResource.loadManagedDataAndNotify(ManagedResource.java:91)
        at org.apache.solr.rest.RestManager.createManagedResource(RestManager.java:694)
        ... 41 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.solr.util.SolrPluginUtils.invokeSetters(SolrPluginUtils.java:1082)
        ... 51 more
Caused by: java.lang.NumberFormatException: For input string: ""0.0""
        at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)
        at sun.misc.FloatingDecimal.parseFloat(FloatingDecimal.java:122)
        at java.lang.Float.parseFloat(Float.java:451)
        at org.apache.solr.ltr.norm.MinMaxNormalizer.setMin(MinMaxNormalizer.java:58)

I tried uploading the model without double quotes in param value for min and max, it also got failed with below error.

"java.lang.IllegalArgumentException: argument type mismatch\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.apache.solr.util.SolrPluginUtils.invokeSetters(SolrPluginUtils.java:1082)\n\tat org.apache.solr.ltr.norm.Normalizer.getInstance(Normalizer.java:49)\n\tat org.apache.solr.ltr.store.rest.ManagedModelStore.fromNormalizerMap(ManagedModelStore.java:293)\n\tat org.apache.solr.ltr.store.rest.ManagedModelStore.createNormalizerFromFeatureMap(ManagedModelStore.java:276)\n\tat org.apache.solr.ltr.store.rest.ManagedModelStore.fromLTRScoringModelMap(ManagedModelStore.java:230)\n\tat org.apache.solr.ltr.store.rest.ManagedModelStore.addModelFromMap(ManagedModelStore.java:133)\n\tat org.apache.solr.ltr.store.rest.ManagedModelStore.applyUpdatesToManagedData(ManagedModelStore.java:162)\n\tat

Also, I can see on the collection that there is escape in double quotes in model on model api (s/s below).


Regards
Kamal Kishore

Reply | Threaded
Open this post in threaded view
|

Re: Facing issue ith MinMaxNormalizer

kamal11
Hi Christine,

Thanks for the reply.

I had identified the same issue. So, applied the below fix in the custom
jar.
 this.max = Float.parseFloat(max.replace("\"", ""));
this.min = Float.parseFloat(min.replace("\"", ""));

And, it worked. Thanks for confirming that this is an issue in solr 6.6.2

Regards
Kamal Kishore.

On Fri, Jun 14, 2019 at 2:17 PM Christine Poerschke (BLOOMBERG/ LONDON) <
[hidden email]> wrote:

> Hello Kamal Kishore,
>
> Thanks for including the Solr version alongside your question! What you
> describe sounds like the https://issues.apache.org/jira/browse/SOLR-11163
> issue which is fixed in 7.0.0 but not 6.6.2 release. The fix is a simple
> two line change to MinMaxNormalizer and perhaps one workaround could be for
> you to build a custom MinMaxNormalizer locally for use with your 6.6.2
> setup.
>
> Hope that helps.
>
> Regards,
>
> Christine
>
> From: [hidden email] At: 06/12/19 12:41:51To:
> [hidden email]
> Subject: Facing issue ith MinMaxNormalizer
>
> Hi All,
>
> Appreciate if someone can help.
> I am using LTR with MinMaxNormalizer in solr 6.6.2.
>
> Model.json
>
>  "class": "org.apache.solr.ltr.model.MultipleAdditiveTreesModel",
>     "name": "XGBOOST-BBB-LTR-Model",
>     "store":"BBB-Feature-Model",
>     "features": [
>         {
>             "name": "TFIDF",
>          "norm":{
>                  "class" : "org.apache.solr.ltr.norm.MinMaxNormalizer",
>                     "params" : { "min":"0.0", "max":"1.0"}
>          }
>         },
>         {
>             "name": "p_ratings_f",
>          "norm":{
>                  "class" : "org.apache.solr.ltr.norm.MinMaxNormalizer",
>                     "params" : { "min":"1.0", "max":"5.0"}
>          }
>         },
>         {
>             "name": "p_instore_trans_cnt_f",
>              "norm":{
>                  "class" : "org.apache.solr.ltr.norm.MinMaxNormalizer",
>                     "params" : { "min":"1.0", "max":"209561.0" }
>          }
>         },
>         {
>             "name": "p_reviews_f",
>              "norm":{
>                  "class" : "org.apache.solr.ltr.norm.MinMaxNormalizer",
>                     "params" : { "min":"0.0", "max":"58375.0"}
>          }
>         }
>
>  The model got uploaded successfully, but when I reloaded the collection,
> it failed and below error is observed:
>
> Caused by: java.lang.RuntimeException: Error invoking setter setMin on
> class : org.apache.solr.ltr.norm.MinMaxNormalizer
>         at
> org.apache.solr.util.SolrPluginUtils.invokeSetters(SolrPluginUtils.java:1084)
>         at
> org.apache.solr.ltr.norm.Normalizer.getInstance(Normalizer.java:49)
>         at
> org.apache.solr.ltr.store.rest.ManagedModelStore.fromNormalizerMap(ManagedModelStore.java:293)
>         at
> org.apache.solr.ltr.store.rest.ManagedModelStore.createNormalizerFromFeatureMap(ManagedModelStore.java:276)
>         at
> org.apache.solr.ltr.store.rest.ManagedModelStore.fromLTRScoringModelMap(ManagedModelStore.java:230)
>         at
> org.apache.solr.ltr.store.rest.ManagedModelStore.addModelFromMap(ManagedModelStore.java:133)
>         at
> org.apache.solr.ltr.store.rest.ManagedModelStore.loadStoredModels(ManagedModelStore.java:126)
>         at
> org.apache.solr.ltr.search.LTRQParserPlugin.onManagedResourceInitialized(LTRQParserPlugin.java:133)
>         at
> org.apache.solr.rest.ManagedResource.notifyObserversDuringInit(ManagedResource.java:115)
>         at
> org.apache.solr.rest.ManagedResource.loadManagedDataAndNotify(ManagedResource.java:91)
>         at
> org.apache.solr.rest.RestManager.createManagedResource(RestManager.java:694)
>         ... 41 more
> Caused by: java.lang.reflect.InvocationTargetException
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:498)
>         at
> org.apache.solr.util.SolrPluginUtils.invokeSetters(SolrPluginUtils.java:1082)
>         ... 51 more
> Caused by: java.lang.NumberFormatException: For input string: ""0.0""
>         at
> sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)
>         at sun.misc.FloatingDecimal.parseFloat(FloatingDecimal.java:122)
>         at java.lang.Float.parseFloat(Float.java:451)
>         at
> org.apache.solr.ltr.norm.MinMaxNormalizer.setMin(MinMaxNormalizer.java:58)
>
> I tried uploading the model without double quotes in param value for min
> and max, it also got failed with below error.
>
> "java.lang.IllegalArgumentException: argument type mismatch\n\tat
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:498)\n\tat
> org.apache.solr.util.SolrPluginUtils.invokeSetters(SolrPluginUtils.java:1082)\n\tat
> org.apache.solr.ltr.norm.Normalizer.getInstance(Normalizer.java:49)\n\tat
> org.apache.solr.ltr.store.rest.ManagedModelStore.fromNormalizerMap(ManagedModelStore.java:293)\n\tat
> org.apache.solr.ltr.store.rest.ManagedModelStore.createNormalizerFromFeatureMap(ManagedModelStore.java:276)\n\tat
> org.apache.solr.ltr.store.rest.ManagedModelStore.fromLTRScoringModelMap(ManagedModelStore.java:230)\n\tat
> org.apache.solr.ltr.store.rest.ManagedModelStore.addModelFromMap(ManagedModelStore.java:133)\n\tat
> org.apache.solr.ltr.store.rest.ManagedModelStore.applyUpdatesToManagedData(ManagedModelStore.java:162)\n\tat
>
>
> Also, I can see on the collection that there is escape in double quotes in
> model on model api (s/s below).
>
>
> Regards
> Kamal Kishore
>
>