Using an & in an indexed field and then querying for it.

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Using an & in an indexed field and then querying for it.

rhys J
I have some fields that have text like so:

Reliable Van & Storage.

They indexed fine when I used curl and csv files to read them into the core.

Now when I try to query for them, I get errors.

If I try escaping it like so \&, I get the following error:

on_data({
  "responseHeader":{
    "status":400,
    "QTime":1,
    "params":{
      "q":"name1:( reliable van \\",
      "core":"dbtr",
      "json.wrf":"on_data",
      "hl":"true",
      "indent":"on",
      "start":"0",
      "stor )":"",
      "callback":"?",
      "rows":"50",
      "version":"2.2",
      "wt":"json"}},
  "error":{
    "metadata":[
      "error-class","org.apache.solr.common.SolrException",
      "root-error-class","org.apache.solr.parser.TokenMgrError"],
    "msg":"org.apache.solr.search.SyntaxError: Cannot parse 'name1:(
reliable van \\': Lexical error at line 1, column 23.  Encountered:
<EOF> after : \"\"",
    "code":400}})

If I try html encoding it like so: &amp; I get the following error:



on_data({
  "responseHeader":{
    "status":400,
    "QTime":3,
    "params":{
      "q":"name1:( reliable van ",
      "core":"dbtr",
      "json.wrf":"on_data",
      "hl":"true",
      "indent":"on",
      "amp; stor )":"",
      "start":"0",
      "callback":"?",
      "rows":"50",
      "version":"2.2",
      "wt":"json"}},
  "error":{
    "metadata":[
      "error-class","org.apache.solr.common.SolrException",
      "root-error-class","org.apache.solr.parser.ParseException"],
    "msg":"org.apache.solr.search.SyntaxError: Cannot parse 'name1:(
reliable van ': Encountered \"<EOF>\" at line 1, column 21.\nWas
expecting one of:\n    <AND> ...\n    <OR> ...\n    <NOT> ...\n
\"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \")\"
...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM>
...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n
\"{\" ...\n    <LPARAMS> ...\n    \"filter(\" ...\n    <NUMBER> ...\n
  ",
    "code":400}})


How can I search for a field that has an & without breaking the
parser, or is it not possible because & is used as a special
character?

Thanks,

Rhys
Reply | Threaded
Open this post in threaded view
|

Re: Using an & in an indexed field and then querying for it.

David Hastings
its breaking on the & because its in the url and you are most likely
sending a get request to solr.  you should send it as post or as %26

On Mon, Nov 25, 2019 at 2:32 PM rhys J <[hidden email]> wrote:

> I have some fields that have text like so:
>
> Reliable Van & Storage.
>
> They indexed fine when I used curl and csv files to read them into the
> core.
>
> Now when I try to query for them, I get errors.
>
> If I try escaping it like so \&, I get the following error:
>
> on_data({
>   "responseHeader":{
>     "status":400,
>     "QTime":1,
>     "params":{
>       "q":"name1:( reliable van \\",
>       "core":"dbtr",
>       "json.wrf":"on_data",
>       "hl":"true",
>       "indent":"on",
>       "start":"0",
>       "stor )":"",
>       "callback":"?",
>       "rows":"50",
>       "version":"2.2",
>       "wt":"json"}},
>   "error":{
>     "metadata":[
>       "error-class","org.apache.solr.common.SolrException",
>       "root-error-class","org.apache.solr.parser.TokenMgrError"],
>     "msg":"org.apache.solr.search.SyntaxError: Cannot parse 'name1:(
> reliable van \\': Lexical error at line 1, column 23.  Encountered:
> <EOF> after : \"\"",
>     "code":400}})
>
> If I try html encoding it like so: &amp; I get the following error:
>
>
>
> on_data({
>   "responseHeader":{
>     "status":400,
>     "QTime":3,
>     "params":{
>       "q":"name1:( reliable van ",
>       "core":"dbtr",
>       "json.wrf":"on_data",
>       "hl":"true",
>       "indent":"on",
>       "amp; stor )":"",
>       "start":"0",
>       "callback":"?",
>       "rows":"50",
>       "version":"2.2",
>       "wt":"json"}},
>   "error":{
>     "metadata":[
>       "error-class","org.apache.solr.common.SolrException",
>       "root-error-class","org.apache.solr.parser.ParseException"],
>     "msg":"org.apache.solr.search.SyntaxError: Cannot parse 'name1:(
> reliable van ': Encountered \"<EOF>\" at line 1, column 21.\nWas
> expecting one of:\n    <AND> ...\n    <OR> ...\n    <NOT> ...\n
> \"+\" ...\n    \"-\" ...\n    <BAREOPER> ...\n    \"(\" ...\n    \")\"
> ...\n    \"*\" ...\n    <QUOTED> ...\n    <TERM> ...\n    <PREFIXTERM>
> ...\n    <WILDTERM> ...\n    <REGEXPTERM> ...\n    \"[\" ...\n
> \"{\" ...\n    <LPARAMS> ...\n    \"filter(\" ...\n    <NUMBER> ...\n
>   ",
>     "code":400}})
>
>
> How can I search for a field that has an & without breaking the
> parser, or is it not possible because & is used as a special
> character?
>
> Thanks,
>
> Rhys
>
Reply | Threaded
Open this post in threaded view
|

Re: Using an & in an indexed field and then querying for it.

rhys J
On Mon, Nov 25, 2019 at 2:36 PM David Hastings <[hidden email]>
wrote:

> its breaking on the & because its in the url and you are most likely
> sending a get request to solr.  you should send it as post or as %26
>
>
The package I am using doesn't have a postJSON function available, so I'm
using their getJSON function.

I changed the & to %26, and that fixed things.

Thanks,

Rhys
Reply | Threaded
Open this post in threaded view
|

Re: Using an & in an indexed field and then querying for it.

David Hastings
This again is why its recommended to have something inbetween your js and
solr, also gets have limitations so if eventually you start getting more
complications you may hit it:

https://stackoverflow.com/questions/2659952/maximum-length-of-http-get-request

On Mon, Nov 25, 2019 at 2:40 PM rhys J <[hidden email]> wrote:

> On Mon, Nov 25, 2019 at 2:36 PM David Hastings <
> [hidden email]>
> wrote:
>
> > its breaking on the & because its in the url and you are most likely
> > sending a get request to solr.  you should send it as post or as %26
> >
> >
> The package I am using doesn't have a postJSON function available, so I'm
> using their getJSON function.
>
> I changed the & to %26, and that fixed things.
>
> Thanks,
>
> Rhys
>