Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

Colvin Cowie
Hello. I encountered this issue too and wrote this up before I found this
thread, but I thought I might as well post it still, if it helps...

Currently I'm trying to move our product on to Solr 8.1.1. We are currently
using 6.6.6, so things have definitely moved on.

We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down Solr
(and we also secure our zookeeper). Here's an example for solradmin as the
user and password

{
    "authentication": {
        "blockUnknown": true,
        "class": "solr.BasicAuthPlugin",
        "credentials": {
            "solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
        }
    },
    "authorization": {
        "class": "solr.RuleBasedAuthorizationPlugin",
        "permissions": [
            {
                "name": "all",
                "role": "admin"
            }
        ],
        "user-role": {
            "solradmin": "admin"
        }
    }
}


On Solr 8.1.1, using our previously working security.json, running queries
(through the admin UI currently) I non-deterministically get 401 responses
on queries when a collection has more than 1 shard. Increasing the number
of shards in the collection makes the errors more likely.

{
  "responseHeader":{
    "zkConnected":true,
    "status":401,
    "QTime":30,
    "params":{
      "q":"*:*",
      "_":"1559474550365"}},
  "error":{
    "metadata":[

"error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",

"root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
    "msg":"Error from server at null: Expected mime type
application/octet-stream but got text/html. <html>\n<head>\n<meta
http-equiv=\"Content-Type\"
content=\"text/html;charset=utf-8\"/>\n<title>Error 401 require
authentication</title>\n</head>\n<body><h2>HTTP ERROR 401</h2>\n<p>Problem
accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n<pre>
 require authentication</pre></p>\n</body>\n</html>\n",
    "code":401}}

The security stats indicate this is happening because the requests do not
have credentials with them, e.g.
http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin

 org.apache.solr.security.BasicAuthPlugin
    class:
        org.apache.solr.security.BasicAuthPlugin
    description:
        Authentication Plugin org.apache.solr.security.BasicAuthPlugin
    stats
        SECURITY./authentication.authenticated:
            182
        SECURITY./authentication.errors.count:
            0
        SECURITY./authentication.failMissingCredentials:
            58
        SECURITY./authentication.failWrongCredentials:
            0
        SECURITY./authentication.passThrough:
            0
        SECURITY./authentication.requestTimes.meanRate:
            0.4183414110946125
        SECURITY./authentication.requests:
            240
        SECURITY./authentication.totalTime:
            117791100

I assume that this is connected to the changes around
https://issues.apache.org/jira/browse/SOLR-7896 and
https://issues.apache.org/jira/browse/SOLR-13344 I've tested with Solr
7.6.0 and it appears to be unaffected

Repro steps:
   # Extract solr 8.1.1.
   # bin\solr start -e cloud
        1 node / [default port] / [default collection name] / 4 shards / 1
replica / [_default configuration]
   # server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd putfile
/security.json <example-security.json file with content from example above>

   # Execute repeated GETS to
http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot of them,
but not all, will fail with 401s


Also as a side note, because the authentication is now done through the
form login rather than the browser basic auth, if you go directly to a non
UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*) you have
to authenticate to it using the browser's basic auth prompt. Which is
slightly annoying since the query page in the Admin UI generates links to
it for the queries you run, and you've already authenticated to get there.
But it's not a massive burden or anything... I guess I just preferred
having the browser BA prompt.

Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

Jason Gerlowski
I'm also able to reproduce this bug on master.  A few more notes about
the bad behavior:

- the behavior occurs regardless of the specific permissions
configured in security.json.  (i.e. whether the top permission is
"all", or "security-edit", or there are no permissions at all.)
- I tried looking for a pattern in which requests saw the 401s, but
didn't have any luck.  The 401 occurs when talking to the whole
collection or targeting individual cores directly.  It occurs when
curl hits a host containing a replica for the collection in question,
and when it doesnt. etc.  This distinguishes it from SOLR-13472, which
seems more specific to collection structure/layout.

I'll create a bug for this in JIRA.

On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <[hidden email]> wrote:

>
> Hello. I encountered this issue too and wrote this up before I found this
> thread, but I thought I might as well post it still, if it helps...
>
> Currently I'm trying to move our product on to Solr 8.1.1. We are currently
> using 6.6.6, so things have definitely moved on.
>
> We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down Solr
> (and we also secure our zookeeper). Here's an example for solradmin as the
> user and password
>
> {
>     "authentication": {
>         "blockUnknown": true,
>         "class": "solr.BasicAuthPlugin",
>         "credentials": {
>             "solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
>         }
>     },
>     "authorization": {
>         "class": "solr.RuleBasedAuthorizationPlugin",
>         "permissions": [
>             {
>                 "name": "all",
>                 "role": "admin"
>             }
>         ],
>         "user-role": {
>             "solradmin": "admin"
>         }
>     }
> }
>
>
> On Solr 8.1.1, using our previously working security.json, running queries
> (through the admin UI currently) I non-deterministically get 401 responses
> on queries when a collection has more than 1 shard. Increasing the number
> of shards in the collection makes the errors more likely.
>
> {
>   "responseHeader":{
>     "zkConnected":true,
>     "status":401,
>     "QTime":30,
>     "params":{
>       "q":"*:*",
>       "_":"1559474550365"}},
>   "error":{
>     "metadata":[
>
> "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
>
> "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
>     "msg":"Error from server at null: Expected mime type
> application/octet-stream but got text/html. <html>\n<head>\n<meta
> http-equiv=\"Content-Type\"
> content=\"text/html;charset=utf-8\"/>\n<title>Error 401 require
> authentication</title>\n</head>\n<body><h2>HTTP ERROR 401</h2>\n<p>Problem
> accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n<pre>
>  require authentication</pre></p>\n</body>\n</html>\n",
>     "code":401}}
>
> The security stats indicate this is happening because the requests do not
> have credentials with them, e.g.
> http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
>
>  org.apache.solr.security.BasicAuthPlugin
>     class:
>         org.apache.solr.security.BasicAuthPlugin
>     description:
>         Authentication Plugin org.apache.solr.security.BasicAuthPlugin
>     stats
>         SECURITY./authentication.authenticated:
>             182
>         SECURITY./authentication.errors.count:
>             0
>         SECURITY./authentication.failMissingCredentials:
>             58
>         SECURITY./authentication.failWrongCredentials:
>             0
>         SECURITY./authentication.passThrough:
>             0
>         SECURITY./authentication.requestTimes.meanRate:
>             0.4183414110946125
>         SECURITY./authentication.requests:
>             240
>         SECURITY./authentication.totalTime:
>             117791100
>
> I assume that this is connected to the changes around
> https://issues.apache.org/jira/browse/SOLR-7896 and
> https://issues.apache.org/jira/browse/SOLR-13344 I've tested with Solr
> 7.6.0 and it appears to be unaffected
>
> Repro steps:
>    # Extract solr 8.1.1.
>    # bin\solr start -e cloud
>         1 node / [default port] / [default collection name] / 4 shards / 1
> replica / [_default configuration]
>    # server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd putfile
> /security.json <example-security.json file with content from example above>
>
>    # Execute repeated GETS to
> http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot of them,
> but not all, will fail with 401s
>
>
> Also as a side note, because the authentication is now done through the
> form login rather than the browser basic auth, if you go directly to a non
> UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*) you have
> to authenticate to it using the browser's basic auth prompt. Which is
> slightly annoying since the query page in the Admin UI generates links to
> it for the queries you run, and you've already authenticated to get there.
> But it's not a massive burden or anything... I guess I just preferred
> having the browser BA prompt.
>
> Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

Jason Gerlowski
One last note: as far as I can tell, nothing about this issue is
specific to JSON Faceting or the JSON request API.  It can be
triggered just as easily with "/select?q=*:*".

The bug created for this is: SOLR-13510

On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski <[hidden email]> wrote:

>
> I'm also able to reproduce this bug on master.  A few more notes about
> the bad behavior:
>
> - the behavior occurs regardless of the specific permissions
> configured in security.json.  (i.e. whether the top permission is
> "all", or "security-edit", or there are no permissions at all.)
> - I tried looking for a pattern in which requests saw the 401s, but
> didn't have any luck.  The 401 occurs when talking to the whole
> collection or targeting individual cores directly.  It occurs when
> curl hits a host containing a replica for the collection in question,
> and when it doesnt. etc.  This distinguishes it from SOLR-13472, which
> seems more specific to collection structure/layout.
>
> I'll create a bug for this in JIRA.
>
> On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <[hidden email]> wrote:
> >
> > Hello. I encountered this issue too and wrote this up before I found this
> > thread, but I thought I might as well post it still, if it helps...
> >
> > Currently I'm trying to move our product on to Solr 8.1.1. We are currently
> > using 6.6.6, so things have definitely moved on.
> >
> > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down Solr
> > (and we also secure our zookeeper). Here's an example for solradmin as the
> > user and password
> >
> > {
> >     "authentication": {
> >         "blockUnknown": true,
> >         "class": "solr.BasicAuthPlugin",
> >         "credentials": {
> >             "solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> >         }
> >     },
> >     "authorization": {
> >         "class": "solr.RuleBasedAuthorizationPlugin",
> >         "permissions": [
> >             {
> >                 "name": "all",
> >                 "role": "admin"
> >             }
> >         ],
> >         "user-role": {
> >             "solradmin": "admin"
> >         }
> >     }
> > }
> >
> >
> > On Solr 8.1.1, using our previously working security.json, running queries
> > (through the admin UI currently) I non-deterministically get 401 responses
> > on queries when a collection has more than 1 shard. Increasing the number
> > of shards in the collection makes the errors more likely.
> >
> > {
> >   "responseHeader":{
> >     "zkConnected":true,
> >     "status":401,
> >     "QTime":30,
> >     "params":{
> >       "q":"*:*",
> >       "_":"1559474550365"}},
> >   "error":{
> >     "metadata":[
> >
> > "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> >
> > "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> >     "msg":"Error from server at null: Expected mime type
> > application/octet-stream but got text/html. <html>\n<head>\n<meta
> > http-equiv=\"Content-Type\"
> > content=\"text/html;charset=utf-8\"/>\n<title>Error 401 require
> > authentication</title>\n</head>\n<body><h2>HTTP ERROR 401</h2>\n<p>Problem
> > accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n<pre>
> >  require authentication</pre></p>\n</body>\n</html>\n",
> >     "code":401}}
> >
> > The security stats indicate this is happening because the requests do not
> > have credentials with them, e.g.
> > http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> >
> >  org.apache.solr.security.BasicAuthPlugin
> >     class:
> >         org.apache.solr.security.BasicAuthPlugin
> >     description:
> >         Authentication Plugin org.apache.solr.security.BasicAuthPlugin
> >     stats
> >         SECURITY./authentication.authenticated:
> >             182
> >         SECURITY./authentication.errors.count:
> >             0
> >         SECURITY./authentication.failMissingCredentials:
> >             58
> >         SECURITY./authentication.failWrongCredentials:
> >             0
> >         SECURITY./authentication.passThrough:
> >             0
> >         SECURITY./authentication.requestTimes.meanRate:
> >             0.4183414110946125
> >         SECURITY./authentication.requests:
> >             240
> >         SECURITY./authentication.totalTime:
> >             117791100
> >
> > I assume that this is connected to the changes around
> > https://issues.apache.org/jira/browse/SOLR-7896 and
> > https://issues.apache.org/jira/browse/SOLR-13344 I've tested with Solr
> > 7.6.0 and it appears to be unaffected
> >
> > Repro steps:
> >    # Extract solr 8.1.1.
> >    # bin\solr start -e cloud
> >         1 node / [default port] / [default collection name] / 4 shards / 1
> > replica / [_default configuration]
> >    # server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd putfile
> > /security.json <example-security.json file with content from example above>
> >
> >    # Execute repeated GETS to
> > http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot of them,
> > but not all, will fail with 401s
> >
> >
> > Also as a side note, because the authentication is now done through the
> > form login rather than the browser basic auth, if you go directly to a non
> > UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*) you have
> > to authenticate to it using the browser's basic auth prompt. Which is
> > slightly annoying since the query page in the Admin UI generates links to
> > it for the queries you run, and you've already authenticated to get there.
> > But it's not a massive burden or anything... I guess I just preferred
> > having the browser BA prompt.
> >
> > Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

Jason Gerlowski
Hi Colvin,

We're still taking a look at fixing the bug, but as a workaround in
the meantime, you can look into adding a "forwardCredentials":true
property under the "authentication" section of security.json.  That
seems to fix the issue in my reproduction at least.

e.g.

{
    "authentication": {
        "blockUnknown": true,
        "class": "solr.BasicAuthPlugin",
        "credentials": {
            "solradmin": "<encoded-pw>"
        },
        "forwardCredentials": true
    },
    ...
}

Jason

On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski <[hidden email]> wrote:

>
> One last note: as far as I can tell, nothing about this issue is
> specific to JSON Faceting or the JSON request API.  It can be
> triggered just as easily with "/select?q=*:*".
>
> The bug created for this is: SOLR-13510
>
> On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski <[hidden email]> wrote:
> >
> > I'm also able to reproduce this bug on master.  A few more notes about
> > the bad behavior:
> >
> > - the behavior occurs regardless of the specific permissions
> > configured in security.json.  (i.e. whether the top permission is
> > "all", or "security-edit", or there are no permissions at all.)
> > - I tried looking for a pattern in which requests saw the 401s, but
> > didn't have any luck.  The 401 occurs when talking to the whole
> > collection or targeting individual cores directly.  It occurs when
> > curl hits a host containing a replica for the collection in question,
> > and when it doesnt. etc.  This distinguishes it from SOLR-13472, which
> > seems more specific to collection structure/layout.
> >
> > I'll create a bug for this in JIRA.
> >
> > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <[hidden email]> wrote:
> > >
> > > Hello. I encountered this issue too and wrote this up before I found this
> > > thread, but I thought I might as well post it still, if it helps...
> > >
> > > Currently I'm trying to move our product on to Solr 8.1.1. We are currently
> > > using 6.6.6, so things have definitely moved on.
> > >
> > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock down Solr
> > > (and we also secure our zookeeper). Here's an example for solradmin as the
> > > user and password
> > >
> > > {
> > >     "authentication": {
> > >         "blockUnknown": true,
> > >         "class": "solr.BasicAuthPlugin",
> > >         "credentials": {
> > >             "solradmin": "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > >         }
> > >     },
> > >     "authorization": {
> > >         "class": "solr.RuleBasedAuthorizationPlugin",
> > >         "permissions": [
> > >             {
> > >                 "name": "all",
> > >                 "role": "admin"
> > >             }
> > >         ],
> > >         "user-role": {
> > >             "solradmin": "admin"
> > >         }
> > >     }
> > > }
> > >
> > >
> > > On Solr 8.1.1, using our previously working security.json, running queries
> > > (through the admin UI currently) I non-deterministically get 401 responses
> > > on queries when a collection has more than 1 shard. Increasing the number
> > > of shards in the collection makes the errors more likely.
> > >
> > > {
> > >   "responseHeader":{
> > >     "zkConnected":true,
> > >     "status":401,
> > >     "QTime":30,
> > >     "params":{
> > >       "q":"*:*",
> > >       "_":"1559474550365"}},
> > >   "error":{
> > >     "metadata":[
> > >
> > > "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > >
> > > "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > >     "msg":"Error from server at null: Expected mime type
> > > application/octet-stream but got text/html. <html>\n<head>\n<meta
> > > http-equiv=\"Content-Type\"
> > > content=\"text/html;charset=utf-8\"/>\n<title>Error 401 require
> > > authentication</title>\n</head>\n<body><h2>HTTP ERROR 401</h2>\n<p>Problem
> > > accessing /solr/gettingstarted_shard4_replica_n6/select. Reason:\n<pre>
> > >  require authentication</pre></p>\n</body>\n</html>\n",
> > >     "code":401}}
> > >
> > > The security stats indicate this is happening because the requests do not
> > > have credentials with them, e.g.
> > > http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> > >
> > >  org.apache.solr.security.BasicAuthPlugin
> > >     class:
> > >         org.apache.solr.security.BasicAuthPlugin
> > >     description:
> > >         Authentication Plugin org.apache.solr.security.BasicAuthPlugin
> > >     stats
> > >         SECURITY./authentication.authenticated:
> > >             182
> > >         SECURITY./authentication.errors.count:
> > >             0
> > >         SECURITY./authentication.failMissingCredentials:
> > >             58
> > >         SECURITY./authentication.failWrongCredentials:
> > >             0
> > >         SECURITY./authentication.passThrough:
> > >             0
> > >         SECURITY./authentication.requestTimes.meanRate:
> > >             0.4183414110946125
> > >         SECURITY./authentication.requests:
> > >             240
> > >         SECURITY./authentication.totalTime:
> > >             117791100
> > >
> > > I assume that this is connected to the changes around
> > > https://issues.apache.org/jira/browse/SOLR-7896 and
> > > https://issues.apache.org/jira/browse/SOLR-13344 I've tested with Solr
> > > 7.6.0 and it appears to be unaffected
> > >
> > > Repro steps:
> > >    # Extract solr 8.1.1.
> > >    # bin\solr start -e cloud
> > >         1 node / [default port] / [default collection name] / 4 shards / 1
> > > replica / [_default configuration]
> > >    # server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd putfile
> > > /security.json <example-security.json file with content from example above>
> > >
> > >    # Execute repeated GETS to
> > > http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot of them,
> > > but not all, will fail with 401s
> > >
> > >
> > > Also as a side note, because the authentication is now done through the
> > > form login rather than the browser basic auth, if you go directly to a non
> > > UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*) you have
> > > to authenticate to it using the browser's basic auth prompt. Which is
> > > slightly annoying since the query page in the Admin UI generates links to
> > > it for the queries you run, and you've already authenticated to get there.
> > > But it's not a massive burden or anything... I guess I just preferred
> > > having the browser BA prompt.
> > >
> > > Thanks
Reply | Threaded
Open this post in threaded view
|

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

Colvin Cowie
Hi, thanks I'll give that a go when I get a chance.

I was trying to reply to an older thread (
http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E),
which I don't have in my mailbox, so obviously didn't reply to the right
address to get my response threaded so mine has appeared on its own. Oops.

A JIRA issue was raised on that thread
https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
attention.


On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski <[hidden email]> wrote:

> Hi Colvin,
>
> We're still taking a look at fixing the bug, but as a workaround in
> the meantime, you can look into adding a "forwardCredentials":true
> property under the "authentication" section of security.json.  That
> seems to fix the issue in my reproduction at least.
>
> e.g.
>
> {
>     "authentication": {
>         "blockUnknown": true,
>         "class": "solr.BasicAuthPlugin",
>         "credentials": {
>             "solradmin": "<encoded-pw>"
>         },
>         "forwardCredentials": true
>     },
>     ...
> }
>
> Jason
>
> On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski <[hidden email]>
> wrote:
> >
> > One last note: as far as I can tell, nothing about this issue is
> > specific to JSON Faceting or the JSON request API.  It can be
> > triggered just as easily with "/select?q=*:*".
> >
> > The bug created for this is: SOLR-13510
> >
> > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski <[hidden email]>
> wrote:
> > >
> > > I'm also able to reproduce this bug on master.  A few more notes about
> > > the bad behavior:
> > >
> > > - the behavior occurs regardless of the specific permissions
> > > configured in security.json.  (i.e. whether the top permission is
> > > "all", or "security-edit", or there are no permissions at all.)
> > > - I tried looking for a pattern in which requests saw the 401s, but
> > > didn't have any luck.  The 401 occurs when talking to the whole
> > > collection or targeting individual cores directly.  It occurs when
> > > curl hits a host containing a replica for the collection in question,
> > > and when it doesnt. etc.  This distinguishes it from SOLR-13472, which
> > > seems more specific to collection structure/layout.
> > >
> > > I'll create a bug for this in JIRA.
> > >
> > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> [hidden email]> wrote:
> > > >
> > > > Hello. I encountered this issue too and wrote this up before I found
> this
> > > > thread, but I thought I might as well post it still, if it helps...
> > > >
> > > > Currently I'm trying to move our product on to Solr 8.1.1. We are
> currently
> > > > using 6.6.6, so things have definitely moved on.
> > > >
> > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock
> down Solr
> > > > (and we also secure our zookeeper). Here's an example for solradmin
> as the
> > > > user and password
> > > >
> > > > {
> > > >     "authentication": {
> > > >         "blockUnknown": true,
> > > >         "class": "solr.BasicAuthPlugin",
> > > >         "credentials": {
> > > >             "solradmin":
> "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > >         }
> > > >     },
> > > >     "authorization": {
> > > >         "class": "solr.RuleBasedAuthorizationPlugin",
> > > >         "permissions": [
> > > >             {
> > > >                 "name": "all",
> > > >                 "role": "admin"
> > > >             }
> > > >         ],
> > > >         "user-role": {
> > > >             "solradmin": "admin"
> > > >         }
> > > >     }
> > > > }
> > > >
> > > >
> > > > On Solr 8.1.1, using our previously working security.json, running
> queries
> > > > (through the admin UI currently) I non-deterministically get 401
> responses
> > > > on queries when a collection has more than 1 shard. Increasing the
> number
> > > > of shards in the collection makes the errors more likely.
> > > >
> > > > {
> > > >   "responseHeader":{
> > > >     "zkConnected":true,
> > > >     "status":401,
> > > >     "QTime":30,
> > > >     "params":{
> > > >       "q":"*:*",
> > > >       "_":"1559474550365"}},
> > > >   "error":{
> > > >     "metadata":[
> > > >
> > > >
> "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > > >
> > > >
> "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > >     "msg":"Error from server at null: Expected mime type
> > > > application/octet-stream but got text/html. <html>\n<head>\n<meta
> > > > http-equiv=\"Content-Type\"
> > > > content=\"text/html;charset=utf-8\"/>\n<title>Error 401 require
> > > > authentication</title>\n</head>\n<body><h2>HTTP ERROR
> 401</h2>\n<p>Problem
> > > > accessing /solr/gettingstarted_shard4_replica_n6/select.
> Reason:\n<pre>
> > > >  require authentication</pre></p>\n</body>\n</html>\n",
> > > >     "code":401}}
> > > >
> > > > The security stats indicate this is happening because the requests
> do not
> > > > have credentials with them, e.g.
> > > >
> http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> > > >
> > > >  org.apache.solr.security.BasicAuthPlugin
> > > >     class:
> > > >         org.apache.solr.security.BasicAuthPlugin
> > > >     description:
> > > >         Authentication Plugin
> org.apache.solr.security.BasicAuthPlugin
> > > >     stats
> > > >         SECURITY./authentication.authenticated:
> > > >             182
> > > >         SECURITY./authentication.errors.count:
> > > >             0
> > > >         SECURITY./authentication.failMissingCredentials:
> > > >             58
> > > >         SECURITY./authentication.failWrongCredentials:
> > > >             0
> > > >         SECURITY./authentication.passThrough:
> > > >             0
> > > >         SECURITY./authentication.requestTimes.meanRate:
> > > >             0.4183414110946125
> > > >         SECURITY./authentication.requests:
> > > >             240
> > > >         SECURITY./authentication.totalTime:
> > > >             117791100
> > > >
> > > > I assume that this is connected to the changes around
> > > > https://issues.apache.org/jira/browse/SOLR-7896 and
> > > > https://issues.apache.org/jira/browse/SOLR-13344 I've tested with
> Solr
> > > > 7.6.0 and it appears to be unaffected
> > > >
> > > > Repro steps:
> > > >    # Extract solr 8.1.1.
> > > >    # bin\solr start -e cloud
> > > >         1 node / [default port] / [default collection name] / 4
> shards / 1
> > > > replica / [_default configuration]
> > > >    # server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd
> putfile
> > > > /security.json <example-security.json file with content from example
> above>
> > > >
> > > >    # Execute repeated GETS to
> > > > http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot of
> them,
> > > > but not all, will fail with 401s
> > > >
> > > >
> > > > Also as a side note, because the authentication is now done through
> the
> > > > form login rather than the browser basic auth, if you go directly to
> a non
> > > > UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*)
> you have
> > > > to authenticate to it using the browser's basic auth prompt. Which is
> > > > slightly annoying since the query page in the Admin UI generates
> links to
> > > > it for the queries you run, and you've already authenticated to get
> there.
> > > > But it's not a massive burden or anything... I guess I just preferred
> > > > having the browser BA prompt.
> > > >
> > > > Thanks
>
Reply | Threaded
Open this post in threaded view
|

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

Zheng Lin Edwin Yeo
Hi Jason,

Thanks for your reply.

I have tried to add the "forwardCredentials": true in the security.json,
but I still get the same error.

Regards,
Edwin

On Mon, 3 Jun 2019 at 22:19, Colvin Cowie <[hidden email]>
wrote:

> Hi, thanks I'll give that a go when I get a chance.
>
> I was trying to reply to an older thread (
>
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E
> ),
> which I don't have in my mailbox, so obviously didn't reply to the right
> address to get my response threaded so mine has appeared on its own. Oops.
>
> A JIRA issue was raised on that thread
> https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
> attention.
>
>
> On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski <[hidden email]>
> wrote:
>
> > Hi Colvin,
> >
> > We're still taking a look at fixing the bug, but as a workaround in
> > the meantime, you can look into adding a "forwardCredentials":true
> > property under the "authentication" section of security.json.  That
> > seems to fix the issue in my reproduction at least.
> >
> > e.g.
> >
> > {
> >     "authentication": {
> >         "blockUnknown": true,
> >         "class": "solr.BasicAuthPlugin",
> >         "credentials": {
> >             "solradmin": "<encoded-pw>"
> >         },
> >         "forwardCredentials": true
> >     },
> >     ...
> > }
> >
> > Jason
> >
> > On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski <[hidden email]>
> > wrote:
> > >
> > > One last note: as far as I can tell, nothing about this issue is
> > > specific to JSON Faceting or the JSON request API.  It can be
> > > triggered just as easily with "/select?q=*:*".
> > >
> > > The bug created for this is: SOLR-13510
> > >
> > > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski <[hidden email]>
> > wrote:
> > > >
> > > > I'm also able to reproduce this bug on master.  A few more notes
> about
> > > > the bad behavior:
> > > >
> > > > - the behavior occurs regardless of the specific permissions
> > > > configured in security.json.  (i.e. whether the top permission is
> > > > "all", or "security-edit", or there are no permissions at all.)
> > > > - I tried looking for a pattern in which requests saw the 401s, but
> > > > didn't have any luck.  The 401 occurs when talking to the whole
> > > > collection or targeting individual cores directly.  It occurs when
> > > > curl hits a host containing a replica for the collection in question,
> > > > and when it doesnt. etc.  This distinguishes it from SOLR-13472,
> which
> > > > seems more specific to collection structure/layout.
> > > >
> > > > I'll create a bug for this in JIRA.
> > > >
> > > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> > [hidden email]> wrote:
> > > > >
> > > > > Hello. I encountered this issue too and wrote this up before I
> found
> > this
> > > > > thread, but I thought I might as well post it still, if it helps...
> > > > >
> > > > > Currently I'm trying to move our product on to Solr 8.1.1. We are
> > currently
> > > > > using 6.6.6, so things have definitely moved on.
> > > > >
> > > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock
> > down Solr
> > > > > (and we also secure our zookeeper). Here's an example for solradmin
> > as the
> > > > > user and password
> > > > >
> > > > > {
> > > > >     "authentication": {
> > > > >         "blockUnknown": true,
> > > > >         "class": "solr.BasicAuthPlugin",
> > > > >         "credentials": {
> > > > >             "solradmin":
> > "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > > >         }
> > > > >     },
> > > > >     "authorization": {
> > > > >         "class": "solr.RuleBasedAuthorizationPlugin",
> > > > >         "permissions": [
> > > > >             {
> > > > >                 "name": "all",
> > > > >                 "role": "admin"
> > > > >             }
> > > > >         ],
> > > > >         "user-role": {
> > > > >             "solradmin": "admin"
> > > > >         }
> > > > >     }
> > > > > }
> > > > >
> > > > >
> > > > > On Solr 8.1.1, using our previously working security.json, running
> > queries
> > > > > (through the admin UI currently) I non-deterministically get 401
> > responses
> > > > > on queries when a collection has more than 1 shard. Increasing the
> > number
> > > > > of shards in the collection makes the errors more likely.
> > > > >
> > > > > {
> > > > >   "responseHeader":{
> > > > >     "zkConnected":true,
> > > > >     "status":401,
> > > > >     "QTime":30,
> > > > >     "params":{
> > > > >       "q":"*:*",
> > > > >       "_":"1559474550365"}},
> > > > >   "error":{
> > > > >     "metadata":[
> > > > >
> > > > >
> >
> "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > > > >
> > > > >
> >
> "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > > >     "msg":"Error from server at null: Expected mime type
> > > > > application/octet-stream but got text/html. <html>\n<head>\n<meta
> > > > > http-equiv=\"Content-Type\"
> > > > > content=\"text/html;charset=utf-8\"/>\n<title>Error 401 require
> > > > > authentication</title>\n</head>\n<body><h2>HTTP ERROR
> > 401</h2>\n<p>Problem
> > > > > accessing /solr/gettingstarted_shard4_replica_n6/select.
> > Reason:\n<pre>
> > > > >  require authentication</pre></p>\n</body>\n</html>\n",
> > > > >     "code":401}}
> > > > >
> > > > > The security stats indicate this is happening because the requests
> > do not
> > > > > have credentials with them, e.g.
> > > > >
> >
> http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> > > > >
> > > > >  org.apache.solr.security.BasicAuthPlugin
> > > > >     class:
> > > > >         org.apache.solr.security.BasicAuthPlugin
> > > > >     description:
> > > > >         Authentication Plugin
> > org.apache.solr.security.BasicAuthPlugin
> > > > >     stats
> > > > >         SECURITY./authentication.authenticated:
> > > > >             182
> > > > >         SECURITY./authentication.errors.count:
> > > > >             0
> > > > >         SECURITY./authentication.failMissingCredentials:
> > > > >             58
> > > > >         SECURITY./authentication.failWrongCredentials:
> > > > >             0
> > > > >         SECURITY./authentication.passThrough:
> > > > >             0
> > > > >         SECURITY./authentication.requestTimes.meanRate:
> > > > >             0.4183414110946125
> > > > >         SECURITY./authentication.requests:
> > > > >             240
> > > > >         SECURITY./authentication.totalTime:
> > > > >             117791100
> > > > >
> > > > > I assume that this is connected to the changes around
> > > > > https://issues.apache.org/jira/browse/SOLR-7896 and
> > > > > https://issues.apache.org/jira/browse/SOLR-13344 I've tested with
> > Solr
> > > > > 7.6.0 and it appears to be unaffected
> > > > >
> > > > > Repro steps:
> > > > >    # Extract solr 8.1.1.
> > > > >    # bin\solr start -e cloud
> > > > >         1 node / [default port] / [default collection name] / 4
> > shards / 1
> > > > > replica / [_default configuration]
> > > > >    # server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd
> > putfile
> > > > > /security.json <example-security.json file with content from
> example
> > above>
> > > > >
> > > > >    # Execute repeated GETS to
> > > > > http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot
> of
> > them,
> > > > > but not all, will fail with 401s
> > > > >
> > > > >
> > > > > Also as a side note, because the authentication is now done through
> > the
> > > > > form login rather than the browser basic auth, if you go directly
> to
> > a non
> > > > > UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*)
> > you have
> > > > > to authenticate to it using the browser's basic auth prompt. Which
> is
> > > > > slightly annoying since the query page in the Admin UI generates
> > links to
> > > > > it for the queries you run, and you've already authenticated to get
> > there.
> > > > > But it's not a massive burden or anything... I guess I just
> preferred
> > > > > having the browser BA prompt.
> > > > >
> > > > > Thanks
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

Jason Gerlowski
Hi Edwin,

Thanks for the additional datapoint.  It seemed to work for me, but we
don't really understand the problem yet, so maybe it's not a solid
work around like I'd hoped.  I'm curious to hear whether it works for
Colvin.

To double check though: forwardCredentials is only supported in Solr >
8.0.  You're using an 8.x version, right?

Jason

On Tue, Jun 4, 2019 at 2:45 AM Zheng Lin Edwin Yeo <[hidden email]> wrote:

>
> Hi Jason,
>
> Thanks for your reply.
>
> I have tried to add the "forwardCredentials": true in the security.json,
> but I still get the same error.
>
> Regards,
> Edwin
>
> On Mon, 3 Jun 2019 at 22:19, Colvin Cowie <[hidden email]>
> wrote:
>
> > Hi, thanks I'll give that a go when I get a chance.
> >
> > I was trying to reply to an older thread (
> >
> > http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E
> > ),
> > which I don't have in my mailbox, so obviously didn't reply to the right
> > address to get my response threaded so mine has appeared on its own. Oops.
> >
> > A JIRA issue was raised on that thread
> > https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
> > attention.
> >
> >
> > On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski <[hidden email]>
> > wrote:
> >
> > > Hi Colvin,
> > >
> > > We're still taking a look at fixing the bug, but as a workaround in
> > > the meantime, you can look into adding a "forwardCredentials":true
> > > property under the "authentication" section of security.json.  That
> > > seems to fix the issue in my reproduction at least.
> > >
> > > e.g.
> > >
> > > {
> > >     "authentication": {
> > >         "blockUnknown": true,
> > >         "class": "solr.BasicAuthPlugin",
> > >         "credentials": {
> > >             "solradmin": "<encoded-pw>"
> > >         },
> > >         "forwardCredentials": true
> > >     },
> > >     ...
> > > }
> > >
> > > Jason
> > >
> > > On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski <[hidden email]>
> > > wrote:
> > > >
> > > > One last note: as far as I can tell, nothing about this issue is
> > > > specific to JSON Faceting or the JSON request API.  It can be
> > > > triggered just as easily with "/select?q=*:*".
> > > >
> > > > The bug created for this is: SOLR-13510
> > > >
> > > > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski <[hidden email]>
> > > wrote:
> > > > >
> > > > > I'm also able to reproduce this bug on master.  A few more notes
> > about
> > > > > the bad behavior:
> > > > >
> > > > > - the behavior occurs regardless of the specific permissions
> > > > > configured in security.json.  (i.e. whether the top permission is
> > > > > "all", or "security-edit", or there are no permissions at all.)
> > > > > - I tried looking for a pattern in which requests saw the 401s, but
> > > > > didn't have any luck.  The 401 occurs when talking to the whole
> > > > > collection or targeting individual cores directly.  It occurs when
> > > > > curl hits a host containing a replica for the collection in question,
> > > > > and when it doesnt. etc.  This distinguishes it from SOLR-13472,
> > which
> > > > > seems more specific to collection structure/layout.
> > > > >
> > > > > I'll create a bug for this in JIRA.
> > > > >
> > > > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> > > [hidden email]> wrote:
> > > > > >
> > > > > > Hello. I encountered this issue too and wrote this up before I
> > found
> > > this
> > > > > > thread, but I thought I might as well post it still, if it helps...
> > > > > >
> > > > > > Currently I'm trying to move our product on to Solr 8.1.1. We are
> > > currently
> > > > > > using 6.6.6, so things have definitely moved on.
> > > > > >
> > > > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to lock
> > > down Solr
> > > > > > (and we also secure our zookeeper). Here's an example for solradmin
> > > as the
> > > > > > user and password
> > > > > >
> > > > > > {
> > > > > >     "authentication": {
> > > > > >         "blockUnknown": true,
> > > > > >         "class": "solr.BasicAuthPlugin",
> > > > > >         "credentials": {
> > > > > >             "solradmin":
> > > "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > > > >         }
> > > > > >     },
> > > > > >     "authorization": {
> > > > > >         "class": "solr.RuleBasedAuthorizationPlugin",
> > > > > >         "permissions": [
> > > > > >             {
> > > > > >                 "name": "all",
> > > > > >                 "role": "admin"
> > > > > >             }
> > > > > >         ],
> > > > > >         "user-role": {
> > > > > >             "solradmin": "admin"
> > > > > >         }
> > > > > >     }
> > > > > > }
> > > > > >
> > > > > >
> > > > > > On Solr 8.1.1, using our previously working security.json, running
> > > queries
> > > > > > (through the admin UI currently) I non-deterministically get 401
> > > responses
> > > > > > on queries when a collection has more than 1 shard. Increasing the
> > > number
> > > > > > of shards in the collection makes the errors more likely.
> > > > > >
> > > > > > {
> > > > > >   "responseHeader":{
> > > > > >     "zkConnected":true,
> > > > > >     "status":401,
> > > > > >     "QTime":30,
> > > > > >     "params":{
> > > > > >       "q":"*:*",
> > > > > >       "_":"1559474550365"}},
> > > > > >   "error":{
> > > > > >     "metadata":[
> > > > > >
> > > > > >
> > >
> > "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > > > > >
> > > > > >
> > >
> > "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > > > >     "msg":"Error from server at null: Expected mime type
> > > > > > application/octet-stream but got text/html. <html>\n<head>\n<meta
> > > > > > http-equiv=\"Content-Type\"
> > > > > > content=\"text/html;charset=utf-8\"/>\n<title>Error 401 require
> > > > > > authentication</title>\n</head>\n<body><h2>HTTP ERROR
> > > 401</h2>\n<p>Problem
> > > > > > accessing /solr/gettingstarted_shard4_replica_n6/select.
> > > Reason:\n<pre>
> > > > > >  require authentication</pre></p>\n</body>\n</html>\n",
> > > > > >     "code":401}}
> > > > > >
> > > > > > The security stats indicate this is happening because the requests
> > > do not
> > > > > > have credentials with them, e.g.
> > > > > >
> > >
> > http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> > > > > >
> > > > > >  org.apache.solr.security.BasicAuthPlugin
> > > > > >     class:
> > > > > >         org.apache.solr.security.BasicAuthPlugin
> > > > > >     description:
> > > > > >         Authentication Plugin
> > > org.apache.solr.security.BasicAuthPlugin
> > > > > >     stats
> > > > > >         SECURITY./authentication.authenticated:
> > > > > >             182
> > > > > >         SECURITY./authentication.errors.count:
> > > > > >             0
> > > > > >         SECURITY./authentication.failMissingCredentials:
> > > > > >             58
> > > > > >         SECURITY./authentication.failWrongCredentials:
> > > > > >             0
> > > > > >         SECURITY./authentication.passThrough:
> > > > > >             0
> > > > > >         SECURITY./authentication.requestTimes.meanRate:
> > > > > >             0.4183414110946125
> > > > > >         SECURITY./authentication.requests:
> > > > > >             240
> > > > > >         SECURITY./authentication.totalTime:
> > > > > >             117791100
> > > > > >
> > > > > > I assume that this is connected to the changes around
> > > > > > https://issues.apache.org/jira/browse/SOLR-7896 and
> > > > > > https://issues.apache.org/jira/browse/SOLR-13344 I've tested with
> > > Solr
> > > > > > 7.6.0 and it appears to be unaffected
> > > > > >
> > > > > > Repro steps:
> > > > > >    # Extract solr 8.1.1.
> > > > > >    # bin\solr start -e cloud
> > > > > >         1 node / [default port] / [default collection name] / 4
> > > shards / 1
> > > > > > replica / [_default configuration]
> > > > > >    # server\scripts\cloud-scripts\zkcli -zkhost localhost:9983 -cmd
> > > putfile
> > > > > > /security.json <example-security.json file with content from
> > example
> > > above>
> > > > > >
> > > > > >    # Execute repeated GETS to
> > > > > > http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a lot
> > of
> > > them,
> > > > > > but not all, will fail with 401s
> > > > > >
> > > > > >
> > > > > > Also as a side note, because the authentication is now done through
> > > the
> > > > > > form login rather than the browser basic auth, if you go directly
> > to
> > > a non
> > > > > > UI url (e.g. http://localhost:8983/solr/main_index/select?q=*%3A*)
> > > you have
> > > > > > to authenticate to it using the browser's basic auth prompt. Which
> > is
> > > > > > slightly annoying since the query page in the Admin UI generates
> > > links to
> > > > > > it for the queries you run, and you've already authenticated to get
> > > there.
> > > > > > But it's not a massive burden or anything... I guess I just
> > preferred
> > > > > > having the browser BA prompt.
> > > > > >
> > > > > > Thanks
> > >
> >
Reply | Threaded
Open this post in threaded view
|

Re: Intermittent error 401 with JSON Facet query to retrieve count all collections

Zheng Lin Edwin Yeo
Hi Jason,

Yes. I am using the latest Solr 8.1.1.

The query which I'm using is the JSON Facet query which I faced the error
initially.

Regards,
Edwin

On Tue, 4 Jun 2019 at 20:15, Jason Gerlowski <[hidden email]> wrote:

> Hi Edwin,
>
> Thanks for the additional datapoint.  It seemed to work for me, but we
> don't really understand the problem yet, so maybe it's not a solid
> work around like I'd hoped.  I'm curious to hear whether it works for
> Colvin.
>
> To double check though: forwardCredentials is only supported in Solr >
> 8.0.  You're using an 8.x version, right?
>
> Jason
>
> On Tue, Jun 4, 2019 at 2:45 AM Zheng Lin Edwin Yeo <[hidden email]>
> wrote:
> >
> > Hi Jason,
> >
> > Thanks for your reply.
> >
> > I have tried to add the "forwardCredentials": true in the security.json,
> > but I still get the same error.
> >
> > Regards,
> > Edwin
> >
> > On Mon, 3 Jun 2019 at 22:19, Colvin Cowie <[hidden email]>
> > wrote:
> >
> > > Hi, thanks I'll give that a go when I get a chance.
> > >
> > > I was trying to reply to an older thread (
> > >
> > >
> http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201904.mbox/%3CCAF2DzVXeVZqnixnkbzw0La1ui5N5-RG9PwfMBHG9vmkfBSMzJA%40mail.gmail.com%3E
> > > ),
> > > which I don't have in my mailbox, so obviously didn't reply to the
> right
> > > address to get my response threaded so mine has appeared on its own.
> Oops.
> > >
> > > A JIRA issue was raised on that thread
> > > https://issues.apache.org/jira/browse/SOLR-13421 but it's not had any
> > > attention.
> > >
> > >
> > > On Mon, 3 Jun 2019 at 14:46, Jason Gerlowski <[hidden email]>
> > > wrote:
> > >
> > > > Hi Colvin,
> > > >
> > > > We're still taking a look at fixing the bug, but as a workaround in
> > > > the meantime, you can look into adding a "forwardCredentials":true
> > > > property under the "authentication" section of security.json.  That
> > > > seems to fix the issue in my reproduction at least.
> > > >
> > > > e.g.
> > > >
> > > > {
> > > >     "authentication": {
> > > >         "blockUnknown": true,
> > > >         "class": "solr.BasicAuthPlugin",
> > > >         "credentials": {
> > > >             "solradmin": "<encoded-pw>"
> > > >         },
> > > >         "forwardCredentials": true
> > > >     },
> > > >     ...
> > > > }
> > > >
> > > > Jason
> > > >
> > > > On Mon, Jun 3, 2019 at 9:31 AM Jason Gerlowski <
> [hidden email]>
> > > > wrote:
> > > > >
> > > > > One last note: as far as I can tell, nothing about this issue is
> > > > > specific to JSON Faceting or the JSON request API.  It can be
> > > > > triggered just as easily with "/select?q=*:*".
> > > > >
> > > > > The bug created for this is: SOLR-13510
> > > > >
> > > > > On Mon, Jun 3, 2019 at 9:17 AM Jason Gerlowski <
> [hidden email]>
> > > > wrote:
> > > > > >
> > > > > > I'm also able to reproduce this bug on master.  A few more notes
> > > about
> > > > > > the bad behavior:
> > > > > >
> > > > > > - the behavior occurs regardless of the specific permissions
> > > > > > configured in security.json.  (i.e. whether the top permission is
> > > > > > "all", or "security-edit", or there are no permissions at all.)
> > > > > > - I tried looking for a pattern in which requests saw the 401s,
> but
> > > > > > didn't have any luck.  The 401 occurs when talking to the whole
> > > > > > collection or targeting individual cores directly.  It occurs
> when
> > > > > > curl hits a host containing a replica for the collection in
> question,
> > > > > > and when it doesnt. etc.  This distinguishes it from SOLR-13472,
> > > which
> > > > > > seems more specific to collection structure/layout.
> > > > > >
> > > > > > I'll create a bug for this in JIRA.
> > > > > >
> > > > > > On Sun, Jun 2, 2019 at 9:53 AM Colvin Cowie <
> > > > [hidden email]> wrote:
> > > > > > >
> > > > > > > Hello. I encountered this issue too and wrote this up before I
> > > found
> > > > this
> > > > > > > thread, but I thought I might as well post it still, if it
> helps...
> > > > > > >
> > > > > > > Currently I'm trying to move our product on to Solr 8.1.1. We
> are
> > > > currently
> > > > > > > using 6.6.6, so things have definitely moved on.
> > > > > > >
> > > > > > > We use the BasicAuthPlugin + RuleBasedAuthorizationPlugin to
> lock
> > > > down Solr
> > > > > > > (and we also secure our zookeeper). Here's an example for
> solradmin
> > > > as the
> > > > > > > user and password
> > > > > > >
> > > > > > > {
> > > > > > >     "authentication": {
> > > > > > >         "blockUnknown": true,
> > > > > > >         "class": "solr.BasicAuthPlugin",
> > > > > > >         "credentials": {
> > > > > > >             "solradmin":
> > > > "PIWZwkGnEKxKnqUs3X08xmbmYBaYyAeP3FiKp7fmeHc=
> > > > > > > Lnbp6bEbE7Ap8lXvQDKkUX2Xw53QDgP6Ae8QRT0P5/A="
> > > > > > >         }
> > > > > > >     },
> > > > > > >     "authorization": {
> > > > > > >         "class": "solr.RuleBasedAuthorizationPlugin",
> > > > > > >         "permissions": [
> > > > > > >             {
> > > > > > >                 "name": "all",
> > > > > > >                 "role": "admin"
> > > > > > >             }
> > > > > > >         ],
> > > > > > >         "user-role": {
> > > > > > >             "solradmin": "admin"
> > > > > > >         }
> > > > > > >     }
> > > > > > > }
> > > > > > >
> > > > > > >
> > > > > > > On Solr 8.1.1, using our previously working security.json,
> running
> > > > queries
> > > > > > > (through the admin UI currently) I non-deterministically get
> 401
> > > > responses
> > > > > > > on queries when a collection has more than 1 shard. Increasing
> the
> > > > number
> > > > > > > of shards in the collection makes the errors more likely.
> > > > > > >
> > > > > > > {
> > > > > > >   "responseHeader":{
> > > > > > >     "zkConnected":true,
> > > > > > >     "status":401,
> > > > > > >     "QTime":30,
> > > > > > >     "params":{
> > > > > > >       "q":"*:*",
> > > > > > >       "_":"1559474550365"}},
> > > > > > >   "error":{
> > > > > > >     "metadata":[
> > > > > > >
> > > > > > >
> > > >
> > >
> "error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException",
> > > > > > >
> > > > > > >
> > > >
> > >
> "root-error-class","org.apache.solr.client.solrj.impl.BaseHttpSolrClient$RemoteSolrException"],
> > > > > > >     "msg":"Error from server at null: Expected mime type
> > > > > > > application/octet-stream but got text/html.
> <html>\n<head>\n<meta
> > > > > > > http-equiv=\"Content-Type\"
> > > > > > > content=\"text/html;charset=utf-8\"/>\n<title>Error 401 require
> > > > > > > authentication</title>\n</head>\n<body><h2>HTTP ERROR
> > > > 401</h2>\n<p>Problem
> > > > > > > accessing /solr/gettingstarted_shard4_replica_n6/select.
> > > > Reason:\n<pre>
> > > > > > >  require authentication</pre></p>\n</body>\n</html>\n",
> > > > > > >     "code":401}}
> > > > > > >
> > > > > > > The security stats indicate this is happening because the
> requests
> > > > do not
> > > > > > > have credentials with them, e.g.
> > > > > > >
> > > >
> > >
> http://localhost:8983/solr/#/gettingstarted_shard4_replica_n6/plugins?type=security&entry=org.apache.solr.security.BasicAuthPlugin
> > > > > > >
> > > > > > >  org.apache.solr.security.BasicAuthPlugin
> > > > > > >     class:
> > > > > > >         org.apache.solr.security.BasicAuthPlugin
> > > > > > >     description:
> > > > > > >         Authentication Plugin
> > > > org.apache.solr.security.BasicAuthPlugin
> > > > > > >     stats
> > > > > > >         SECURITY./authentication.authenticated:
> > > > > > >             182
> > > > > > >         SECURITY./authentication.errors.count:
> > > > > > >             0
> > > > > > >         SECURITY./authentication.failMissingCredentials:
> > > > > > >             58
> > > > > > >         SECURITY./authentication.failWrongCredentials:
> > > > > > >             0
> > > > > > >         SECURITY./authentication.passThrough:
> > > > > > >             0
> > > > > > >         SECURITY./authentication.requestTimes.meanRate:
> > > > > > >             0.4183414110946125
> > > > > > >         SECURITY./authentication.requests:
> > > > > > >             240
> > > > > > >         SECURITY./authentication.totalTime:
> > > > > > >             117791100
> > > > > > >
> > > > > > > I assume that this is connected to the changes around
> > > > > > > https://issues.apache.org/jira/browse/SOLR-7896 and
> > > > > > > https://issues.apache.org/jira/browse/SOLR-13344 I've tested
> with
> > > > Solr
> > > > > > > 7.6.0 and it appears to be unaffected
> > > > > > >
> > > > > > > Repro steps:
> > > > > > >    # Extract solr 8.1.1.
> > > > > > >    # bin\solr start -e cloud
> > > > > > >         1 node / [default port] / [default collection name] / 4
> > > > shards / 1
> > > > > > > replica / [_default configuration]
> > > > > > >    # server\scripts\cloud-scripts\zkcli -zkhost localhost:9983
> -cmd
> > > > putfile
> > > > > > > /security.json <example-security.json file with content from
> > > example
> > > > above>
> > > > > > >
> > > > > > >    # Execute repeated GETS to
> > > > > > > http://localhost:8983/solr/gettingstarted/select?q=*%3A* - a
> lot
> > > of
> > > > them,
> > > > > > > but not all, will fail with 401s
> > > > > > >
> > > > > > >
> > > > > > > Also as a side note, because the authentication is now done
> through
> > > > the
> > > > > > > form login rather than the browser basic auth, if you go
> directly
> > > to
> > > > a non
> > > > > > > UI url (e.g.
> http://localhost:8983/solr/main_index/select?q=*%3A*)
> > > > you have
> > > > > > > to authenticate to it using the browser's basic auth prompt.
> Which
> > > is
> > > > > > > slightly annoying since the query page in the Admin UI
> generates
> > > > links to
> > > > > > > it for the queries you run, and you've already authenticated
> to get
> > > > there.
> > > > > > > But it's not a massive burden or anything... I guess I just
> > > preferred
> > > > > > > having the browser BA prompt.
> > > > > > >
> > > > > > > Thanks
> > > >
> > >
>