Issue Using Solr 5.3 Authentication and Authorization Plugins

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Issue Using Solr 5.3 Authentication and Authorization Plugins

Dan Davis-2
SOLR-8004 also appears to work to me.   I manually edited security.json and
did putfile.   I didn't bother with browse permission, because it was
Kevin's workaround.    solr-5.3.1-SNAPSHOT did challenge me for credentials
when going to curl
http://localhost:8983/solr/admin/collections?action=CREATE and so on...

On Thu, Sep 10, 2015 at 11:10 PM, Dan Davis <[hidden email]> wrote:

> Kevin & Noble,
>
> I've manually verified the fix for SOLR-8000, but not yet for SOLR-8004.
>
> I reproduced the initial problem with reloading security.json after
> restarting both Solr and ZooKeeper.   I verified using zkcli.sh that
> ZooKeeper does retain the changes to the file after using
> /solr/admin/authorization, and that therefore the problem was Solr.
>
> After building solr-5.3.1-SNAPSHOT.tgz with ant package (because I don't
> know how to give parameters to ant server), I expanded it, copied in the
> core data, and then started it.   I was prompted for a password, and it let
> me in once the password was given.
>
> I'll probably get to SOLR-8004 shortly, since I have both environments
> built and working.
>
> It also occurs to me that it might be better to forbid all permissions and
> grant specific permissions to specific roles.   Is there a comprehensive
> list of the permissions available?
>
>
> On Tue, Sep 8, 2015 at 1:07 PM, Kevin Lee <[hidden email]>
> wrote:
>
>> Thanks Dan!  Please let us know what you find.  I’m interested to know if
>> this is an issue with anyone else’s setup or if I have an issue in my local
>> configuration that is still preventing it to work on start/restart.
>>
>> - Kevin
>>
>> > On Sep 5, 2015, at 8:45 AM, Dan Davis <[hidden email]> wrote:
>> >
>> > Kevin & Noble,
>> >
>> > I'll take it on to test this.   I've built from source before, and I've
>> > wanted this authorization capability for awhile.
>> >
>> > On Fri, Sep 4, 2015 at 9:59 AM, Kevin Lee <[hidden email]>
>> wrote:
>> >
>> >> Noble,
>> >>
>> >> Does SOLR-8000 need to be re-opened?  Has anyone else been able to test
>> >> the restart fix?
>> >>
>> >> At startup, these are the log messages that say there is no security
>> >> configuration and the plugins aren’t being used even though
>> security.json
>> >> is in Zookeeper:
>> >> 2015-09-04 08:06:21.205 INFO  (main) [   ] o.a.s.c.CoreContainer
>> Security
>> >> conf doesn't exist. Skipping setup for authorization module.
>> >> 2015-09-04 08:06:21.205 INFO  (main) [   ] o.a.s.c.CoreContainer No
>> >> authentication plugin used.
>> >>
>> >> Thanks,
>> >> Kevin
>> >>
>> >>> On Sep 4, 2015, at 5:47 AM, Noble Paul <[hidden email]> wrote:
>> >>>
>> >>> There are no download links for 5.3.x branch  till we do a bug fix
>> >> release
>> >>>
>> >>> If you wish to download the trunk nightly (which is not same as 5.3.0)
>> >>> check here
>> >>
>> https://builds.apache.org/job/Solr-Artifacts-trunk/lastSuccessfulBuild/artifact/solr/package/
>> >>>
>> >>> If you wish to get the binaries for 5.3 branch you will have to make
>> it
>> >>> (you will need to install svn and ant)
>> >>>
>> >>> Here are the steps
>> >>>
>> >>> svn checkout
>> >> http://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_5_3/
>> >>> cd lucene_solr_5_3/solr
>> >>> ant server
>> >>>
>> >>>
>> >>>
>> >>> On Fri, Sep 4, 2015 at 4:11 PM, davidphilip cherian
>> >>> <[hidden email]> wrote:
>> >>>> Hi Kevin/Noble,
>> >>>>
>> >>>> What is the download link to take the latest? What are the steps to
>> >> compile
>> >>>> it, test and use?
>> >>>> We also have a use case to have this feature in solr too. Therefore,
>> >> wanted
>> >>>> to test and above info would help a lot to get started.
>> >>>>
>> >>>> Thanks.
>> >>>>
>> >>>>
>> >>>> On Fri, Sep 4, 2015 at 1:45 PM, Kevin Lee <[hidden email]
>> >
>> >> wrote:
>> >>>>
>> >>>>> Thanks, I downloaded the source and compiled it and replaced the jar
>> >> file
>> >>>>> in the dist and solr-webapp’s WEB-INF/lib directory.  It does seem
>> to
>> >> be
>> >>>>> protecting the Collections API reload command now as long as I
>> upload
>> >> the
>> >>>>> security.json after startup of the Solr instances.  If I shutdown
>> and
>> >> bring
>> >>>>> the instances back up, the security is no longer in place and I
>> have to
>> >>>>> upload the security.json again for it to take effect.
>> >>>>>
>> >>>>> - Kevin
>> >>>>>
>> >>>>>> On Sep 3, 2015, at 10:29 PM, Noble Paul <[hidden email]>
>> wrote:
>> >>>>>>
>> >>>>>> Both these are committed. If you could test with the latest 5.3
>> branch
>> >>>>>> it would be helpful
>> >>>>>>
>> >>>>>> On Wed, Sep 2, 2015 at 5:11 PM, Noble Paul <[hidden email]>
>> >> wrote:
>> >>>>>>> I opened a ticket for the same
>> >>>>>>> https://issues.apache.org/jira/browse/SOLR-8004
>> >>>>>>>
>> >>>>>>> On Wed, Sep 2, 2015 at 1:36 PM, Kevin Lee
>> <[hidden email]
>> >>>
>> >>>>> wrote:
>> >>>>>>>> I’ve found that completely exiting Chrome or Firefox and opening
>> it
>> >>>>> back up re-prompts for credentials when they are required.  It was
>> >>>>> re-prompting with the /browse path where authentication was working
>> >> each
>> >>>>> time I completely exited and started the browser again, however it
>> >> won’t
>> >>>>> re-prompt unless you exit completely and close all running instances
>> >> so I
>> >>>>> closed all instances each time to test.
>> >>>>>>>>
>> >>>>>>>> However, to make sure I ran it via the command line via curl as
>> >>>>> suggested and it still does not give any authentication error when
>> >> trying
>> >>>>> to issue the command via curl.  I get a success response from all
>> the
>> >> Solr
>> >>>>> instances that the reload was successful.
>> >>>>>>>>
>> >>>>>>>> Not sure why the pre-canned permissions aren’t working, but the
>> one
>> >> to
>> >>>>> the request handler at the /browse path is.
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>> On Sep 1, 2015, at 11:03 PM, Noble Paul <[hidden email]>
>> >> wrote:
>> >>>>>>>>>
>> >>>>>>>>> " However, after uploading the new security.json and restarting
>> the
>> >>>>>>>>> web browser,"
>> >>>>>>>>>
>> >>>>>>>>> The browser remembers your login , So it is unlikely to prompt
>> for
>> >> the
>> >>>>>>>>> credentials again.
>> >>>>>>>>>
>> >>>>>>>>> Why don't you try the RELOAD operation using command line
>> (curl) ?
>> >>>>>>>>>
>> >>>>>>>>> On Tue, Sep 1, 2015 at 10:31 PM, Kevin Lee
>> >> <[hidden email]>
>> >>>>> wrote:
>> >>>>>>>>>> The restart issues aside, I’m trying to lockdown usage of the
>> >>>>> Collections API, but that also does not seem to be working either.
>> >>>>>>>>>>
>> >>>>>>>>>> Here is my security.json.  I’m using the
>> “collection-admin-edit”
>> >>>>> permission and assigning it to the “adminRole”.  However, after
>> >> uploading
>> >>>>> the new security.json and restarting the web browser, it doesn’t
>> seem
>> >> to be
>> >>>>> requiring credentials when calling the RELOAD action on the
>> Collections
>> >>>>> API.  The only thing that seems to work is the custom permission
>> >> “browse”
>> >>>>> which is requiring authentication before allowing me to pull up the
>> >> page.
>> >>>>> Am I using the permissions correctly for the
>> >> RuleBasedAuthorizationPlugin?
>> >>>>>>>>>>
>> >>>>>>>>>> {
>> >>>>>>>>>>     "authentication":{
>> >>>>>>>>>>        "class":"solr.BasicAuthPlugin",
>> >>>>>>>>>>        "credentials": {
>> >>>>>>>>>>                     "admin”:”<pass> <salt>",
>> >>>>>>>>>>                     "user": ”<pass> <salt>"
>> >>>>>>>>>>             }
>> >>>>>>>>>>     },
>> >>>>>>>>>>     "authorization":{
>> >>>>>>>>>>        "class":"solr.RuleBasedAuthorizationPlugin",
>> >>>>>>>>>>        "permissions": [
>> >>>>>>>>>>                     {
>> >>>>>>>>>>                             "name":"security-edit",
>> >>>>>>>>>>                             "role":"adminRole"
>> >>>>>>>>>>                     },
>> >>>>>>>>>>                     {
>> >>>>>>>>>>                             "name":"collection-admin-edit”,
>> >>>>>>>>>>                             "role":"adminRole"
>> >>>>>>>>>>                     },
>> >>>>>>>>>>                     {
>> >>>>>>>>>>                             "name":"browse",
>> >>>>>>>>>>                             "collection": "inventory",
>> >>>>>>>>>>                             "path": "/browse",
>> >>>>>>>>>>                             "role":"browseRole"
>> >>>>>>>>>>                     }
>> >>>>>>>>>>             ],
>> >>>>>>>>>>        "user-role": {
>> >>>>>>>>>>                     "admin": [
>> >>>>>>>>>>                             "adminRole",
>> >>>>>>>>>>                             "browseRole"
>> >>>>>>>>>>                     ],
>> >>>>>>>>>>                     "user": [
>> >>>>>>>>>>                             "browseRole"
>> >>>>>>>>>>                     ]
>> >>>>>>>>>>             }
>> >>>>>>>>>>     }
>> >>>>>>>>>> }
>> >>>>>>>>>>
>> >>>>>>>>>> Also tried adding the permission using the Authorization API,
>> but
>> >> no
>> >>>>> effect, still isn’t protecting the Collections API from being
>> invoked
>> >>>>> without a username password.  I do see in the Solr logs that it sees
>> >> the
>> >>>>> updates because it outputs the messages “Updating /security.json …”,
>> >>>>> “Security node changed”, “Initializing authorization plugin:
>> >>>>> solr.RuleBasedAuthorizationPlugin” and “Authentication plugin class
>> >>>>> obtained from ZK: solr.BasicAuthPlugin”.
>> >>>>>>>>>>
>> >>>>>>>>>> Thanks,
>> >>>>>>>>>> Kevin
>> >>>>>>>>>>
>> >>>>>>>>>>> On Sep 1, 2015, at 12:31 AM, Noble Paul <[hidden email]
>> >
>> >>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>> I'm investigating why restarts or first time start does not
>> read
>> >> the
>> >>>>>>>>>>> security.json
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Tue, Sep 1, 2015 at 1:00 PM, Noble Paul <
>> [hidden email]
>> >>>
>> >>>>> wrote:
>> >>>>>>>>>>>> I removed that statement
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> "If activating the authorization plugin doesn't protect the
>> >> admin
>> >>>>> ui,
>> >>>>>>>>>>>> how does one protect access to it?"
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> One does not need to protect the admin UI. You only need to
>> >> protect
>> >>>>>>>>>>>> the relevant API calls . I mean it's OK to not protect the
>> CSS
>> >> and
>> >>>>>>>>>>>> HTML stuff.  But if you perform an action to create a core or
>> >> do a
>> >>>>>>>>>>>> query through admin UI , it automatically will prompt you for
>> >>>>>>>>>>>> credentials (if those APIs are protected)
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Tue, Sep 1, 2015 at 12:41 PM, Kevin Lee
>> >>>>> <[hidden email]> wrote:
>> >>>>>>>>>>>>> Thanks for the clarification!
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> So is the wiki page incorrect at
>> >>>>>>>>>>>>>
>> >>>>>
>> >>
>> https://cwiki.apache.org/confluence/display/solr/Basic+Authentication+Plugin
>> >>>>> which says that the admin ui will require authentication once the
>> >>>>> authorization plugin is activated?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> "An authorization plugin is also available to configure Solr
>> >> with
>> >>>>> permissions to perform various activities in the system. Once
>> >> activated,
>> >>>>> access to the Solr Admin UI and all requests will need to be
>> >> authenticated
>> >>>>> and users will be required to have the proper authorization for all
>> >>>>> requests, including using the Admin UI and making any API calls."
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> If activating the authorization plugin doesn't protect the
>> >> admin
>> >>>>> ui, how does one protect access to it?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Also, the issue I'm having is not just at restart.
>> According
>> >> to
>> >>>>> the docs security.json should be uploaded to Zookeeper before
>> starting
>> >> any
>> >>>>> of the Solr instances.  However, I tried to upload security.json
>> before
>> >>>>> starting any of the Solr instances, but it would not pick up the
>> >> security
>> >>>>> config until after the Solr instances are already running and then
>> >>>>> uploading the security.json again.  I can see in the logs at startup
>> >> that
>> >>>>> the Solr instances don't see any plugin enabled even though
>> >> security.json
>> >>>>> is already in zookeeper and then after they are started and the
>> >>>>> security.json is uploaded again I see it reconfigure to use the
>> plugin.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>> Kevin
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Aug 31, 2015, at 11:22 PM, Noble Paul <
>> >> [hidden email]>
>> >>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Admin UI is not protected by any of these permissions.
>> Only if
>> >>>>> you try
>> >>>>>>>>>>>>>> to perform a protected operation , it asks for a password.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> I'll investigate the restart problem and report my
>> findings
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Tue, Sep 1, 2015 at 3:10 AM, Kevin Lee
>> >>>>> <[hidden email]> wrote:
>> >>>>>>>>>>>>>>> Anyone else running into any issues trying to get the
>> >>>>> authentication and authorization plugins in 5.3 working?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On Aug 29, 2015, at 2:30 AM, Kevin Lee
>> >>>>> <[hidden email]> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Hi,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I’m trying to use the new basic auth plugin for Solr 5.3
>> and
>> >>>>> it doesn’t seem to be working quite right.  Not sure if I’m missing
>> >> steps
>> >>>>> or there is a bug.  I am able to get it to protect access to a URL
>> >> under a
>> >>>>> collection, but am unable to get it to secure access to the Admin
>> UI.
>> >> In
>> >>>>> addition, after stopping the Solr and Zookeeper instances, the
>> >>>>> security.json is still in Zookeeper, however Solr is allowing
>> access to
>> >>>>> everything again like the security configuration isn’t in place.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Contents of security.json taken from wiki page, but
>> edited
>> >> to
>> >>>>> produce valid JSON.  Had to move comma after 3rd from last “}” up to
>> >> just
>> >>>>> after the last “]”.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> {
>> >>>>>>>>>>>>>>>> "authentication":{
>> >>>>>>>>>>>>>>>> "class":"solr.BasicAuthPlugin",
>> >>>>>>>>>>>>>>>>
>> >>>>> "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0=
>> >>>>> Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
>> >>>>>>>>>>>>>>>> },
>> >>>>>>>>>>>>>>>> "authorization":{
>> >>>>>>>>>>>>>>>> "class":"solr.RuleBasedAuthorizationPlugin",
>> >>>>>>>>>>>>>>>> "permissions":[{"name":"security-edit",
>> >>>>>>>>>>>>>>>> "role":"admin"}],
>> >>>>>>>>>>>>>>>> "user-role":{"solr":"admin"}
>> >>>>>>>>>>>>>>>> }}
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Here are the steps I followed:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Upload security.json to zookeeper
>> >>>>>>>>>>>>>>>> ./zkcli.sh -z
>> localhost:2181,localhost:2182,localhost:2183
>> >>>>> -cmd putfile /security.json ~/solr/security.json
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Use zkCli.sh from Zookeeper to ensure the security.json
>> is
>> >> in
>> >>>>> Zookeeper at /security.json.  It is there and looks like what was
>> >>>>> originally uploaded.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Start Solr Instances
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Attempt to create a permission, however get the following
>> >>>>> error:
>> >>>>>>>>>>>>>>>> {
>> >>>>>>>>>>>>>>>> "responseHeader":{
>> >>>>>>>>>>>>>>>> "status":400,
>> >>>>>>>>>>>>>>>> "QTime":0},
>> >>>>>>>>>>>>>>>> "error":{
>> >>>>>>>>>>>>>>>> "msg":"No authorization plugin configured",
>> >>>>>>>>>>>>>>>> "code":400}}
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Upload security.json again.
>> >>>>>>>>>>>>>>>> ./zkcli.sh -z
>> localhost:2181,localhost:2182,localhost:2183
>> >>>>> -cmd putfile /security.json ~/solr/security.json
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Issue the following to try to create the permission again
>> >> and
>> >>>>> this time it’s successful.
>> >>>>>>>>>>>>>>>> // Create a permission for mysearch endpoint
>> >>>>>>>>>>>>>>>>     curl --user solr:SolrRocks -H
>> >>>>> 'Content-type:application/json' -d '{"set-permission":
>> >>>>> {"name":"mycollection-search","collection":
>> >>>>> “mycollection","path":”/mysearch","role": "search-user"}}'
>> >>>>> http://localhost:8983/solr/admin/authorization
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> {
>> >>>>>>>>>>>>>>>> "responseHeader":{
>> >>>>>>>>>>>>>>>> "status":0,
>> >>>>>>>>>>>>>>>> "QTime":7}}
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Issue the following commands to add users
>> >>>>>>>>>>>>>>>> curl --user solr:SolrRocks
>> >>>>> http://localhost:8983/solr/admin/authentication -H
>> >>>>> 'Content-type:application/json' -d '{"set-user": {"admin" :
>> “password"
>> >> }}’
>> >>>>>>>>>>>>>>>> curl --user solr:SolrRocks
>> >>>>> http://localhost:8983/solr/admin/authentication -H
>> >>>>> 'Content-type:application/json' -d '{"set-user": {"user" :
>> “password"
>> >> }}'
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Issue the following command to add permission to users
>> >>>>>>>>>>>>>>>> curl -u solr:SolrRocks -H
>> 'Content-type:application/json' -d
>> >>>>> '{ "set-user-role" : {"admin": ["search-user", "admin"]}}'
>> >>>>> http://localhost:8983/solr/admin/authorization
>> >>>>>>>>>>>>>>>> curl -u solr:SolrRocks -H
>> 'Content-type:application/json' -d
>> >>>>> '{ "set-user-role" : {"user": ["search-user"]}}'
>> >>>>> http://localhost:8983/solr/admin/authorization
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> After executing the above, access to /mysearch is
>> protected
>> >>>>> until I restart the Solr and Zookeeper instances.  However, the
>> admin
>> >> UI is
>> >>>>> never protected like the Wiki page says it should be once activated.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>
>> >>
>> https://cwiki.apache.org/confluence/display/solr/Rule-Based+Authorization+Plugin
>> >>>>> <
>> >>>>>
>> >>
>> https://cwiki.apache.org/confluence/display/solr/Rule-Based+Authorization+Plugin
>> >>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Why does the authentication and authorization plugin not
>> >> stay
>> >>>>> activated after restart and why is the Admin UI never protected?
>> Am I
>> >>>>> missing any steps?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Thanks,
>> >>>>>>>>>>>>>>>> Kevin
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> --
>> >>>>>>>>>>>>>> -----------------------------------------------------
>> >>>>>>>>>>>>>> Noble Paul
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> -----------------------------------------------------
>> >>>>>>>>>>>> Noble Paul
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>> --
>> >>>>>>>>>>> -----------------------------------------------------
>> >>>>>>>>>>> Noble Paul
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> -----------------------------------------------------
>> >>>>>>>>> Noble Paul
>> >>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> --
>> >>>>>>> -----------------------------------------------------
>> >>>>>>> Noble Paul
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> -----------------------------------------------------
>> >>>>>> Noble Paul
>> >>>>>
>> >>>>>
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> -----------------------------------------------------
>> >>> Noble Paul
>> >>
>> >>
>>
>>
>
12