How to log full URL using Jetty RequestLogHandler in Solr 7

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How to log full URL using Jetty RequestLogHandler in Solr 7

dimaf
After migration from Solr 6 to Solr 7, Jetty RequestLogHandler logs only the
path and parameters of URL instead of saving full URL as it does in Solr 6.

Solr 6:
127.0.0.1 - - [07/Nov/2018:09:34:27 -0700] "GET
//localhost:7777/solr/admin/collections?action=CLUSTERSTATUS&wt=json
HTTP/1.1" 200 3088

Solr 7:
127.0.0.1 - - [07/Nov/2018:08:36:13 -0700] "GET
/solr/admin/collections?action=CLUSTERSTATUS&wt=json HTTP/1.1" 200 820 "-"
"monitor/1.0.1"

On each host, several Solr instances are run and all of them save requests
into one file. I am not sure that it is the best practice...

So the question is how Jetty/Solr can be configured to log full URL?

Thanks!



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html
Reply | Threaded
Open this post in threaded view
|

Re: How to log full URL using Jetty RequestLogHandler in Solr 7

Shawn Heisey-2
On 11/7/2018 8:01 AM, dimaf wrote:
> After migration from Solr 6 to Solr 7, Jetty RequestLogHandler logs only the
> path and parameters of URL instead of saving full URL as it does in Solr 6.
<snip>
> So the question is how Jetty/Solr can be configured to log full URL?

That is 100 percent Jetty config.  No Solr code is involved.  Which
means that the expertise to answer your question might not be available
here.  I have checked a few different versions and the request log
config that is included in jetty.xml with the Solr project looks
identical between Solr 6.x and Solr 7.x.  It uses a Jetty class called
NCSARequestLog.  The request log config is commented out by default.

Various 6.x versions of Solr included various 9.3.x versions of Jetty. 
The Jetty version was changed to 9.4.x in version 7.3.0, and 6.x got
Jetty 9.4.x in version 6.6.3.

I have reached out to the #jetty IRC channel on this topic.  As of right
now, there has been no response.  Getting a response could take quite a
while.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: How to log full URL using Jetty RequestLogHandler in Solr 7

dimaf
Reply | Threaded
Open this post in threaded view
|

Re: How to log full URL using Jetty RequestLogHandler in Solr 7

Shawn Heisey-2
In reply to this post by Shawn Heisey-2
On 11/7/2018 8:58 AM, Shawn Heisey wrote:
> I have reached out to the #jetty IRC channel on this topic.  As of
> right now, there has been no response.  Getting a response could take
> quite a while.

I was told that the #jetty IRC channel is no longer an official support
avenue.  They have someone staying in the channel to inform people of
the change.  It was recommended to open an issue here:

https://github.com/eclipse/jetty.project/issues/new

I do not have the precise information about the Solr/Jetty versions
you're running, which will be needed for an issue.  Are you using the
request log config unchanged from what Solr includes in jetty.xml
(commented)?  Is that config the same on the 6.x version as it is on the
7.x version?

It might be a good idea to take the issue to their mailing list for
discussion before opening an issue.

I think it's likely that this change was intentional by the Jetty team
in their 9.4.x version.  The actual HTTP request that the client makes
will be precisely what your example from Solr 7.x contains ... which
means that the information logged by the 6.x version doesn't precisely
match the actual request.

Thanks,
Shawn

Reply | Threaded
Open this post in threaded view
|

Re: How to log full URL using Jetty RequestLogHandler in Solr 7

dimaf
Yes, jetty.xml looks the same for Solr 6 and Solr 7, no difference for the
log request section:

diff ~dbl/solr/solr-6.6.0/server/etc/jetty.xml
~dbl/solr/solr-7.4.0/server/etc/jetty.xml
45c45
<     <Set name="idleTimeout" type="int"><Property
name="solr.jetty.threads.idle.timeout" default="5000"/></Set>
---
>     <Set name="idleTimeout" type="int"><Property
> name="solr.jetty.threads.idle.timeout" default="120000"/></Set>
107a108,115
>        <Call name="addRule">
>          <Arg>
>            <New
> class="org.eclipse.jetty.rewrite.handler.RewritePatternRule">
>              <Set name="pattern">/api/*</Set>
>              <Set name="replacement">/solr/____v2</Set>
>            </New>
>          </Arg>
>        </Call>
154c162
<                 <Set name="extended">false</Set>
---
>                 <Set name="extended">true</Set>

Thanks,
Dmitry.



--
Sent from: http://lucene.472066.n3.nabble.com/Solr-User-f472068.html