Possible to HTTP POST (very long) queries?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Possible to HTTP POST (very long) queries?

Brian Lucas
I'm attempting to send a very long / large query to Solr and I'm
encountering an issue under Jetty for this query that I'm passing.  It's
more than likely hitting a hard-coded somewhere for the maximum length of
the URL that was designed as a safety measure.   I'm not sure if Tomcat
would fare differently.  

 

java.io.IOException: FULL

     at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:249)

 

There's no need to post the query, as it's long and esoteric (probably 3,000
or so characters - I'm sure that's a bad idea as a passed URL, hence the
following question).

 

Predictably, people might suggest reducing the size of the query but there
are unavoidable reasons that this query is extremely large (such as
satisfying certain criteria).  

 

I'm wondering if it's possible to run a POST instead of a GET when querying
Solr, or does this need to be custom coded?  Seems like this might be a way
to get around it.

 

Thanks,

Brian

Reply | Threaded
Open this post in threaded view
|

Re: Possible to HTTP POST (very long) queries?

Chris Hostetter-3

: I'm wondering if it's possible to run a POST instead of a GET when querying
: Solr, or does this need to be custom coded?  Seems like this might be a way
: to get around it.

Yep, Yonik fixed this arround April 7th, Jira's down right now, but it was
tracked in SOLR-7...

        http://issues.apache.org/jira/browse/SOLR-7




-Hoss

Reply | Threaded
Open this post in threaded view
|

RE: Possible to HTTP POST (very long) queries?

Brian Lucas
Chris, thanks for the reply.  I have an easy question:  What's the syntax?  

Changing "GET" to "POST" in the form gives me:

Apr 19, 2006 6:37:05 PM org.apache.solr.core.SolrException log
SEVERE: org.xmlpull.v1.XmlPullParserException: only whitespace content
allowed before start tag and not q (position: START_DOCUMENT seen q... @1:1)

     at org.xmlpull.mxp1.MXParser.parseProlog(MXParser.java:1519)
 
Should I be doing an XML POST with the <select><q>...</q></select> syntax
instead?

Thanks, Brian

-----Original Message-----
From: Chris Hostetter [mailto:[hidden email]]
Sent: Tuesday, April 18, 2006 10:49 PM
To: [hidden email]
Subject: Re: Possible to HTTP POST (very long) queries?


: I'm wondering if it's possible to run a POST instead of a GET when
querying
: Solr, or does this need to be custom coded?  Seems like this might be a
way
: to get around it.

Yep, Yonik fixed this arround April 7th, Jira's down right now, but it was
tracked in SOLR-7...

        http://issues.apache.org/jira/browse/SOLR-7




-Hoss

Reply | Threaded
Open this post in threaded view
|

RE: Possible to HTTP POST (very long) queries?

Chris Hostetter-3

: Chris, thanks for the reply.  I have an easy question:  What's the syntax?
:
: Changing "GET" to "POST" in the form gives me:

No special syntax, just regular old CGI form encoding -- just changing the
GET to a POST in teh form should work okay.  Are you sure you are using a
nightly build from after April 7th?  The exception you are getting looks
like you are using an older build where all POSTs were interpreted as
updates regardless of the URL.

NOTE: if you are using Jetty, which the example setup does, there apears
to be a slight bug because Jetty handles "empty" query params differnetly
between GET and POST;  I'll file an issue to track this as soon as Jira
wakes up -- but the work arround is to make sure you don't include any
options you don't need when you POST your query (ie: "stylesheet")


-Hoss

Reply | Threaded
Open this post in threaded view
|

RE: Possible to HTTP POST (very long) queries?

Brian Lucas
Hi Chris,

Two things which may not necessarily be related:

1. Regarding HTTP POST, I dropped in the 'start.jar' from the 04-19-2006
build and still have the same error.  Something as simple as this example
should work, right?
----------
<HTML>
<BODY>
<form method="POST" action="http://localhost:8080/solr/select">
<input type="text" name="q" value="story_id:1">
<input type="submit" value="search">
</form>
</BODY>
</HTML>
---------

Still getting the same error:

<result status="1">
org.xmlpull.v1.XmlPullParserException: only whitespace content allowed
before start tag and not q (position: START_DOCUMENT seen q... @1:1)


2. I can no longer 'ant compile'/'dist' the newest version(s) (04/16/06 -
04/19/06) of Solr due to the error:    
-------
[javac]
/home/root/disks/src/solr-nightly/src/java/org/apache/solr/util/AbstractSolr
TestCase.java:8: package junit.framework does not exist
    [javac] import junit.framework.TestCase;
...
    [javac] 6 errors
    [javac] 4 warnings
-------
As well as a few other errors (with Sun's Java SDK 1.5.0_6).  I previously
ran 'ant compile'/'dist' on 04-07-2006 just fine  -- am I missing any extra
framework?  I don't see this mentioned anywhere on the list so I'm guessing
it's a problem on my end.

Sorry if I'm missing something obvious here.

Thanks again.

Brian


-----Original Message-----
From: Chris Hostetter [mailto:[hidden email]]
Sent: Wednesday, April 19, 2006 1:03 PM
To: [hidden email]
Subject: RE: Possible to HTTP POST (very long) queries?


: Chris, thanks for the reply.  I have an easy question:  What's the syntax?
:
: Changing "GET" to "POST" in the form gives me:

No special syntax, just regular old CGI form encoding -- just changing the
GET to a POST in teh form should work okay.  Are you sure you are using a
nightly build from after April 7th?  The exception you are getting looks
like you are using an older build where all POSTs were interpreted as
updates regardless of the URL.

NOTE: if you are using Jetty, which the example setup does, there apears
to be a slight bug because Jetty handles "empty" query params differnetly
between GET and POST;  I'll file an issue to track this as soon as Jira
wakes up -- but the work arround is to make sure you don't include any
options you don't need when you POST your query (ie: "stylesheet")


-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: Possible to HTTP POST (very long) queries?

Yonik Seeley
On 4/19/06, Brian Lucas <[hidden email]> wrote:
> 1. Regarding HTTP POST, I dropped in the 'start.jar' from the 04-19-2006
> build and still have the same error.

The start.jar is from the jetty appserver.  You want a new solr.war in
the webapps directory.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: Possible to HTTP POST (very long) queries?

Mike Baranczak-2
In reply to this post by Brian Lucas
On Apr 19, 2006, at 4:10 PM, Brian Lucas wrote:

>
> 2. I can no longer 'ant compile'/'dist' the newest version(s)  
> (04/16/06 -
> 04/19/06) of Solr due to the error:
> -------
> [javac]
> /home/root/disks/src/solr-nightly/src/java/org/apache/solr/util/
> AbstractSolr
> TestCase.java:8: package junit.framework does not exist
>     [javac] import junit.framework.TestCase;
> ...


No, it's not just you. The build requires the JUnit library, which is  
not provided. I'm guessing that whoever put that in is using an IDE  
that automatically includes JUnit in the classpath. For now, you can  
download the jar from www.junit.org and drop it into lib/.

-MB

Reply | Threaded
Open this post in threaded view
|

RE: Possible to HTTP POST (very long) queries?

Brian Lucas
In reply to this post by Yonik Seeley
Chris/Yonik, running solr.war file through Tomcat (instead of the start.jar
Jetty file) fixes the problem.  I was running Jetty because I could very
easily run multiple instances just by changing the port in the jetty.xml
file.  Sounds like I could file a bug on this, but it is certainly a
fringe/uncommon case ("Search POST Problem in Jetty start.jar")?

Thanks, Brian


-----Original Message-----
From: Yonik Seeley [mailto:[hidden email]]
Sent: Wednesday, April 19, 2006 2:21 PM
To: [hidden email]
Subject: Re: Possible to HTTP POST (very long) queries?

On 4/19/06, Brian Lucas <[hidden email]> wrote:
> 1. Regarding HTTP POST, I dropped in the 'start.jar' from the 04-19-2006
> build and still have the same error.

The start.jar is from the jetty appserver.  You want a new solr.war in
the webapps directory.

-Yonik

Reply | Threaded
Open this post in threaded view
|

RE: Possible to HTTP POST (very long) queries?

Chris Hostetter-3

: Chris/Yonik, running solr.war file through Tomcat (instead of the start.jar
: Jetty file) fixes the problem.  I was running Jetty because I could very
: easily run multiple instances just by changing the port in the jetty.xml
: file.  Sounds like I could file a bug on this, but it is certainly a
: fringe/uncommon case ("Search POST Problem in Jetty start.jar")?

Yonik's already fixed the code so that it's more robust in case other
servlet containers have hte same behavior.  The fix ss available in
subversion now, and will be in tonight's nightly build...

        http://issues.apache.org/jira/browse/SOLR-12




-Hoss

Reply | Threaded
Open this post in threaded view
|

Re: Possible to HTTP POST (very long) queries?

Chris Hostetter-3
In reply to this post by Mike Baranczak-2

: > TestCase.java:8: package junit.framework does not exist
: >     [javac] import junit.framework.TestCase;

: No, it's not just you. The build requires the JUnit library, which is

There has allways been a JUnit dependency, but I recently beefed up the
test suite coverage and made the change so that the "dist" target requires
the teests be compiled and run.

I'm sorry I didn't email an announcement about the change, but it had been
discussed on solr-dev in a few threads, and I assumed anyone who would be
building from source would be subscribed there.

: not provided. I'm guessing that whoever put that in is using an IDE
: that automatically includes JUnit in the classpath. For now, you can
: download the jar from www.junit.org and drop it into lib/.

Actually it's was a concious choice not to include junit.jar in the
solr/lib directly -- a lot of people smarter then me (mainly Erik
Hatcher) have said that the only *right* way to use JUnit with Ant is to
copy the junit.jar into your ANT_HOME/lib directory.  There are lots of
CLASSSPATH issues that you have to be wary of...

http://www.nabble.com/Re%3A-Contrib-in-oblivion-p1458269.html
http://www.nabble.com/Re%3A-inital-observations-from-a-newbie-p2970514.html
http://ant.apache.org/faq.html#delegating-classloader

-Hoss