size limitation when adding document?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

size limitation when adding document?

Andrew Tan-3
 
Hi ,

I am new to solr and just try it out. I copied solr.war (from 1.2.0
distribution) into tomcat 5.5.26's webapps directory and started tomcat.
Then I use the java SimplePostTool to add documents. when the document
is small, things are fine. However, when I tried to add document
(greater than 8KB) into solr server, I got the following error message :

<result status="1">java.io.EOFException: no more data available -
expected end tags &lt;/field&gt;&lt;/doc&gt;&lt;/add&gt; to close start
tag &lt;field&gt; from line 35 and start tag &lt;doc&gt; from line 3 and
start tag &lt;add&gt; from line 2, parser stopped on START_TAG seen
...uiror Termination Fee if the preceding relates to antitrust or com...
@35:859
        at org.xmlpull.mxp1.MXParser.fillBuf(MXParser.java:3015)
        at org.xmlpull.mxp1.MXParser.more(MXParser.java:3026)
        at org.xmlpull.mxp1.MXParser.nextImpl(MXParser.java:1384)
        at org.xmlpull.mxp1.MXParser.next(MXParser.java:1093)
        at org.xmlpull.mxp1.MXParser.nextText(MXParser.java:1058)
        at
org.apache.solr.handler.XmlUpdateRequestHandler.readDoc(XmlUpdateRequest
Handler.java:332)
        at
org.apache.solr.handler.XmlUpdateRequestHandler.update(XmlUpdateRequestH
andler.java:162)
        at
org.apache.solr.handler.XmlUpdateRequestHandler.doLegacyUpdate(XmlUpdate
RequestHandler.java:355)
        at
org.apache.solr.servlet.SolrUpdateServlet.doPost(SolrUpdateServlet.java:
58)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
tionFilterChain.java:269)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
erChain.java:188)
        at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.j
ava:185)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(Applica
tionFilterChain.java:215)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilt
erChain.java:188)
        at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValv
e.java:213)
        at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValv
e.java:174)
        at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java
:127)
        at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java
:117)
        at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.
java:108)
        at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:1
74)
        at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:87
4)
        at
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.proc
essConnection(Http11BaseProtocol.java:665)
        at
org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint
.java:528)
        at
org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollow
erWorkerThread.java:81)
        at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool
.java:689)
        at java.lang.Thread.run(Thread.java:595)
</result>

I did checked the tomcat maxPostSize setting  and it was set to 0 to
disable post size limitation.

Thank you very much in advance,
Andrew


Practical Law Company Limited. Registered in England and Wales. Registered Number: 02889191. Registered Office: 19 Hatfields, London SE1 8DJ. Telephone: +44(0)20 7202 1200. http://www.practicallaw.com. DISCLAIMERS: This e-mail is confidential to the ordinary user of the e-mail address to which it was addressed; it may contain personal views which are not the views of PLC unless specifically stated.  If you have received it in error, please delete it from your system and notify the sender immediately; do not use, copy or disclose the information or otherwise use it or any part of it in any form whatsoever nor act in reliance on it.  All liability for loss or damage caused by viruses is excluded.  PLC uses the MessageLabs SkyScan service to monitor all outgoing and incoming e-mails for viruses, prohibited file-type attachments and inappropriate content in accordance with its Information and Communications Systems policy.  You signify your consent to these terms by any further e-mail communication.
Reply | Threaded
Open this post in threaded view
|

Re: size limitation when adding document?

Yonik Seeley-2
On Thu, Mar 27, 2008 at 1:21 PM, Andrew Tan <[hidden email]> wrote:

>  I am new to solr and just try it out. I copied solr.war (from 1.2.0
>  distribution) into tomcat 5.5.26's webapps directory and started tomcat.
>  Then I use the java SimplePostTool to add documents. when the document
>  is small, things are fine. However, when I tried to add document
>  (greater than 8KB) into solr server, I got the following error message :
>
>  <result status="1">java.io.EOFException: no more data available -
>  expected end tags &lt;/field&gt;&lt;/doc&gt;&lt;/add&gt; to close start
>  tag &lt;field&gt; from line 35 and start tag &lt;doc&gt; from line 3 and
>  start tag &lt;add&gt; from line 2, parser stopped on START_TAG seen
>  ...uiror Termination Fee if the preceding relates to antitrust or com...
>  @35:859

This looks like the XML may not be valid.
I just tried a big document (120K) and a big single field (75K).  Both
worked fine.
Are you sure the field values have reserved XML chars escaped
correctly?  Perhaps try opening the file in an XML editor.

-Yonik
Reply | Threaded
Open this post in threaded view
|

Re: size limitation when adding document?

timrobertson100
Today I added a single 9gig tab file into solr, with the resulting index
being 16gig.3 hours to load and is performing mightily fine (jvm -Xmx3G)


On Thu, Mar 27, 2008 at 7:08 PM, Yonik Seeley <[hidden email]> wrote:

> On Thu, Mar 27, 2008 at 1:21 PM, Andrew Tan <[hidden email]>
> wrote:
> >  I am new to solr and just try it out. I copied solr.war (from 1.2.0
> >  distribution) into tomcat 5.5.26's webapps directory and started
> tomcat.
> >  Then I use the java SimplePostTool to add documents. when the document
> >  is small, things are fine. However, when I tried to add document
> >  (greater than 8KB) into solr server, I got the following error message
> :
> >
> >  <result status="1">java.io.EOFException: no more data available -
> >  expected end tags &lt;/field&gt;&lt;/doc&gt;&lt;/add&gt; to close start
> >  tag &lt;field&gt; from line 35 and start tag &lt;doc&gt; from line 3
> and
> >  start tag &lt;add&gt; from line 2, parser stopped on START_TAG seen
> >  ...uiror Termination Fee if the preceding relates to antitrust or
> com...
> >  @35:859
>
> This looks like the XML may not be valid.
> I just tried a big document (120K) and a big single field (75K).  Both
> worked fine.
> Are you sure the field values have reserved XML chars escaped
> correctly?  Perhaps try opening the file in an XML editor.
>
> -Yonik
>