Solr 4 Alpha SolrJ Indexing Issue

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr 4 Alpha SolrJ Indexing Issue

Briggs Thompson
Hello All,

I am using 4.0 Alpha and running into an issue with indexing using
HttpSolrServer (SolrJ).

Relevant java code:
            HttpSolrServer solrServer = new HttpSolrServer(MY_SERVER);
            solrServer.setRequestWriter(new BinaryRequestWriter());

Relevant Solrconfig.xml content:

  <requestHandler name="/update" class="solr.UpdateRequestHandler"  />

  <requestHandler name="/update/javabin"
class="solr.BinaryUpdateRequestHandler" />

Indexing documents works perfectly fine (using addBeans()), however, when
trying to do deletes I am seeing issues. I tried to do
a solrServer.deleteByQuery("*:*") followed by a commit and optimize, and
nothing is deleted.

The response from delete request is a "success", and even in the solr logs
I see the following:

INFO: [coupon] webapp=/solr path=/update/javabin
params={wt=javabin&version=2} {deleteByQuery=*:*} 0 1
Jul 18, 2012 11:15:34 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit{flags=0,version=0,optimize=true,openSearcher=true,waitSearcher=false,expungeDeletes=false,softCommit=false}



I tried removing the binaryRequestWriter and have the request send out in
default format, and I get the following error.

SEVERE: org.apache.solr.common.SolrException: Unsupported ContentType:
application/octet-stream  Not in: [application/xml, text/csv, text/json,
application/csv, application/javabin, text/xml, application/json]

at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:86)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1561)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)


I thought that an optimize does the same thing as expungeDeletes, but in
the log I see expungeDeletes=false. Is there a way to force that using
SolrJ?

Thanks in advance,
Briggs
Reply | Threaded
Open this post in threaded view
|

Re: Solr 4 Alpha SolrJ Indexing Issue

Briggs Thompson
I have realized this is not specific to SolrJ but to my instance of Solr. Using curl to delete by query is not working either. 

Running 
curl http://localhost:8983/solr/coupon/update -H "Content-Type: text/xml" --data-binary '<delete><query>*:*</query></delete>'

Yields this in the logs:
INFO: [coupon] webapp=/solr path=/update params={stream.body=<delete><query>*:*</query></delete>} {deleteByQuery=*:*} 0 0

But the corpus of documents in the core do not change. 

My solrconfig is pretty barebones at this point, but I attached it in case anyone sees something strange. Anyone have any idea why documents aren't getting deleted?

Thanks in advance,
Briggs Thompson

On Wed, Jul 18, 2012 at 12:54 PM, Briggs Thompson <[hidden email]> wrote:
Hello All,

I am using 4.0 Alpha and running into an issue with indexing using HttpSolrServer (SolrJ). 

Relevant java code:
            HttpSolrServer solrServer = new HttpSolrServer(MY_SERVER);
            solrServer.setRequestWriter(new BinaryRequestWriter());

Relevant Solrconfig.xml content:
  <requestHandler name="/update" class="solr.UpdateRequestHandler"  />
  <requestHandler name="/update/javabin" class="solr.BinaryUpdateRequestHandler" />

Indexing documents works perfectly fine (using addBeans()), however, when trying to do deletes I am seeing issues. I tried to do a solrServer.deleteByQuery("*:*") followed by a commit and optimize, and nothing is deleted. 

The response from delete request is a "success", and even in the solr logs I see the following:
INFO: [coupon] webapp=/solr path=/update/javabin params={wt=javabin&version=2} {deleteByQuery=*:*} 0 1
Jul 18, 2012 11:15:34 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start commit{flags=0,version=0,optimize=true,openSearcher=true,waitSearcher=false,expungeDeletes=false,softCommit=false}


I tried removing the binaryRequestWriter and have the request send out in default format, and I get the following error. 
SEVERE: org.apache.solr.common.SolrException: Unsupported ContentType: application/octet-stream  Not in: [application/xml, text/csv, text/json, application/csv, application/javabin, text/xml, application/json]
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:86)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1561)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:636)


I thought that an optimize does the same thing as expungeDeletes, but in the log I see expungeDeletes=false. Is there a way to force that using SolrJ?

Thanks in advance,
Briggs



solrconfig.xml (6K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Solr 4 Alpha SolrJ Indexing Issue

Brendan Grainger
Hi Briggs,

I'm not sure about Solr 4.0, but do you need to commit?

> curl http://localhost:8983/solr/coupon/update?commit=true -H "Content-Type: text/xml" --data-binary '<delete><query>*:*</query></delete>'


Brendan


www.kuripai.com

On Jul 18, 2012, at 7:11 PM, Briggs Thompson wrote:

> I have realized this is not specific to SolrJ but to my instance of Solr. Using curl to delete by query is not working either.
>
> Running
> curl http://localhost:8983/solr/coupon/update -H "Content-Type: text/xml" --data-binary '<delete><query>*:*</query></delete>'
>
> Yields this in the logs:
> INFO: [coupon] webapp=/solr path=/update params={stream.body=<delete><query>*:*</query></delete>} {deleteByQuery=*:*} 0 0
>
> But the corpus of documents in the core do not change.
>
> My solrconfig is pretty barebones at this point, but I attached it in case anyone sees something strange. Anyone have any idea why documents aren't getting deleted?
>
> Thanks in advance,
> Briggs Thompson
>
> On Wed, Jul 18, 2012 at 12:54 PM, Briggs Thompson <[hidden email]> wrote:
> Hello All,
>
> I am using 4.0 Alpha and running into an issue with indexing using HttpSolrServer (SolrJ).
>
> Relevant java code:
>             HttpSolrServer solrServer = new HttpSolrServer(MY_SERVER);
>             solrServer.setRequestWriter(new BinaryRequestWriter());
>
> Relevant Solrconfig.xml content:
>   <requestHandler name="/update" class="solr.UpdateRequestHandler"  />
>   <requestHandler name="/update/javabin" class="solr.BinaryUpdateRequestHandler" />
>
> Indexing documents works perfectly fine (using addBeans()), however, when trying to do deletes I am seeing issues. I tried to do a solrServer.deleteByQuery("*:*") followed by a commit and optimize, and nothing is deleted.
>
> The response from delete request is a "success", and even in the solr logs I see the following:
> INFO: [coupon] webapp=/solr path=/update/javabin params={wt=javabin&version=2} {deleteByQuery=*:*} 0 1
> Jul 18, 2012 11:15:34 AM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: start commit{flags=0,version=0,optimize=true,openSearcher=true,waitSearcher=false,expungeDeletes=false,softCommit=false}
>
>
> I tried removing the binaryRequestWriter and have the request send out in default format, and I get the following error.
> SEVERE: org.apache.solr.common.SolrException: Unsupported ContentType: application/octet-stream  Not in: [application/xml, text/csv, text/json, application/csv, application/javabin, text/xml, application/json]
> at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:86)
> at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1561)
> at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:442)
> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:263)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:225)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:927)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
> at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1001)
> at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:579)
> at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
>
>
> I thought that an optimize does the same thing as expungeDeletes, but in the log I see expungeDeletes=false. Is there a way to force that using SolrJ?
>
> Thanks in advance,
> Briggs
>
>
> <solrconfig.xml>

Reply | Threaded
Open this post in threaded view
|

Re: Solr 4 Alpha SolrJ Indexing Issue

Yury Kats
In reply to this post by Briggs Thompson
On 7/18/2012 7:11 PM, Briggs Thompson wrote:
> I have realized this is not specific to SolrJ but to my instance of Solr. Using curl to delete by query is not working either.

Can be this: https://issues.apache.org/jira/browse/SOLR-3432
Reply | Threaded
Open this post in threaded view
|

Re: Solr 4 Alpha SolrJ Indexing Issue

Briggs Thompson
Yury,

Thank you so much! That was it. Man, I spent a good long while trouble
shooting this. Probably would have spent quite a bit more time. I
appreciate your help!!

-Briggs

On Wed, Jul 18, 2012 at 9:35 PM, Yury Kats <[hidden email]> wrote:

> On 7/18/2012 7:11 PM, Briggs Thompson wrote:
> > I have realized this is not specific to SolrJ but to my instance of
> Solr. Using curl to delete by query is not working either.
>
> Can be this: https://issues.apache.org/jira/browse/SOLR-3432
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr 4 Alpha SolrJ Indexing Issue

Mark Miller-3
we really need to resolve that issue soon...

On Jul 19, 2012, at 12:08 AM, Briggs Thompson wrote:

> Yury,
>
> Thank you so much! That was it. Man, I spent a good long while trouble
> shooting this. Probably would have spent quite a bit more time. I
> appreciate your help!!
>
> -Briggs
>
> On Wed, Jul 18, 2012 at 9:35 PM, Yury Kats <[hidden email]> wrote:
>
>> On 7/18/2012 7:11 PM, Briggs Thompson wrote:
>>> I have realized this is not specific to SolrJ but to my instance of
>> Solr. Using curl to delete by query is not working either.
>>
>> Can be this: https://issues.apache.org/jira/browse/SOLR-3432
>>

- Mark Miller
lucidimagination.com











Reply | Threaded
Open this post in threaded view
|

Re: Solr 4 Alpha SolrJ Indexing Issue

Briggs Thompson
This is unrelated for the most part, but the javabin update request handler
does not seem to be working properly when calling solrj
method*HttpSolrServer.deleteById(List<String> ids)
*. A single Id gets deleted from the index as opposed to the full list. It
appears properly in the logs - shows delete of all Ids sent, although all
but one remain in the index.

I confirmed that the default update request handler deletes the list
properly, so this appears to be a problem with
the BinaryUpdateRequestHandler.

Not an issue for me, just spreading the word.

Thanks,
Briggs

On Thu, Jul 19, 2012 at 9:00 AM, Mark Miller <[hidden email]> wrote:

> we really need to resolve that issue soon...
>
> On Jul 19, 2012, at 12:08 AM, Briggs Thompson wrote:
>
> > Yury,
> >
> > Thank you so much! That was it. Man, I spent a good long while trouble
> > shooting this. Probably would have spent quite a bit more time. I
> > appreciate your help!!
> >
> > -Briggs
> >
> > On Wed, Jul 18, 2012 at 9:35 PM, Yury Kats <[hidden email]> wrote:
> >
> >> On 7/18/2012 7:11 PM, Briggs Thompson wrote:
> >>> I have realized this is not specific to SolrJ but to my instance of
> >> Solr. Using curl to delete by query is not working either.
> >>
> >> Can be this: https://issues.apache.org/jira/browse/SOLR-3432
> >>
>
> - Mark Miller
> lucidimagination.com
>
>
>
>
>
>
>
>
>
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Solr 4 Alpha SolrJ Indexing Issue

Mark Miller-3
https://issues.apache.org/jira/browse/SOLR-3649

On Thu, Jul 19, 2012 at 3:34 PM, Briggs Thompson <
[hidden email]> wrote:

> This is unrelated for the most part, but the javabin update request handler
> does not seem to be working properly when calling solrj
> method*HttpSolrServer.deleteById(List<String> ids)
> *. A single Id gets deleted from the index as opposed to the full list. It
> appears properly in the logs - shows delete of all Ids sent, although all
> but one remain in the index.
>
> I confirmed that the default update request handler deletes the list
> properly, so this appears to be a problem with
> the BinaryUpdateRequestHandler.
>
> Not an issue for me, just spreading the word.
>
> Thanks,
> Briggs
>
> On Thu, Jul 19, 2012 at 9:00 AM, Mark Miller <[hidden email]>
> wrote:
>
> > we really need to resolve that issue soon...
> >
> > On Jul 19, 2012, at 12:08 AM, Briggs Thompson wrote:
> >
> > > Yury,
> > >
> > > Thank you so much! That was it. Man, I spent a good long while trouble
> > > shooting this. Probably would have spent quite a bit more time. I
> > > appreciate your help!!
> > >
> > > -Briggs
> > >
> > > On Wed, Jul 18, 2012 at 9:35 PM, Yury Kats <[hidden email]> wrote:
> > >
> > >> On 7/18/2012 7:11 PM, Briggs Thompson wrote:
> > >>> I have realized this is not specific to SolrJ but to my instance of
> > >> Solr. Using curl to delete by query is not working either.
> > >>
> > >> Can be this: https://issues.apache.org/jira/browse/SOLR-3432
> > >>
> >
> > - Mark Miller
> > lucidimagination.com
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>



--
- Mark

http://www.lucidimagination.com
Reply | Threaded
Open this post in threaded view
|

Re: Solr 4 Alpha SolrJ Indexing Issue

Briggs Thompson
Thanks Mark!

On Thu, Jul 19, 2012 at 4:07 PM, Mark Miller <[hidden email]> wrote:

> https://issues.apache.org/jira/browse/SOLR-3649
>
> On Thu, Jul 19, 2012 at 3:34 PM, Briggs Thompson <
> [hidden email]> wrote:
>
> > This is unrelated for the most part, but the javabin update request
> handler
> > does not seem to be working properly when calling solrj
> > method*HttpSolrServer.deleteById(List<String> ids)
> > *. A single Id gets deleted from the index as opposed to the full list.
> It
> > appears properly in the logs - shows delete of all Ids sent, although all
> > but one remain in the index.
> >
> > I confirmed that the default update request handler deletes the list
> > properly, so this appears to be a problem with
> > the BinaryUpdateRequestHandler.
> >
> > Not an issue for me, just spreading the word.
> >
> > Thanks,
> > Briggs
> >
> > On Thu, Jul 19, 2012 at 9:00 AM, Mark Miller <[hidden email]>
> > wrote:
> >
> > > we really need to resolve that issue soon...
> > >
> > > On Jul 19, 2012, at 12:08 AM, Briggs Thompson wrote:
> > >
> > > > Yury,
> > > >
> > > > Thank you so much! That was it. Man, I spent a good long while
> trouble
> > > > shooting this. Probably would have spent quite a bit more time. I
> > > > appreciate your help!!
> > > >
> > > > -Briggs
> > > >
> > > > On Wed, Jul 18, 2012 at 9:35 PM, Yury Kats <[hidden email]>
> wrote:
> > > >
> > > >> On 7/18/2012 7:11 PM, Briggs Thompson wrote:
> > > >>> I have realized this is not specific to SolrJ but to my instance of
> > > >> Solr. Using curl to delete by query is not working either.
> > > >>
> > > >> Can be this: https://issues.apache.org/jira/browse/SOLR-3432
> > > >>
> > >
> > > - Mark Miller
> > > lucidimagination.com
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
>
>
>
> --
> - Mark
>
> http://www.lucidimagination.com
>