Nutch 2 solrindex fails with no error

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Nutch 2 solrindex fails with no error

Bai Shen
I have a nutch 2 setup that I got working with solr about a month ago.  I
had to shelve it for a little while and I've recently come back to it.

Everything seems to be working fine except for the solr indexing.  To my
knowledge, nothing has changed between then and now, but whenever I go to
perform a solrindex, nothing gets index.  My hbase, hadoop, and solr logs
are all devoid of errors.  The only thing I get in the command line is the
following.

SolrIndexerJob: starting
SolrIndexerJob: done.


Any suggestions of where to look to begin troubleshooting this would be
appreciated.  I'm baffled.

Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 2 solrindex fails with no error

Eyeris
Hi Bai Shen.
I have no used yet nutch 2, but verify your started urls file.




----- Mensaje original -----
De: "Bai Shen" <[hidden email]>
Para: [hidden email]
Enviados: Viernes, 14 de Septiembre 2012 16:33:27
Asunto: Nutch 2 solrindex fails with no error

I have a nutch 2 setup that I got working with solr about a month ago.  I
had to shelve it for a little while and I've recently come back to it.

Everything seems to be working fine except for the solr indexing.  To my
knowledge, nothing has changed between then and now, but whenever I go to
perform a solrindex, nothing gets index.  My hbase, hadoop, and solr logs
are all devoid of errors.  The only thing I get in the command line is the
following.

SolrIndexerJob: starting
SolrIndexerJob: done.


Any suggestions of where to look to begin troubleshooting this would be
appreciated.  I'm baffled.

Thanks.


10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 2 solrindex fails with no error

lewis john mcgibbney
In reply to this post by Bai Shen
Solr logs?

On Fri, Sep 14, 2012 at 9:33 PM, Bai Shen <[hidden email]> wrote:

> I have a nutch 2 setup that I got working with solr about a month ago.  I
> had to shelve it for a little while and I've recently come back to it.
>
> Everything seems to be working fine except for the solr indexing.  To my
> knowledge, nothing has changed between then and now, but whenever I go to
> perform a solrindex, nothing gets index.  My hbase, hadoop, and solr logs
> are all devoid of errors.  The only thing I get in the command line is the
> following.
>
> SolrIndexerJob: starting
> SolrIndexerJob: done.
>
>
> Any suggestions of where to look to begin troubleshooting this would be
> appreciated.  I'm baffled.
>
> Thanks.



--
Lewis
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 2 solrindex fails with no error

Bai Shen
This is the solr log output when I run a solrindex.

Sep 17, 2012 10:38:55 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: start
commit(optimize=false,waitFlush=true,waitSearcher=true,expungeDeletes=false)
Sep 17, 2012 10:38:55 AM org.apache.solr.search.SolrIndexSearcher <init>
INFO: Opening Searcher@3f302c08 main
Sep 17, 2012 10:38:55 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@3f302c08 main from Searcher@447ec055 main

fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=2976,cumulative_hits=2928,cumulative_hitratio=0.98,cumulative_inserts=24,cumulative_evictions=0}
Sep 17, 2012 10:38:55 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@3f302c08 main

fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=2976,cumulative_hits=2928,cumulative_hitratio=0.98,cumulative_inserts=24,cumulative_evictions=0}
Sep 17, 2012 10:38:55 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@3f302c08 main from Searcher@447ec055 main

filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=1001,cumulative_hits=991,cumulative_hitratio=0.99,cumulative_inserts=133,cumulative_evictions=0}
Sep 17, 2012 10:38:55 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@3f302c08 main

filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=1001,cumulative_hits=991,cumulative_hitratio=0.99,cumulative_inserts=133,cumulative_evictions=0}
Sep 17, 2012 10:38:55 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@3f302c08 main from Searcher@447ec055 main

queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=499,cumulative_hits=472,cumulative_hitratio=0.94,cumulative_inserts=51,cumulative_evictions=0}
Sep 17, 2012 10:38:55 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@3f302c08 main

queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=499,cumulative_hits=472,cumulative_hitratio=0.94,cumulative_inserts=51,cumulative_evictions=0}
Sep 17, 2012 10:38:55 AM org.apache.solr.update.DirectUpdateHandler2 commit
INFO: end_commit_flush
Sep 17, 2012 10:38:55 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming Searcher@3f302c08 main from Searcher@447ec055 main

documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=14471,cumulative_hits=12662,cumulative_hitratio=0.87,cumulative_inserts=1809,cumulative_evictions=1177}
Sep 17, 2012 10:38:55 AM org.apache.solr.search.SolrIndexSearcher warm
INFO: autowarming result for Searcher@3f302c08 main

documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=14471,cumulative_hits=12662,cumulative_hitratio=0.87,cumulative_inserts=1809,cumulative_evictions=1177}
Sep 17, 2012 10:38:55 AM org.apache.solr.core.QuerySenderListener
newSearcher
INFO: QuerySenderListener sending requests to Searcher@3f302c08 main
Sep 17, 2012 10:38:55 AM org.apache.solr.core.QuerySenderListener
newSearcher
INFO: QuerySenderListener done.
Sep 17, 2012 10:38:55 AM org.apache.solr.core.SolrCore registerSearcher
INFO: [] Registered new searcher Searcher@3f302c08 main
Sep 17, 2012 10:38:55 AM org.apache.solr.search.SolrIndexSearcher close
INFO: Closing Searcher@447ec055 main

fieldValueCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=2976,cumulative_hits=2928,cumulative_hitratio=0.98,cumulative_inserts=24,cumulative_evictions=0}

filterCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=1001,cumulative_hits=991,cumulative_hitratio=0.99,cumulative_inserts=133,cumulative_evictions=0}

queryResultCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=499,cumulative_hits=472,cumulative_hitratio=0.94,cumulative_inserts=51,cumulative_evictions=0}

documentCache{lookups=0,hits=0,hitratio=0.00,inserts=0,evictions=0,size=0,warmupTime=0,cumulative_lookups=14471,cumulative_hits=12662,cumulative_hitratio=0.87,cumulative_inserts=1809,cumulative_evictions=1177}
Sep 17, 2012 10:38:55 AM
org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: {commit=} 0 12
Sep 17, 2012 10:38:55 AM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update
params={waitSearcher=true&waitFlush=true&wt=javabin&version=2&commit=true}
status=0 QTime=12


On Sat, Sep 15, 2012 at 7:36 AM, Lewis John Mcgibbney <
[hidden email]> wrote:

> Solr logs?
>
> On Fri, Sep 14, 2012 at 9:33 PM, Bai Shen <[hidden email]> wrote:
> > I have a nutch 2 setup that I got working with solr about a month ago.  I
> > had to shelve it for a little while and I've recently come back to it.
> >
> > Everything seems to be working fine except for the solr indexing.  To my
> > knowledge, nothing has changed between then and now, but whenever I go to
> > perform a solrindex, nothing gets index.  My hbase, hadoop, and solr logs
> > are all devoid of errors.  The only thing I get in the command line is
> the
> > following.
> >
> > SolrIndexerJob: starting
> > SolrIndexerJob: done.
> >
> >
> > Any suggestions of where to look to begin troubleshooting this would be
> > appreciated.  I'm baffled.
> >
> > Thanks.
>
>
>
> --
> Lewis
>
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 2 solrindex fails with no error

Bai Shen
In reply to this post by lewis john mcgibbney
The problem appears to be that Nutch is not sending anything to solr.  But
I can't seem to find a reason in nutch as to why this is.

On Sat, Sep 15, 2012 at 7:36 AM, Lewis John Mcgibbney <
[hidden email]> wrote:

> Solr logs?
>
> On Fri, Sep 14, 2012 at 9:33 PM, Bai Shen <[hidden email]> wrote:
> > I have a nutch 2 setup that I got working with solr about a month ago.  I
> > had to shelve it for a little while and I've recently come back to it.
> >
> > Everything seems to be working fine except for the solr indexing.  To my
> > knowledge, nothing has changed between then and now, but whenever I go to
> > perform a solrindex, nothing gets index.  My hbase, hadoop, and solr logs
> > are all devoid of errors.  The only thing I get in the command line is
> the
> > following.
> >
> > SolrIndexerJob: starting
> > SolrIndexerJob: done.
> >
> >
> > Any suggestions of where to look to begin troubleshooting this would be
> > appreciated.  I'm baffled.
> >
> > Thanks.
>
>
>
> --
> Lewis
>
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 2 solrindex fails with no error

alxsss

You can use -reindex option, since updt markers are not set properly in 2.0 release.

 

 

-----Original Message-----
From: Bai Shen <[hidden email]>
To: user <[hidden email]>
Sent: Mon, Sep 17, 2012 10:16 am
Subject: Re: Nutch 2 solrindex fails with no error


The problem appears to be that Nutch is not sending anything to solr.  But
I can't seem to find a reason in nutch as to why this is.

On Sat, Sep 15, 2012 at 7:36 AM, Lewis John Mcgibbney <
[hidden email]> wrote:

> Solr logs?
>
> On Fri, Sep 14, 2012 at 9:33 PM, Bai Shen <[hidden email]> wrote:
> > I have a nutch 2 setup that I got working with solr about a month ago.  I
> > had to shelve it for a little while and I've recently come back to it.
> >
> > Everything seems to be working fine except for the solr indexing.  To my
> > knowledge, nothing has changed between then and now, but whenever I go to
> > perform a solrindex, nothing gets index.  My hbase, hadoop, and solr logs
> > are all devoid of errors.  The only thing I get in the command line is
> the
> > following.
> >
> > SolrIndexerJob: starting
> > SolrIndexerJob: done.
> >
> >
> > Any suggestions of where to look to begin troubleshooting this would be
> > appreciated.  I'm baffled.
> >
> > Thanks.
>
>
>
> --
> Lewis
>

 
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 2 solrindex fails with no error

Bai Shen
That was it.

Thanks.


Any ideas when that will be fixed?  2.1?


On Mon, Sep 17, 2012 at 1:25 PM, <[hidden email]> wrote:

>
> You can use -reindex option, since updt markers are not set properly in
> 2.0 release.
>
>
>
>
>
> -----Original Message-----
> From: Bai Shen <[hidden email]>
> To: user <[hidden email]>
> Sent: Mon, Sep 17, 2012 10:16 am
> Subject: Re: Nutch 2 solrindex fails with no error
>
>
> The problem appears to be that Nutch is not sending anything to solr.  But
> I can't seem to find a reason in nutch as to why this is.
>
> On Sat, Sep 15, 2012 at 7:36 AM, Lewis John Mcgibbney <
> [hidden email]> wrote:
>
> > Solr logs?
> >
> > On Fri, Sep 14, 2012 at 9:33 PM, Bai Shen <[hidden email]>
> wrote:
> > > I have a nutch 2 setup that I got working with solr about a month ago.
>  I
> > > had to shelve it for a little while and I've recently come back to it.
> > >
> > > Everything seems to be working fine except for the solr indexing.  To
> my
> > > knowledge, nothing has changed between then and now, but whenever I go
> to
> > > perform a solrindex, nothing gets index.  My hbase, hadoop, and solr
> logs
> > > are all devoid of errors.  The only thing I get in the command line is
> > the
> > > following.
> > >
> > > SolrIndexerJob: starting
> > > SolrIndexerJob: done.
> > >
> > >
> > > Any suggestions of where to look to begin troubleshooting this would be
> > > appreciated.  I'm baffled.
> > >
> > > Thanks.
> >
> >
> >
> > --
> > Lewis
> >
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Nutch 2 solrindex fails with no error

Ferdy Galema
Yes it should be in 2.1 as it is currently in the 2.x branch:
https://issues.apache.org/jira/browse/NUTCH-1456

On Mon, Sep 17, 2012 at 7:56 PM, Bai Shen <[hidden email]> wrote:

> That was it.
>
> Thanks.
>
>
> Any ideas when that will be fixed?  2.1?
>
>
> On Mon, Sep 17, 2012 at 1:25 PM, <[hidden email]> wrote:
>
> >
> > You can use -reindex option, since updt markers are not set properly in
> > 2.0 release.
> >
> >
> >
> >
> >
> > -----Original Message-----
> > From: Bai Shen <[hidden email]>
> > To: user <[hidden email]>
> > Sent: Mon, Sep 17, 2012 10:16 am
> > Subject: Re: Nutch 2 solrindex fails with no error
> >
> >
> > The problem appears to be that Nutch is not sending anything to solr.
>  But
> > I can't seem to find a reason in nutch as to why this is.
> >
> > On Sat, Sep 15, 2012 at 7:36 AM, Lewis John Mcgibbney <
> > [hidden email]> wrote:
> >
> > > Solr logs?
> > >
> > > On Fri, Sep 14, 2012 at 9:33 PM, Bai Shen <[hidden email]>
> > wrote:
> > > > I have a nutch 2 setup that I got working with solr about a month
> ago.
> >  I
> > > > had to shelve it for a little while and I've recently come back to
> it.
> > > >
> > > > Everything seems to be working fine except for the solr indexing.  To
> > my
> > > > knowledge, nothing has changed between then and now, but whenever I
> go
> > to
> > > > perform a solrindex, nothing gets index.  My hbase, hadoop, and solr
> > logs
> > > > are all devoid of errors.  The only thing I get in the command line
> is
> > > the
> > > > following.
> > > >
> > > > SolrIndexerJob: starting
> > > > SolrIndexerJob: done.
> > > >
> > > >
> > > > Any suggestions of where to look to begin troubleshooting this would
> be
> > > > appreciated.  I'm baffled.
> > > >
> > > > Thanks.
> > >
> > >
> > >
> > > --
> > > Lewis
> > >
> >
> >
> >
>