[jira] Resolved: (NUTCH-333) SegmentMerger and SegmentReader should use NutchJob

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[jira] Resolved: (NUTCH-333) SegmentMerger and SegmentReader should use NutchJob

Chris Mattmann (Jira)

     [ https://issues.apache.org/jira/browse/NUTCH-333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Dennis Kubes resolved NUTCH-333.

    Resolution: Fixed

Issue resolved

> SegmentMerger and SegmentReader should use NutchJob
> ---------------------------------------------------
>                 Key: NUTCH-333
>                 URL: https://issues.apache.org/jira/browse/NUTCH-333
>             Project: Nutch
>          Issue Type: Bug
>    Affects Versions: 0.9.0
>            Reporter: [hidden email]
>            Priority: Minor
>         Attachments: nutch333.patch, use-nutch-job_patch.txt
> I have a job jar that is nutch with additions. I can launch this job jar on a pure hadoop platform usually without issue.  I can run nutch jobs -- update db, invert links, etc. -- without issue.  Recently I tried to do the same with SegmentMerg'ing only it would fail complaining about ClassNotFound:
> 2006-07-28 20:43:54,371 WARN org.apache.hadoop.mapred.JobTracker: job init failed
> java.io.IOException: java.lang.ClassNotFoundException: org.apache.nutch.segment.SegmentMerger$ObjectInputFormat
>         at org.apache.hadoop.mapred.JobInProgress.initTasks(JobInProgress.java:130)
>         at org.apache.hadoop.mapred.JobTracker$JobInitThread.run(JobTracker.java:310)
>         at java.lang.Thread.run(Thread.java:595)
> java.io.IOException: Job failed!
> After digging and chatting today with Stefan, the SegmentMerger and SegmentReader classes are not like the others.   Others make a new JobConf inside in their job setup by doing a 'new NutchJob' whereas Segment* does 'new JobConf'.  Sure enough, if I make the change, all works.
> NutchJob triggers the setting of the job jar into the configuration (JobConf.findContainingJar is run).  This doesn't happen for 'new JobConf'.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.