Maven

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Maven

Fuad Efendi
Maven uses repository of dependencies (jar files, libraries); it can connect
and download necessary jar files. I noticed that all Maven-based Apache
projects use ANT build scripts at first; Maven build is provided to
standardize "Look-and-Feel" of a project web-pages.

Reply | Threaded
Open this post in threaded view
|

Unable to complete a full fetch, reason Child Error

Gal Nitzan
During fetch all tasktrackers aborting the fetch with:

task_m_b45ma2 Child Error
java.io.IOException: Task process exit with nonzero status.
        at
org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)





Reply | Threaded
Open this post in threaded view
|

Re: Unable to complete a full fetch, reason Child Error

Doug Cutting
Gal Nitzan wrote:
> During fetch all tasktrackers aborting the fetch with:
>
> task_m_b45ma2 Child Error
> java.io.IOException: Task process exit with nonzero status.
>         at
> org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
>         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)
>

What's reported just before this in this tasktracker's log?

What's reported around this time in the jobtracker's log?

Doug
Reply | Threaded
Open this post in threaded view
|

Re: Unable to complete a full fetch, reason Child Error

Mike Smith-8
Hi,

This problem is killer! I've been strugelling with this for about a month!
It doesn't happen all the time, because of this problem the largest crawl I
could ever done is about 1 million pages.  I have three machines, 3
datanode, 1 data replicate, 1 job tracker, here is what I get:

nameserver tasktracker log file:

060219 142405 task_r_125kgt 0.14583334% reduce > copy >
060219 142406 task_r_125kgt 0.14583334% reduce > copy >
060219 142407 task_m_grycae  Error running child
060219 142407 task_m_grycae java.io.IOException: timed out waiting for
response
060219 142407 task_m_grycae     at org.apache.hadoop.ipc.Client.call(
Client.java:303)
060219 142407 task_m_grycae     at org.apache.hadoop.ipc.RPC$Invoker.invoke(
RPC.java:141)
060219 142407 task_m_grycae     at
org.apache.hadoop.mapred.$Proxy0.progress(Unknown
Source)
060219 142407 task_m_grycae     at
org.apache.hadoop.mapred.Task.reportProgress(Task.java:112)
060219 142407 task_m_grycae     at org.apache.hadoop.mapred.Task$1.setStatus
(Task.java:93)
060219 142407 task_m_grycae     at
org.apache.nutch.fetcher.Fetcher.reportStatus(Fetcher.java:276)
060219 142407 task_m_grycae     at org.apache.nutch.fetcher.Fetcher.run(
Fetcher.java:325)
060219 142407 task_m_grycae     at org.apache.hadoop.mapred.MapTask.run(
MapTask.java:129)
060219 142407 task_m_grycae     at
org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:637)
060219 142407 task_m_grycae 0.825607% 108745 pages, 5259 errors,
15.6pages/s, 2418 kb/s,
060219 142407 task_r_125kgt 0.14583334% reduce > copy >
060219 142408 task_m_grycae  Parent died.  Exiting task_m_grycae
060219 142408 task_r_125kgt 0.14583334% reduce > copy >
060219 142408 Server connection on port 50050 from xxxxxx: exiting
060219 142408 Server connection on port 50050 from xxxxxx: exiting
060219 142408 task_m_grycae Child Error
java.io.IOException: Task process exit with nonzero status.
        at org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)
060219 142411 task_m_grycae done; removing files.
060219 142413 task_r_125kgt 0.14583334% reduce > copy >


One of the datanode tasktracker log file:

060219 142611 task_m_2yfbgf  fetching
http://codex.wordpress.org/Managing_Plugins
060219 142611 task_m_2yfbgf  fetching
http://www.scubaboard.com/cms/search.php
060219 142611 task_m_2yfbgf Error reading child output
java.io.IOException: Bad file descriptor
        at java.io.FileInputStream.readBytes(Native Method)
        at java.io.FileInputStream.read(FileInputStream.java:194)
        at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java
:411)
        at sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java
:453)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:183)
        at java.io.InputStreamReader.read(InputStreamReader.java:167)
        at java.io.BufferedReader.fill(BufferedReader.java:136)
        at java.io.BufferedReader.readLine(BufferedReader.java:299)
        at java.io.BufferedReader.readLine(BufferedReader.java:362)
        at org.apache.hadoop.mapred.TaskRunner.logStream(TaskRunner.java
:170)
        at org.apache.hadoop.mapred.TaskRunner.access$100(TaskRunner.java
:29)
        at org.apache.hadoop.mapred.TaskRunner$1.run(TaskRunner.java:137)
060219 142611 task_m_2yfbgf 0.019530244% 2170 pages, 61 errors,
12.3pages/s, 1975 kb/s,
060219 142612 Server connection on port 50051 from xxxxxx: exiting
060219 142612 Server connection on port 50051 from xxxxxx: exiting
060219 142612 task_m_2yfbgf Child Error
java.io.IOException: Task process exit with nonzero status.
        at org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)
060219 142615 task_m_2yfbgf done; removing files.

The other datanode looks fine.


Thanks, Mike


On 2/16/06, Doug Cutting <[hidden email]> wrote:

>
> Gal Nitzan wrote:
> > During fetch all tasktrackers aborting the fetch with:
> >
> > task_m_b45ma2 Child Error
> > java.io.IOException: Task process exit with nonzero status.
> >         at
> > org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
> >         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)
> >
>
> What's reported just before this in this tasktracker's log?
>
> What's reported around this time in the jobtracker's log?
>
> Doug
>
Reply | Threaded
Open this post in threaded view
|

Re: Unable to complete a full fetch, reason Child Error

Gal Nitzan
I just got it again and I noticed something strange. One of the task
trackers seems to have finished the task: task_m_4b1l34 which you can
see is not done yet.

Than it gets a new task: task_r_1f4jb0. It completes it immediately and
when this task is done all 4 other trackers die with that dreadful
message :) ....

The story is here: (I removed all the parsing messages for shorter
message)

060220 102853 task_m_4b1l34 0.039386094% 93 pages, 0 errors, 0.6
pages/s, 98 kb/s,
060220 102853 Task task_m_4b1l34 is done.
060220 102853 task_r_1f4jb0 0.2% reduce > copy >
060220 102853 Server connection on port 50050 from 127.0.0.1: exiting
060220 102853 Server connection on port 50050 from 127.0.0.1: exiting
060220 102854 task_r_1f4jb0 0.2% reduce > copy >
060220 102854 task_r_1f4jb0 Got 1 map output locations.
060220 102855 task_r_1f4jb0  Child starting
060220 102855 Server connection on port 50050 from 127.0.0.1: starting
060220 102855 task_r_1f4jb0  Client connection to 0.0.0.0:50050:
starting
060220 102856 task_r_1f4jb0
parsing /tmp/hadoop/mapred/local/taskTracker/task_r_1f4jb0/job.xml
060220 102856 task_r_1f4jb0  Client connection to 212.143.22.185:9000:
starting
060220 102856 task_r_1f4jb0  Using URL normalizer:
org.apache.nutch.net.RegexUrlNormalizer
060220 102856 task_r_1f4jb0  loading
file:/home/nutchuser/trunk/conf/regex-normalize.xml
060220 102856 task_r_1f4jb0  Plugins: looking
in: /tmp/hadoop/mapred/local/taskTracker/task_r_1f4jb0/work/plugins
060220 102857 Server connection on port 50050 from 127.0.0.1: starting
060220 102857 task_r_1f4jb0  Client connection to 0.0.0.0:50050:
starting
060220 102857 task_r_1f4jb0  Plugin Auto-activation mode: [true]
060220 102857 task_r_1f4jb0  Registered Plugins:
060220 102857 task_r_1f4jb0     CyberNeko HTML Parser (lib-nekohtml)
060220 102857 task_r_1f4jb0     Site Query Filter (query-site)
060220 102857 task_r_1f4jb0     Http / Https Protocol Plug-in
(protocol-httpclient)
060220 102857 task_r_1f4jb0     Html Parse Plug-in (parse-html)
060220 102857 task_r_1f4jb0     Jakarta Commons HTTP Client
(lib-commons-httpclient)
060220 102857 task_r_1f4jb0     Basic Indexing Filter (index-basic)
060220 102857 task_r_1f4jb0     Text Parse Plug-in (parse-text)
060220 102857 task_r_1f4jb0     Regex URL Filter (urlfilter-regex)
060220 102857 task_r_1f4jb0     Basic Query Filter (query-basic)
060220 102857 task_r_1f4jb0     HTTP Framework (lib-http)
060220 102857 task_r_1f4jb0     Speedbit Parse Filter plugin
(parse-speedbit)
060220 102857 task_r_1f4jb0     URL Query Filter (query-url)
060220 102857 task_r_1f4jb0     Speedbit Query Filter (query-speedbit)
060220 102857 task_r_1f4jb0     the nutch core extension points
(nutch-extensionpoints)
060220 102857 task_r_1f4jb0     More Indexing Filter (index-more)
060220 102857 task_r_1f4jb0     Speedbit Indexing Filter
(index-speedbit)
060220 102857 task_r_1f4jb0  Registered Extension-Points:
060220 102857 task_r_1f4jb0     Nutch Protocol
(org.apache.nutch.protocol.Protocol)
060220 102857 task_r_1f4jb0     Nutch URL Filter
(org.apache.nutch.net.URLFilter)
060220 102857 task_r_1f4jb0     HTML Parse Filter
(org.apache.nutch.parse.HtmlParseFilter)
060220 102857 task_r_1f4jb0     Nutch Online Search Results Clustering
Plugin (org.apache.nutch.clustering.OnlineClusterer)
060220 102857 task_r_1f4jb0     Nutch Indexing Filter
(org.apache.nutch.indexer.IndexingFilter)
060220 102857 task_r_1f4jb0     Nutch Content Parser
(org.apache.nutch.parse.Parser)
060220 102857 task_r_1f4jb0     Ontology Model Loader
(org.apache.nutch.ontology.Ontology)
060220 102857 task_r_1f4jb0     Nutch Analysis
(org.apache.nutch.analysis.NutchAnalyzer)
060220 102857 task_r_1f4jb0     Nutch Query Filter
(org.apache.nutch.searcher.QueryFilter)
060220 102857 task_r_1f4jb0  found resource regex-urlfilter.txt at
file:/home/nutchuser/trunk/conf/regex-urlfilter.txt
060220 102857 task_r_1f4jb0 0.75000536% reduce > reduce
060220 102858 task_r_1f4jb0 0.75769955% reduce > reduce
060220 102859 task_r_1f4jb0 0.77046275% reduce > reduce
060220 102900 task_r_1f4jb0 0.7867212% reduce > reduce
060220 102902 task_r_1f4jb0 0.80398625% reduce > reduce
060220 102903 task_r_1f4jb0 0.8100066% reduce > reduce
060220 102904 task_r_1f4jb0 0.8282599% reduce > reduce
060220 102905 task_r_1f4jb0 0.8453781% reduce > reduce
060220 102906 task_r_1f4jb0 0.8640073% reduce > reduce
060220 102907 task_r_1f4jb0 0.8819447% reduce > reduce
060220 102908 task_r_1f4jb0 0.89916956% reduce > reduce
060220 102909 task_r_1f4jb0 0.91789067% reduce > reduce
060220 102910 task_r_1f4jb0 0.93869305% reduce > reduce
060220 102911 task_r_1f4jb0 0.9614557% reduce > reduce
060220 102912 task_r_1f4jb0 0.98324126% reduce > reduce
060220 102915 task_r_1f4jb0 1.0% reduce > reduce
060220 102915 Task task_r_1f4jb0 is done.
060220 102916 Server connection on port 50050 from 127.0.0.1: exiting
060220 102916 Server connection on port 50050 from 127.0.0.1: exiting
060220 102945 task_m_1dgza done; removing files.
060220 102948 task_m_4b1l34 done; removing files.
060220 102951 task_m_8t33q5 done; removing files.

All the other trackers got:

060220 102946 task_m_11tcmy Child Error
java.io.IOException: Task process exit with nonzero status.
        at
org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)


Hope it helps.

Gal.





On Sun, 2006-02-19 at 12:27 -0800, Mike Smith wrote:

> Hi,
>
> This problem is killer! I've been strugelling with this for about a month!
> It doesn't happen all the time, because of this problem the largest crawl I
> could ever done is about 1 million pages.  I have three machines, 3
> datanode, 1 data replicate, 1 job tracker, here is what I get:
>
> nameserver tasktracker log file:
>
> 060219 142405 task_r_125kgt 0.14583334% reduce > copy >
> 060219 142406 task_r_125kgt 0.14583334% reduce > copy >
> 060219 142407 task_m_grycae  Error running child
> 060219 142407 task_m_grycae java.io.IOException: timed out waiting for
> response
> 060219 142407 task_m_grycae     at org.apache.hadoop.ipc.Client.call(
> Client.java:303)
> 060219 142407 task_m_grycae     at org.apache.hadoop.ipc.RPC$Invoker.invoke(
> RPC.java:141)
> 060219 142407 task_m_grycae     at
> org.apache.hadoop.mapred.$Proxy0.progress(Unknown
> Source)
> 060219 142407 task_m_grycae     at
> org.apache.hadoop.mapred.Task.reportProgress(Task.java:112)
> 060219 142407 task_m_grycae     at org.apache.hadoop.mapred.Task$1.setStatus
> (Task.java:93)
> 060219 142407 task_m_grycae     at
> org.apache.nutch.fetcher.Fetcher.reportStatus(Fetcher.java:276)
> 060219 142407 task_m_grycae     at org.apache.nutch.fetcher.Fetcher.run(
> Fetcher.java:325)
> 060219 142407 task_m_grycae     at org.apache.hadoop.mapred.MapTask.run(
> MapTask.java:129)
> 060219 142407 task_m_grycae     at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:637)
> 060219 142407 task_m_grycae 0.825607% 108745 pages, 5259 errors,
> 15.6pages/s, 2418 kb/s,
> 060219 142407 task_r_125kgt 0.14583334% reduce > copy >
> 060219 142408 task_m_grycae  Parent died.  Exiting task_m_grycae
> 060219 142408 task_r_125kgt 0.14583334% reduce > copy >
> 060219 142408 Server connection on port 50050 from xxxxxx: exiting
> 060219 142408 Server connection on port 50050 from xxxxxx: exiting
> 060219 142408 task_m_grycae Child Error
> java.io.IOException: Task process exit with nonzero status.
>         at org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
>         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)
> 060219 142411 task_m_grycae done; removing files.
> 060219 142413 task_r_125kgt 0.14583334% reduce > copy >
>
>
> One of the datanode tasktracker log file:
>
> 060219 142611 task_m_2yfbgf  fetching
> http://codex.wordpress.org/Managing_Plugins
> 060219 142611 task_m_2yfbgf  fetching
> http://www.scubaboard.com/cms/search.php
> 060219 142611 task_m_2yfbgf Error reading child output
> java.io.IOException: Bad file descriptor
>         at java.io.FileInputStream.readBytes(Native Method)
>         at java.io.FileInputStream.read(FileInputStream.java:194)
>         at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java
> :411)
>         at sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java
> :453)
>         at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:183)
>         at java.io.InputStreamReader.read(InputStreamReader.java:167)
>         at java.io.BufferedReader.fill(BufferedReader.java:136)
>         at java.io.BufferedReader.readLine(BufferedReader.java:299)
>         at java.io.BufferedReader.readLine(BufferedReader.java:362)
>         at org.apache.hadoop.mapred.TaskRunner.logStream(TaskRunner.java
> :170)
>         at org.apache.hadoop.mapred.TaskRunner.access$100(TaskRunner.java
> :29)
>         at org.apache.hadoop.mapred.TaskRunner$1.run(TaskRunner.java:137)
> 060219 142611 task_m_2yfbgf 0.019530244% 2170 pages, 61 errors,
> 12.3pages/s, 1975 kb/s,
> 060219 142612 Server connection on port 50051 from xxxxxx: exiting
> 060219 142612 Server connection on port 50051 from xxxxxx: exiting
> 060219 142612 task_m_2yfbgf Child Error
> java.io.IOException: Task process exit with nonzero status.
>         at org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
>         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)
> 060219 142615 task_m_2yfbgf done; removing files.
>
> The other datanode looks fine.
>
>
> Thanks, Mike
>
>
> On 2/16/06, Doug Cutting <[hidden email]> wrote:
> >
> > Gal Nitzan wrote:
> > > During fetch all tasktrackers aborting the fetch with:
> > >
> > > task_m_b45ma2 Child Error
> > > java.io.IOException: Task process exit with nonzero status.
> > >         at
> > > org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
> > >         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)
> > >
> >
> > What's reported just before this in this tasktracker's log?
> >
> > What's reported around this time in the jobtracker's log?
> >
> > Doug
> >


Reply | Threaded
Open this post in threaded view
|

Re: Unable to complete a full fetch, reason Child Error

Doug Cutting
In reply to this post by Mike Smith-8
Mike Smith wrote:
> 060219 142408 task_m_grycae  Parent died.  Exiting task_m_grycae

This means the child process, executing the task, was unable to ping its
parent process (the task tracker).

> 060219 142408 task_m_grycae Child Error
> java.io.IOException: Task process exit with nonzero status.
>         at org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
>         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)

And this means that the parent was really still alive, and has noticed
that the child killed itself.

It would be good to know how the child failed to contact its parent.  We
should probably log a stack trace when this happens.  I just made that
change in Hadoop and will propagate it to Nutch.

Doug
Reply | Threaded
Open this post in threaded view
|

Re: Unable to complete a full fetch, reason Child Error

Gal Nitzan
Still got the same...

I'm not sure if it is relevant to this issue but the call you added to
Fetcher.java:

     job.setBoolean("mapred.speculative.execution", false);

Doesn't work. All task trackers still fetch together though I have only
3 sites in the fetchlist.

The task trackers fetch the same pages...

I have used latest build from hadoop trunk.

Gal.


On Fri, 2006-02-24 at 14:15 -0800, Doug Cutting wrote:

> Mike Smith wrote:
> > 060219 142408 task_m_grycae  Parent died.  Exiting task_m_grycae
>
> This means the child process, executing the task, was unable to ping its
> parent process (the task tracker).
>
> > 060219 142408 task_m_grycae Child Error
> > java.io.IOException: Task process exit with nonzero status.
> >         at org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:144)
> >         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)
>
> And this means that the parent was really still alive, and has noticed
> that the child killed itself.
>
> It would be good to know how the child failed to contact its parent.  We
> should probably log a stack trace when this happens.  I just made that
> change in Hadoop and will propagate it to Nutch.
>
> Doug
>


Reply | Threaded
Open this post in threaded view
|

Re: Unable to complete a full fetch, reason Child Error

Mike Smith-8
Hi Doug

I did some more testings using the last svn. Childs still die without any
clear log after a while.

I used two machines through Hadoop, both are datanode and tasktracker and
one is namenode and jobtracker. I started with 2000 seed nodes and it went
fine till 4th cycle, reached about 600,000 pages and the next round was for
3,000,000 pages to fetch. It failed again with this exception in the middle
of fetching:

060302 232934 task_m_7lbv7e  fetching
http://www.findarticles.com/p/articles/mi_m0KJI/is_9_115/ai_107836357
060302 232934 task_m_7lbv7e  fetching
http://www.wholehealthmd.com/hc/resourceareas_supp/1,1442,544,00.html
060302 232934 task_m_7lbv7e  fetching
http://www.dow.com/haltermann/products/d-petro.htm
060302 232934 task_m_7lbv7e 0.7877368% 700644 pages, 24594 errors,
14.0pages/s, 2254 kb/s,
060302 232934 task_m_7lbv7e  fetching
http://www.findarticles.com/p/articles/mi_hb3594/is_199510/ai_n8541042
060302 232934 task_m_7lbv7e Error reading child output
java.io.IOException: Bad file descriptor
        at java.io.FileInputStream.readBytes(Native Method)
        at java.io.FileInputStream.read(FileInputStream.java:194)
        at sun.nio.cs.StreamDecoder$CharsetSD.readBytes(StreamDecoder.java
:411)
        at sun.nio.cs.StreamDecoder$CharsetSD.implRead(StreamDecoder.java
:453)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:183)
        at java.io.InputStreamReader.read(InputStreamReader.java:167)
        at java.io.BufferedReader.fill(BufferedReader.java:136)
        at java.io.BufferedReader.readLine(BufferedReader.java:299)
        at java.io.BufferedReader.readLine(BufferedReader.java:362)
        at org.apache.hadoop.mapred.TaskRunner.logStream(TaskRunner.java
:299)
        at org.apache.hadoop.mapred.TaskRunner.access$100(TaskRunner.java
:32)
        at org.apache.hadoop.mapred.TaskRunner$1.run(TaskRunner.java:266)
060302 232934 task_m_7lbv7e 0.7877451% 700644 pages, 24594 errors,
14.0pages/s, 2254 kb/s,
060302 232934 task_m_7lbv7e 0.7877451% 700644 pages, 24594 errors,
14.0pages/s, 2254 kb/s,
060302 232934 Server connection on port 50050 from 164.67.195.27: exiting
060302 232934 Server connection on port 50050 from 164.67.195.27: exiting
060302 232934 task_m_7lbv7e Child Error
java.io.IOException: Task process exit with nonzero status.
        at org.apache.hadoop.mapred.TaskRunner.runChild(TaskRunner.java:273)
        at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:145)
060302 232937 task_m_7lbv7e done; removing files.


And this is console output:



060303 010945  map 86%  reduce 0%
060303 012033  map 86%  reduce 6%
060303 012223  map 87%  reduce 6%
060303 014623  map 88%  reduce 6%
060303 021304  map 89%  reduce 6%
060303 022921  map 50%  reduce 0%
060303 022921 SEVERE error, caught Exception in main()
java.io.IOException: Job failed!
        at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:310)
        at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:366)
        at org.apache.nutch.fetcher.Fetcher.doMain(Fetcher.java:400)
        at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:411)


This error has been around for large scale crawl since couple months ago. I
was wondering if anybody else has had the same issue for large scale crawl.

Thanks, Mike.






On 2/26/06, Gal Nitzan <[hidden email]> wrote:

>
> Still got the same...
>
> I'm not sure if it is relevant to this issue but the call you added to
> Fetcher.java:
>
>     job.setBoolean("mapred.speculative.execution", false);
>
> Doesn't work. All task trackers still fetch together though I have only
> 3 sites in the fetchlist.
>
> The task trackers fetch the same pages...
>
> I have used latest build from hadoop trunk.
>
> Gal.
>
>
> On Fri, 2006-02-24 at 14:15 -0800, Doug Cutting wrote:
> > Mike Smith wrote:
> > > 060219 142408 task_m_grycae  Parent died.  Exiting task_m_grycae
> >
> > This means the child process, executing the task, was unable to ping its
> > parent process (the task tracker).
> >
> > > 060219 142408 task_m_grycae Child Error
> > > java.io.IOException: Task process exit with nonzero status.
> > >         at org.apache.hadoop.mapred.TaskRunner.runChild(
> TaskRunner.java:144)
> > >         at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:97)
> >
> > And this means that the parent was really still alive, and has noticed
> > that the child killed itself.
> >
> > It would be good to know how the child failed to contact its parent.  We
> > should probably log a stack trace when this happens.  I just made that
> > change in Hadoop and will propagate it to Nutch.
> >
> > Doug
> >
>
>
>