java.lang.NullPointerException im Sortetr/MergeQueue

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

java.lang.NullPointerException im Sortetr/MergeQueue

Ion Badita
Hi!

I tried to run wordcount example with a 1,9 GB input file, on 6 Nodes.
Hadoop has split the job in: 31 maps and one reduce.

All the maps fail with this stacktrace

java.lang.NullPointerException
        at org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.merge(SequenceFile.java:2394)
        at org.apache.hadoop.io.SequenceFile$Sorter.merge(SequenceFile.java:2087)
        at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:498)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:191)
        at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1372)


I tried the example with hadoop 0.10.1 and nightly build hadoop-2007-02-02 (with the same input), i got the same exception.

Any help will be appreciated.
Thanks!

John

Reply | Threaded
Open this post in threaded view
|

Re: java.lang.NullPointerException im Sortetr/MergeQueue

Evan Liu
I've just come across exactly the same problem. Can anyone be so kind to
help? Thanks!

sincerely,

Liu Qingyan - Evan Liu
Engineer of Platform, Yahoo! China
-------------------------------------
Backyard: liuqingy
Tel: +86-10-65985498
-------------------------------------
Devoted, initiative, peaceful. All for work.
Stay hungry, stay foolish, stay cold.



Ion Badita wrote:

> Hi!
>
> I tried to run wordcount example with a 1,9 GB input file, on 6 Nodes.
> Hadoop has split the job in: 31 maps and one reduce.
>
> All the maps fail with this stacktrace
>
> java.lang.NullPointerException
>     at
> org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.merge(SequenceFile.java:2394)
>
>     at
> org.apache.hadoop.io.SequenceFile$Sorter.merge(SequenceFile.java:2087)
>     at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:498)
>
>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:191)
>     at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1372)
>
>
> I tried the example with hadoop 0.10.1 and nightly build
> hadoop-2007-02-02 (with the same input), i got the same exception.
>
> Any help will be appreciated.
> Thanks!
>
> John
>
>
Reply | Threaded
Open this post in threaded view
|

RE: java.lang.NullPointerException im Sortetr/MergeQueue

Devaraj Das
What version of hadoop are you using? I believe this problem got resolved a
while back (and 0.11.x has the fix).

Thanks
Devaraj.

> -----Original Message-----
> From: Evan Liu [mailto:[hidden email]]
> Sent: Monday, March 12, 2007 5:32 AM
> To: [hidden email]
> Subject: Re: java.lang.NullPointerException im Sortetr/MergeQueue
>
> I've just come across exactly the same problem. Can anyone be so kind to
> help? Thanks!
>
> sincerely,
>
> Liu Qingyan - Evan Liu
> Engineer of Platform, Yahoo! China
> -------------------------------------
> Backyard: liuqingy
> Tel: +86-10-65985498
> -------------------------------------
> Devoted, initiative, peaceful. All for work.
> Stay hungry, stay foolish, stay cold.
>
>
>
> Ion Badita wrote:
> > Hi!
> >
> > I tried to run wordcount example with a 1,9 GB input file, on 6 Nodes.
> > Hadoop has split the job in: 31 maps and one reduce.
> >
> > All the maps fail with this stacktrace
> >
> > java.lang.NullPointerException
> >     at
> >
> org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.merge(SequenceFile.jav
> a:2394)
> >
> >     at
> > org.apache.hadoop.io.SequenceFile$Sorter.merge(SequenceFile.java:2087)
> >     at
> >
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:4
> 98)
> >
> >     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:191)
> >     at
> > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1372)
> >
> >
> > I tried the example with hadoop 0.10.1 and nightly build
> > hadoop-2007-02-02 (with the same input), i got the same exception.
> >
> > Any help will be appreciated.
> > Thanks!
> >
> > John
> >
> >

Reply | Threaded
Open this post in threaded view
|

Re: java.lang.NullPointerException im Sortetr/MergeQueue

Evan Liu
*Started:* Wed Mar 14 19:49:15 PDT 2007
*Version:* 0.10.1, r495049
*Compiled:* Wed Jan 10 16:02:27 PST 2007 by cutting

Does thies version has the bug?

thanks

Liu Qingyan - Evan Liu
Engineer of Platform, Yahoo! China
-------------------------------------
Backyard: liuqingy
Tel: +86-10-65985498
-------------------------------------
Devoted, initiative, peaceful. All for work.
Stay hungry, stay foolish, stay cold.



Devaraj Das wrote:

> What version of hadoop are you using? I believe this problem got resolved a
> while back (and 0.11.x has the fix).
>
> Thanks
> Devaraj.
>
>  
>> -----Original Message-----
>> From: Evan Liu [mailto:[hidden email]]
>> Sent: Monday, March 12, 2007 5:32 AM
>> To: [hidden email]
>> Subject: Re: java.lang.NullPointerException im Sortetr/MergeQueue
>>
>> I've just come across exactly the same problem. Can anyone be so kind to
>> help? Thanks!
>>
>> sincerely,
>>
>> Liu Qingyan - Evan Liu
>> Engineer of Platform, Yahoo! China
>> -------------------------------------
>> Backyard: liuqingy
>> Tel: +86-10-65985498
>> -------------------------------------
>> Devoted, initiative, peaceful. All for work.
>> Stay hungry, stay foolish, stay cold.
>>
>>
>>
>> Ion Badita wrote:
>>    
>>> Hi!
>>>
>>> I tried to run wordcount example with a 1,9 GB input file, on 6 Nodes.
>>> Hadoop has split the job in: 31 maps and one reduce.
>>>
>>> All the maps fail with this stacktrace
>>>
>>> java.lang.NullPointerException
>>>     at
>>>
>>>      
>> org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.merge(SequenceFile.jav
>> a:2394)
>>    
>>>     at
>>> org.apache.hadoop.io.SequenceFile$Sorter.merge(SequenceFile.java:2087)
>>>     at
>>>
>>>      
>> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:4
>> 98)
>>    
>>>     at org.apache.hadoop.mapred.MapTask.run(MapTask.java:191)
>>>     at
>>> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1372)
>>>
>>>
>>> I tried the example with hadoop 0.10.1 and nightly build
>>> hadoop-2007-02-02 (with the same input), i got the same exception.
>>>
>>> Any help will be appreciated.
>>> Thanks!
>>>
>>> John
>>>
>>>
>>>      
>
>
>