Hadoop Too SLow..

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Hadoop Too SLow..

Volkan Ebil
Hi,

 

I have got a cluster with 3 machines : 1 Master and 2 slaves

I set -the mapred max tasks 5

       -mapred map tasks 17 and reduce tasks 2

 

I start crawl with depth 2 topN 2 but it runs approximately 25 minute.

I start a local crawl with 1 computer  and it finishes in 2 minutes .

 

The difference is very big .Is it normal or am i wrong in configurations.

 

I tryed with different settings of map and reduce tasks but i have observed
that these numbers

don't have direct impact on speed  

 

 

Help needed urgently.

Thanks