Hadoop Too SLow..

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Hadoop Too SLow..

Volkan Ebil


I have got a cluster with 3 machines : 1 Master and 2 slaves

I set -the mapred max tasks 5

       -mapred map tasks 17 and reduce tasks 2


I start crawl with depth 2 topN 2 but it runs approximately 25 minute.

I start a local crawl with 1 computer  and it finishes in 2 minutes .


The difference is very big .Is it normal or am i wrong in configurations.


I tryed with different settings of map and reduce tasks but i have observed
that these numbers

don't have direct impact on speed  



Help needed urgently.