MapReduce parameter sensitivity

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

MapReduce parameter sensitivity

Anurup Pavuluri

Hi,

I was wondering if anyone could throw some light on how the performance
of MapReduce is effected by parameters such as number of Maps, number of
Reduces, buffer sizes etc.

If someone could tell which parameters effect the performance I will try
to experiment with them myself.

Thanks,
Anurup

Reply | Threaded
Open this post in threaded view
|

Re: MapReduce parameter sensitivity

howard chen
On 12/7/06, Anurup Pavuluri <[hidden email]> wrote:

>
> Hi,
>
> I was wondering if anyone could throw some light on how the performance
> of MapReduce is effected by parameters such as number of Maps, number of
> Reduces, buffer sizes etc.
>
> If someone could tell which parameters effect the performance I will try
> to experiment with them myself.
>
> Thanks,
> Anurup
>
>

from the hadoop page, i forgot the link,said that...

if numbers of node, n

mapper = n * 10
reducer = n * 3

would give good performance
Reply | Threaded
Open this post in threaded view
|

Re: MapReduce parameter sensitivity

Lee-4
In reply to this post by Anurup Pavuluri
This wiki page should help:
http://wiki.apache.org/lucene-hadoop/HowManyMapsAndReduces

On 12/6/06, Anurup Pavuluri <[hidden email]> wrote:

>
>
> Hi,
>
> I was wondering if anyone could throw some light on how the performance
> of MapReduce is effected by parameters such as number of Maps, number of
> Reduces, buffer sizes etc.
>
> If someone could tell which parameters effect the performance I will try
> to experiment with them myself.
>
> Thanks,
> Anurup
>
>