CapacityScheduler vs. FairScheduler

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

CapacityScheduler vs. FairScheduler

Lars Francke
Hi,

I've been using Hadoop for years and have always just taken for granted that FairScheduler = Cloudera and CapacityScheduler = Hortonworks/Yahoo. There are some comparisons but all of them are years old and somewhat (if not entirely) outdated.

The documentation doesn't really help and neither does the Javadoc. The code of both is fairly complex.

So my question is: How do these two Schedulers really differ today? What are some features that one has that the other doesn't? Are there any fundamental differences (anymore)?

Any insight is welcome.

Thank you!

Cheers,
Lars
Reply | Threaded
Open this post in threaded view
|

Re: CapacityScheduler vs. FairScheduler

Alvin Chyan
I have the same question.

Thanks!

Alvin Chyan
Lead Software Engineer, Data
901 Marshall St, Suite 200, Redwood City, CA 94063


turn.com   |   @TurnPlatform

This message is Turn Confidential, except for information included that is already available to the public. If this message was sent to you accidentally, please delete it.


On Fri, Jun 3, 2016 at 11:04 AM, Lars Francke <[hidden email]> wrote:
Hi,

I've been using Hadoop for years and have always just taken for granted that FairScheduler = Cloudera and CapacityScheduler = Hortonworks/Yahoo. There are some comparisons but all of them are years old and somewhat (if not entirely) outdated.

The documentation doesn't really help and neither does the Javadoc. The code of both is fairly complex.

So my question is: How do these two Schedulers really differ today? What are some features that one has that the other doesn't? Are there any fundamental differences (anymore)?

Any insight is welcome.

Thank you!

Cheers,
Lars

Reply | Threaded
Open this post in threaded view
|

Re: CapacityScheduler vs. FairScheduler

sandeep vura
Hi,

I too have same doubt !! Please clarify.

Regards,
sandeep.v

On Fri, Jun 10, 2016 at 6:08 AM, Alvin Chyan <[hidden email]> wrote:
I have the same question.

Thanks!

Alvin Chyan
Lead Software Engineer, Data
901 Marshall St, Suite 200, Redwood City, CA 94063


turn.com   |   @TurnPlatform

This message is Turn Confidential, except for information included that is already available to the public. If this message was sent to you accidentally, please delete it.


On Fri, Jun 3, 2016 at 11:04 AM, Lars Francke <[hidden email]> wrote:
Hi,

I've been using Hadoop for years and have always just taken for granted that FairScheduler = Cloudera and CapacityScheduler = Hortonworks/Yahoo. There are some comparisons but all of them are years old and somewhat (if not entirely) outdated.

The documentation doesn't really help and neither does the Javadoc. The code of both is fairly complex.

So my question is: How do these two Schedulers really differ today? What are some features that one has that the other doesn't? Are there any fundamental differences (anymore)?

Any insight is welcome.

Thank you!

Cheers,
Lars


Reply | Threaded
Open this post in threaded view
|

Re: CapacityScheduler vs. FairScheduler

Wangda Tan
Hi folks,

I've a short write-up about feature-wise comparison between fair scheduler and capacity scheduler for latest Hadoop.


HTH,

- Wangda

On Sun, Jun 19, 2016 at 11:21 PM, sandeep vura <[hidden email]> wrote:
Hi,

I too have same doubt !! Please clarify.

Regards,
sandeep.v

On Fri, Jun 10, 2016 at 6:08 AM, Alvin Chyan <[hidden email]> wrote:
I have the same question.

Thanks!

Alvin Chyan
Lead Software Engineer, Data
901 Marshall St, Suite 200, Redwood City, CA 94063


turn.com   |   @TurnPlatform

This message is Turn Confidential, except for information included that is already available to the public. If this message was sent to you accidentally, please delete it.


On Fri, Jun 3, 2016 at 11:04 AM, Lars Francke <[hidden email]> wrote:
Hi,

I've been using Hadoop for years and have always just taken for granted that FairScheduler = Cloudera and CapacityScheduler = Hortonworks/Yahoo. There are some comparisons but all of them are years old and somewhat (if not entirely) outdated.

The documentation doesn't really help and neither does the Javadoc. The code of both is fairly complex.

So my question is: How do these two Schedulers really differ today? What are some features that one has that the other doesn't? Are there any fundamental differences (anymore)?

Any insight is welcome.

Thank you!

Cheers,
Lars