Re: YARN+MR APAC Sync Up Meeting Minutes 07/25

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: YARN+MR APAC Sync Up Meeting Minutes 07/25

Wanqiang Ji
Sorry for the previous mail with wrong format, I had fixed it.
—————————————
Participants:
Cloudera, Tencent, Taiwan University, Xiaomi, ByteDance, JD, Didi, Alibaba

ByteDance:
       Version: 2.6.0
       1000+ patches
       Fair scheduler
       Improvements: like CS global scheduling
       Scale: 40,000 hosts last year!!
       Flink + Spark + MR + other computations
       Not server side federation: client side to route apps to clusters
       Largest cluster 10,000 nodes

Alibaba:
       Version: 3.1.0 (upgraded 1.5 year before)
       CS + Global scheduling
       Overcommitment with opportunistic containers -> increase utilization
       Scheduler activities (share more details in upcoming meetup)

Xiaomi:
       2.6.0 -> 3.1.0 (upgrading in progress)
       Migrate FS to CS
       Investigating federation
       TODO: Open umbrella for migration

Hadoop Meetup 8/10 discussion:
       Location TBD, most likely will be a one day event
       Setting up a review-board, we are glad to have 1 or 2 contract-point
from each company in this board, please nominate to [hidden email]
       Call for more talks!!!


Wanqiang Ji

Wanqiang Ji <[hidden email]>于2019年7月25日 周四14:49写道:

> July 25, 2019
>
> Participants: Cloudera, Tencent, Taiwan University, Xiaomi, ByteDance, JD,
> Didi, Alibaba
>
> ByteDance
>
>    -
>
>    Version: 2.6.0
>    -
>
>    1000+ patches
>    -
>
>    Fair scheduler
>    -
>
>       Improvements: like CS global scheduling
>       -
>
>    Scale: 40,000 hosts last year!!
>    -
>
>    Flink + Spark + MR + other computations
>    -
>
>    Not server side federation: client side to route apps to clusters
>    -
>
>    Largest cluster 10,000 nodes
>
> Alibaba
>
>    -
>
>    Version: 3.1.0 (upgraded 1.5 year before)
>    -
>
>    CS + Global scheduling
>    -
>
>    Overcommitment with opportunistic containers -> increase utilization
>    -
>
>    Scheduler activities (share more details in upcoming meetup)
>
> Xiaomi
>
>    -
>
>    2.6.0 -> 3.1.0 (upgrading in progress)
>    -
>
>    Migrate FS to CS
>    -
>
>    Investigating federation
>    -
>
>    TODO: Open umbrella for migration
>
> Hadoop Meetup 8/10 discussion
>
>    -
>
>    Location TBD, most likely will be a one day event
>    -
>
>    Setting up a review-board, we are glad to have 1 or 2 contract-point
>    from each company in this board, please nominate to
>    [hidden email]
>    -
>
>    Call for more talks!!!
>
>
>
> FYI
> Wanqiang Ji
>
> 俊平堵 <[hidden email]>于2019年7月25日 周四11:53写道:
>
>> Hi Folks,
>>
>>      Kindly remind that we have YARN+MR APAC sync today, and you are
>> welcome to join:
>>
>>
>> Time and Date:07/25 1:00 pm (CST Time)
>>
>> Zoom link:Zoom | https://cloudera.zoom.us/j/880548968
>>
>> Summary:
>>
>> https://docs.google.com/document/d/1GY55sXrekVd-aDyRY7uzaX0hMDPyh3T-AL1kUY2TI5M
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>>
>> Wangda Tan <[hidden email]> 于2019年6月28日周五 上午2:57写道:
>>
>> > Hi folks,
>> >
>> > Here's the Hadoop Community Sync Up proposal/schedule:
>> >
>> https://docs.google.com/document/d/1GfNpYKhNUERAEH7m3yx6OfleoF3MqoQk3nJ7xqHD9nY/edit#heading=h.xh4zfwj8ppmn
>> >
>> > And here's calendar file:
>> >
>> >
>> >
>> https://calendar.google.com/calendar/ical/hadoop.community.sync.up%40gmail.com/public/basic.ics
>> >
>> > We gave it a try this week for YARN+MR and Submarine sync, feedbacks
>> from
>> > participants seems pretty good, lots of new information shared during
>> sync
>> > up, and companies are using/developing Hadoop can better know each
>> other.
>> >
>> > Next week there're 4 community sync-ups (Two Submarine for different
>> > timezones, one YARN+MR, one storage), please join to whichever you're
>> > interested:
>> >
>> > [image: image.png]
>> >
>> > Zoom info and notes can be found in the Google calendar invitation.
>> >
>> > Thanks,
>> > Wangda
>> >
>>
>