About extra containers being allocated in distributed shell example.

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

About extra containers being allocated in distributed shell example.

Smita Deshpande

Hi,

                In YARN distributed shell example, I am setting up my request for containers to the RM using the following call   (I am asking for 9 containers here)

  private ContainerRequest setupContainerAskForRM(Resource capability) {}

                But when actually RMCallbackHandler allocates containers in following call   (I am getting 23 containers here)

  @Override

         public void onContainersAllocated(List<Container> allocatedContainers) {}

 

I am getting extra containers which expire after 600 seconds.

Will these extra launched containers which are not doing anything will have any performance issue in my application?

 

At one point in my application, out of 19K containers 12K containers expired because they were not used. Can anybody suggest any workaround on this or is it a bug?

 

-Smita

Reply | Threaded
Open this post in threaded view
|

RE: About extra containers being allocated in distributed shell example.

Smita Deshpande

Any suggestion/workaround on this one?

 

-Smita

 

From: Smita Deshpande
Sent: Tuesday, September 16, 2014 3:00 PM
To: '[hidden email]'
Subject: About extra containers being allocated in distributed shell example.

 

Hi,

                In YARN distributed shell example, I am setting up my request for containers to the RM using the following call   (I am asking for 9 containers here)

  private ContainerRequest setupContainerAskForRM(Resource capability) {}

                But when actually RMCallbackHandler allocates containers in following call   (I am getting 23 containers here)

  @Override

         public void onContainersAllocated(List<Container> allocatedContainers) {}

 

I am getting extra containers which expire after 600 seconds.

Will these extra launched containers which are not doing anything will have any performance issue in my application?

 

At one point in my application, out of 19K containers 12K containers expired because they were not used. Can anybody suggest any workaround on this or is it a bug?

 

-Smita

Reply | Threaded
Open this post in threaded view
|

RE: About extra containers being allocated in distributed shell example.

Rohith Sharma K S

This looks to be an open issue https://issues.apache.org/jira/i#browse/YARN-1902.

 

Thanks & Regards

Rohith Sharma K S

 

From: Smita Deshpande [mailto:[hidden email]]
Sent: 22 September 2014 10:45
To: [hidden email]
Subject: RE: About extra containers being allocated in distributed shell example.

 

Any suggestion/workaround on this one?

 

-Smita

 

From: Smita Deshpande
Sent: Tuesday, September 16, 2014 3:00 PM
To: '[hidden email]'
Subject: About extra containers being allocated in distributed shell example.

 

Hi,

                In YARN distributed shell example, I am setting up my request for containers to the RM using the following call   (I am asking for 9 containers here)

  private ContainerRequest setupContainerAskForRM(Resource capability) {}

                But when actually RMCallbackHandler allocates containers in following call   (I am getting 23 containers here)

  @Override

         public void onContainersAllocated(List<Container> allocatedContainers) {}

 

I am getting extra containers which expire after 600 seconds.

Will these extra launched containers which are not doing anything will have any performance issue in my application?

 

At one point in my application, out of 19K containers 12K containers expired because they were not used. Can anybody suggest any workaround on this or is it a bug?

 

-Smita

Reply | Threaded
Open this post in threaded view
|

RE: About extra containers being allocated in distributed shell example.

Smita Deshpande

Hi Rohith,

                Thanks for your information.

-Smita

 

From: Rohith Sharma K S [mailto:[hidden email]]
Sent: Tuesday, September 23, 2014 4:16 PM
To: [hidden email]
Subject: RE: About extra containers being allocated in distributed shell example.

 

This looks to be an open issue https://issues.apache.org/jira/i#browse/YARN-1902.

 

Thanks & Regards

Rohith Sharma K S

 

From: Smita Deshpande [[hidden email]]
Sent: 22 September 2014 10:45
To: [hidden email]
Subject: RE: About extra containers being allocated in distributed shell example.

 

Any suggestion/workaround on this one?

 

-Smita

 

From: Smita Deshpande
Sent: Tuesday, September 16, 2014 3:00 PM
To: '[hidden email]'
Subject: About extra containers being allocated in distributed shell example.

 

Hi,

                In YARN distributed shell example, I am setting up my request for containers to the RM using the following call   (I am asking for 9 containers here)

  private ContainerRequest setupContainerAskForRM(Resource capability) {}

                But when actually RMCallbackHandler allocates containers in following call   (I am getting 23 containers here)

  @Override

         public void onContainersAllocated(List<Container> allocatedContainers) {}

 

I am getting extra containers which expire after 600 seconds.

Will these extra launched containers which are not doing anything will have any performance issue in my application?

 

At one point in my application, out of 19K containers 12K containers expired because they were not used. Can anybody suggest any workaround on this or is it a bug?

 

-Smita