[Erorr:]vieiwng Web UI on EMR cluster

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[Erorr:]vieiwng Web UI on EMR cluster

Divya Gehlot
Hi,
I am on EMR 4.7 with Spark 1.6.1   and Hadoop 2.7.2 
When I am trying to view Any of the web UI of the cluster either hadoop or Spark ,I am getting below error
"

This site can’t be reached

"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.

Would really appreciate the help.

Thanks,
Divya 
Reply | Threaded
Open this post in threaded view
|

Re: [Erorr:]vieiwng Web UI on EMR cluster

Mohammad Tariq
Hi Divya,

Do you you have inbounds enabled on port 50070 of your NN machine. Also, it's a good idea to have the public DNS in your /etc/hosts for proper name resolution.
 



On Tue, Sep 13, 2016 at 9:28 AM, Divya Gehlot <[hidden email]> wrote:
Hi,
I am on EMR 4.7 with Spark 1.6.1   and Hadoop 2.7.2 
When I am trying to view Any of the web UI of the cluster either hadoop or Spark ,I am getting below error
"

This site can’t be reached

"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.

Would really appreciate the help.

Thanks,
Divya 

Reply | Threaded
Open this post in threaded view
|

Re: [Erorr:]vieiwng Web UI on EMR cluster

Jonathan Kelly
I would not recommend opening port 50070 on your cluster, as that would give the entire world access to your data on HDFS. Instead, you should follow the instructions found here to create a secure tunnel to the cluster, through which you can proxy requests to the UIs using a browser plugin like FoxyProxy: https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html

~ Jonathan

On Mon, Sep 12, 2016 at 10:40 PM Mohammad Tariq <[hidden email]> wrote:
Hi Divya,

Do you you have inbounds enabled on port 50070 of your NN machine. Also, it's a good idea to have the public DNS in your /etc/hosts for proper name resolution.
 


 
http://

On Tue, Sep 13, 2016 at 9:28 AM, Divya Gehlot <[hidden email]> wrote:
Hi,
I am on EMR 4.7 with Spark 1.6.1   and Hadoop 2.7.2 
When I am trying to view Any of the web UI of the cluster either hadoop or Spark ,I am getting below error
"

This site can’t be reached

"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.

Would really appreciate the help.

Thanks,
Divya 

Reply | Threaded
Open this post in threaded view
|

Re: [Erorr:]vieiwng Web UI on EMR cluster

Divya Gehlot
Hi,
Thanks all for your prompt response.
I followed the instruction in the docs EMR SSH tunnel  shared by Jonathan.
I am on MAC and set up foxy proxy in my chrome browser 

Divyas-MacBook-Pro:.ssh divyag$ ssh  -N -D 8157 [hidden email]

channel 3: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 5: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 23: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 23: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 8: open failed: administratively prohibited: open failed


What am I missing now ?


Thanks,

Divya 


On 13 September 2016 at 14:23, Jonathan Kelly <[hidden email]> wrote:
I would not recommend opening port 50070 on your cluster, as that would give the entire world access to your data on HDFS. Instead, you should follow the instructions found here to create a secure tunnel to the cluster, through which you can proxy requests to the UIs using a browser plugin like FoxyProxy: https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html

~ Jonathan

On Mon, Sep 12, 2016 at 10:40 PM Mohammad Tariq <[hidden email]> wrote:
Hi Divya,

Do you you have inbounds enabled on port 50070 of your NN machine. Also, it's a good idea to have the public DNS in your /etc/hosts for proper name resolution.
 


 
http://

On Tue, Sep 13, 2016 at 9:28 AM, Divya Gehlot <[hidden email]> wrote:
Hi,
I am on EMR 4.7 with Spark 1.6.1   and Hadoop 2.7.2 
When I am trying to view Any of the web UI of the cluster either hadoop or Spark ,I am getting below error
"

This site can’t be reached

"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.

Would really appreciate the help.

Thanks,
Divya 




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

FoxyProxy_Chrome_Screen Shot.png (220K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: [Erorr:]vieiwng Web UI on EMR cluster

Divya Gehlot
Hi ,
Thank you all..
Hurray ...I am able to view the hadoop web UI now  @ 8088 . even Spark Hisroty server Web UI @ 18080 
But unable to figure out the Spark UI web port ...
Tried with 4044,4040.. ..
getting below error 

This site can’t be reached

How can I find out the Spark port ?

Would really appreciate the help.

Thanks,
Divya 


On 13 September 2016 at 15:09, Divya Gehlot <[hidden email]> wrote:
Hi,
Thanks all for your prompt response.
I followed the instruction in the docs EMR SSH tunnel  shared by Jonathan.
I am on MAC and set up foxy proxy in my chrome browser 

Divyas-MacBook-Pro:.ssh divyag$ ssh  -N -D 8157 [hidden email]

channel 3: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 5: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 23: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 23: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 8: open failed: administratively prohibited: open failed


What am I missing now ?


Thanks,

Divya 


On 13 September 2016 at 14:23, Jonathan Kelly <[hidden email]> wrote:
I would not recommend opening port 50070 on your cluster, as that would give the entire world access to your data on HDFS. Instead, you should follow the instructions found here to create a secure tunnel to the cluster, through which you can proxy requests to the UIs using a browser plugin like FoxyProxy: https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html

~ Jonathan

On Mon, Sep 12, 2016 at 10:40 PM Mohammad Tariq <[hidden email]> wrote:
Hi Divya,

Do you you have inbounds enabled on port 50070 of your NN machine. Also, it's a good idea to have the public DNS in your /etc/hosts for proper name resolution.
 


 
http://

On Tue, Sep 13, 2016 at 9:28 AM, Divya Gehlot <[hidden email]> wrote:
Hi,
I am on EMR 4.7 with Spark 1.6.1   and Hadoop 2.7.2 
When I am trying to view Any of the web UI of the cluster either hadoop or Spark ,I am getting below error
"

This site can’t be reached

"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.

Would really appreciate the help.

Thanks,
Divya 



Reply | Threaded
Open this post in threaded view
|

Re: [Erorr:]vieiwng Web UI on EMR cluster

Jonathan Kelly
Yes, Spark on EMR runs on YARN, so there is only a Spark UI when a Spark app is running. To expand on what Natu says, the best way to view the Spark UI for both running and completed Spark apps is to start from the YARN ResourceManager UI (port 8088) and to click the "Application Master" link (for running apps) or "History" link (for completed apps).

~ Jonathan

On Tue, Sep 13, 2016 at 2:30 AM Natu Lauchande <[hidden email]> wrote:
Hi,

I think the spark UI will be accessible whenever you launch a spark app in the cluster it should be the Application Tracker link.


Regards,
Natu

On Tue, Sep 13, 2016 at 9:37 AM, Divya Gehlot <[hidden email]> wrote:
Hi ,
Thank you all..
Hurray ...I am able to view the hadoop web UI now  @ 8088 . even Spark Hisroty server Web UI @ 18080 
But unable to figure out the Spark UI web port ...
Tried with 4044,4040.. ..
getting below error 

This site can’t be reached

How can I find out the Spark port ?

Would really appreciate the help.

Thanks,
Divya 


On 13 September 2016 at 15:09, Divya Gehlot <[hidden email]> wrote:
Hi,
Thanks all for your prompt response.
I followed the instruction in the docs EMR SSH tunnel  shared by Jonathan.
I am on MAC and set up foxy proxy in my chrome browser 

Divyas-MacBook-Pro:.ssh divyag$ ssh  -N -D 8157 [hidden email]

channel 3: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 3: open failed: connect failed: Connection refused

channel 4: open failed: connect failed: Connection refused

channel 5: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 23: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 23: open failed: connect failed: Connection refused

channel 22: open failed: connect failed: Connection refused

channel 8: open failed: administratively prohibited: open failed


What am I missing now ?


Thanks,

Divya 


On 13 September 2016 at 14:23, Jonathan Kelly <[hidden email]> wrote:
I would not recommend opening port 50070 on your cluster, as that would give the entire world access to your data on HDFS. Instead, you should follow the instructions found here to create a secure tunnel to the cluster, through which you can proxy requests to the UIs using a browser plugin like FoxyProxy: https://docs.aws.amazon.com/ElasticMapReduce/latest/ManagementGuide/emr-ssh-tunnel.html

~ Jonathan

On Mon, Sep 12, 2016 at 10:40 PM Mohammad Tariq <[hidden email]> wrote:
Hi Divya,

Do you you have inbounds enabled on port 50070 of your NN machine. Also, it's a good idea to have the public DNS in your /etc/hosts for proper name resolution.
 


 
http://

On Tue, Sep 13, 2016 at 9:28 AM, Divya Gehlot <[hidden email]> wrote:
Hi,
I am on EMR 4.7 with Spark 1.6.1   and Hadoop 2.7.2 
When I am trying to view Any of the web UI of the cluster either hadoop or Spark ,I am getting below error
"

This site can’t be reached

"
Has anybody using EMR and able to view WebUI .
Could you please share the steps.

Would really appreciate the help.

Thanks,
Divya