Fwd: hadoop statistacally logs out while running MR job in hadoop 2.7.2

classic Classic list List threaded Threaded
1 message Options
M G
Reply | Threaded
Open this post in threaded view
|

Fwd: hadoop statistacally logs out while running MR job in hadoop 2.7.2

M G
Hi, 
I am having some trouble with running hadoop jobs in Pseudo cluster and in cluster mode in ubuntu 16.04.

While running a vanila hadoop/hdfs installation - my hadoop user gets logged out and all of the processes that are run by this user are closed.
I don't see anything indicating in logs (/var/log/systemd, journalctl or dmesg) that explains why the user gets logged out.

Seems like I am not the only who has problems with this issue:

Is it possible that some problem around the UserGroupInformation class (that can under some circumstances cause a logout), with maybe some changes in systemd in ubuntu 16.04 can cause this behavior?

The last lines of hadoop log that I get before the logout:
>>>
16/07/13 16:45:37 DEBUG ipc.ProtobufRpcEngine: Call: getJobReport took 4ms
16/07/13 16:45:37 DEBUG security.UserGroupInformation: PrivilegedAction as:hduser (auth:SIMPLE) from:org.apache.hadoop.mapreduce.Job.updateStatus(Job.java:320)
16/07/13 16:45:37 DEBUG ipc.Client: IPC Client (1360814716) connection to mishka/127.0.1.1:37339 from hduser sending #375
16/07/13 16:45:37 DEBUG ipc.Client: IPC Client (1360814716) connection to mishka/127.0.1.1:37339 from hduser got value #375
16/07/13 16:45:37 DEBUG ipc.ProtobufRpcEngine: Call: getJobReport took 2ms
Terminated
hduser@mishka:~$ 16/07/13 16:45:37 DEBUG ipc.Client: stopping client from cache: org.apache.hadoop.ipc.Client@4e7ab839
exit

journalctl:
Jul 12 16:06:44 mishka systemd-logind[978]: Removed session 7.
Jul 12 16:06:44 mishka systemd-logind[978]: Removed session 6.
Jul 12 16:06:44 mishka systemd-logind[978]: Removed session 5.
Jul 12 16:06:44 mishka systemd-logind[978]: Removed session 8.
syslog:
Jul 12 16:06:43 mishka systemd[4172]: Stopped target Default.
Jul 12 16:06:43 mishka systemd[4172]: Reached target Shutdown.
Jul 12 16:06:44 mishka systemd[4172]: Starting Exit the Session...
Jul 12 16:06:44 mishka systemd[4172]: Stopped target Basic System.
Jul 12 16:06:44 mishka systemd[4172]: Stopped target Sockets.
Jul 12 16:06:44 mishka systemd[4172]: Stopped target Paths.
Jul 12 16:06:44 mishka systemd[4172]: Stopped target Timers.
Jul 12 16:06:44 mishka systemd[4172]: Received SIGRTMIN+24 from PID 10101 (kill).
Jul 12 16:06:44 mishka systemd[1]: Stopped User Manager for UID 1001.
Jul 12 16:06:44 mishka systemd[1]: Removed slice User Slice of hduser.