Start client side daemon

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Start client side daemon

Kun Ren
Hi Genius,

I understand that we use the command to start namenode and datanode. But I don't know how HDFS starts client side and creates the Client side object(Like DistributedFileSystem), and client side RPC server? Could you please point it out how HDFS start the client side dameon?
If the client side uses the same RPC server with server side, Can I understand that the client side has to be located at either Namenode or Datanode?

Thanks so much.
Kun
Reply | Threaded
Open this post in threaded view
|

Re: Start client side daemon

Rakesh Radhakrishnan-2
Hi Kun,

HDFS won't start any client side object(for example, DistributedFileSystem). I can say, HDFS Client -> user applications access the file system using the HDFS client, a library that exports the HDFS file system interface. Perhaps, you can visit api docs, https://hadoop.apache.org/docs/r2.6.1/api/org/apache/hadoop/fs/FileSystem.html.

Namenode has RPC server that listens to requests from data nodes, clients. Datanode has RPC Server which will be used for inter data node communications. I think, its worth reading the following to get more information.

Regards,
Rakesh
Intel

On Fri, Jul 22, 2016 at 7:21 PM, Kun Ren <[hidden email]> wrote:
Hi Genius,

I understand that we use the command to start namenode and datanode. But I don't know how HDFS starts client side and creates the Client side object(Like DistributedFileSystem), and client side RPC server? Could you please point it out how HDFS start the client side dameon?
If the client side uses the same RPC server with server side, Can I understand that the client side has to be located at either Namenode or Datanode?

Thanks so much.
Kun

Reply | Threaded
Open this post in threaded view
|

Re: Start client side daemon

Kun Ren
Thanks for your reply. So The clients can be located at any machine that has the HDFS client library, correct?

On Fri, Jul 22, 2016 at 10:50 AM, Rakesh Radhakrishnan <[hidden email]> wrote:
Hi Kun,

HDFS won't start any client side object(for example, DistributedFileSystem). I can say, HDFS Client -> user applications access the file system using the HDFS client, a library that exports the HDFS file system interface. Perhaps, you can visit api docs, https://hadoop.apache.org/docs/r2.6.1/api/org/apache/hadoop/fs/FileSystem.html.

Namenode has RPC server that listens to requests from data nodes, clients. Datanode has RPC Server which will be used for inter data node communications. I think, its worth reading the following to get more information.

Regards,
Rakesh
Intel

On Fri, Jul 22, 2016 at 7:21 PM, Kun Ren <[hidden email]> wrote:
Hi Genius,

I understand that we use the command to start namenode and datanode. But I don't know how HDFS starts client side and creates the Client side object(Like DistributedFileSystem), and client side RPC server? Could you please point it out how HDFS start the client side dameon?
If the client side uses the same RPC server with server side, Can I understand that the client side has to be located at either Namenode or Datanode?

Thanks so much.
Kun


Reply | Threaded
Open this post in threaded view
|

Re: Start client side daemon

Rakesh Radhakrishnan-2
Sorry, could you please tell me, what do you meant by "located" ?

Like I mentioned earlier, userĀ applications will talk to the HDFS via clients. For example, operations like create/open/write/read/delete files etc via client.

Thanks,
Rakesh


On Fri, Jul 22, 2016 at 8:38 PM, Kun Ren <[hidden email]> wrote:
Thanks for your reply. So The clients can be located at any machine that has the HDFS client library, correct?

On Fri, Jul 22, 2016 at 10:50 AM, Rakesh Radhakrishnan <[hidden email]> wrote:
Hi Kun,

HDFS won't start any client side object(for example, DistributedFileSystem). I can say, HDFS Client -> user applications access the file system using the HDFS client, a library that exports the HDFS file system interface. Perhaps, you can visit api docs, https://hadoop.apache.org/docs/r2.6.1/api/org/apache/hadoop/fs/FileSystem.html.

Namenode has RPC server that listens to requests from data nodes, clients. Datanode has RPC Server which will be used for inter data node communications. I think, its worth reading the following to get more information.

Regards,
Rakesh
Intel

On Fri, Jul 22, 2016 at 7:21 PM, Kun Ren <[hidden email]> wrote:
Hi Genius,

I understand that we use the command to start namenode and datanode. But I don't know how HDFS starts client side and creates the Client side object(Like DistributedFileSystem), and client side RPC server? Could you please point it out how HDFS start the client side dameon?
If the client side uses the same RPC server with server side, Can I understand that the client side has to be located at either Namenode or Datanode?

Thanks so much.
Kun



Reply | Threaded
Open this post in threaded view
|

Re: Start client side daemon

Kun Ren
Got it, thanks.

On Fri, Jul 22, 2016 at 11:43 AM, Rakesh Radhakrishnan <[hidden email]> wrote:
Sorry, could you please tell me, what do you meant by "located" ?

Like I mentioned earlier, userĀ applications will talk to the HDFS via clients. For example, operations like create/open/write/read/delete files etc via client.

Thanks,
Rakesh


On Fri, Jul 22, 2016 at 8:38 PM, Kun Ren <[hidden email]> wrote:
Thanks for your reply. So The clients can be located at any machine that has the HDFS client library, correct?

On Fri, Jul 22, 2016 at 10:50 AM, Rakesh Radhakrishnan <[hidden email]> wrote:
Hi Kun,

HDFS won't start any client side object(for example, DistributedFileSystem). I can say, HDFS Client -> user applications access the file system using the HDFS client, a library that exports the HDFS file system interface. Perhaps, you can visit api docs, https://hadoop.apache.org/docs/r2.6.1/api/org/apache/hadoop/fs/FileSystem.html.

Namenode has RPC server that listens to requests from data nodes, clients. Datanode has RPC Server which will be used for inter data node communications. I think, its worth reading the following to get more information.

Regards,
Rakesh
Intel

On Fri, Jul 22, 2016 at 7:21 PM, Kun Ren <[hidden email]> wrote:
Hi Genius,

I understand that we use the command to start namenode and datanode. But I don't know how HDFS starts client side and creates the Client side object(Like DistributedFileSystem), and client side RPC server? Could you please point it out how HDFS start the client side dameon?
If the client side uses the same RPC server with server side, Can I understand that the client side has to be located at either Namenode or Datanode?

Thanks so much.
Kun