[jira] Created: (HADOOP-1298) adding user info to file

classic Classic list List threaded Threaded
296 messages Options
1234 ... 15
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
adding user info to file
------------------------

                 Key: HADOOP-1298
                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
             Project: Hadoop
          Issue Type: New Feature
          Components: dfs, fs
            Reporter: Kurtis Heimerl


I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.

So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.

I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kurtis Heimerl updated HADOOP-1298:
-----------------------------------

    Attachment: hadoop-user-munncha.patch

Here's the current state of the work.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

Re: [jira] Created: (HADOOP-1298) adding user info to file

Kurtis Heimerl
In reply to this post by Jan Høydahl (Jira)
The current state of things is included in the patch associated with this
ticket.

Right now the user and group info is read from config. I was hoping for some
feedback as to this design for the next iteration.

Thanks.

On 4/25/07, Kurtis Heimerl (JIRA) <[hidden email]> wrote:

>
> adding user info to file
> ------------------------
>
>                  Key: HADOOP-1298
>                  URL: https://issues.apache.org/jira/browse/HADOOP-1298
>              Project: Hadoop
>           Issue Type: New Feature
>           Components: dfs, fs
>             Reporter: Kurtis Heimerl
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step
> is this change, which associates user info with files. Following this I'll
> assoicate permissions info, then block methods based on that user info, then
> authorization of the user info.
>
> So, right now i've implemented adding user info to files. I'm looking for
> feedback before I clean this up and make it offical.
>
> I wasn't sure what release, i'm working off trunk.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12491851 ]

Raghu Angadi commented on HADOOP-1298:
--------------------------------------

The patch not include User.java.

Anything you add to each INode is going to increase name node memory by a lot. Make sure that common case where many files and directories that use same permissions will use the same physical object. i.e. memory cost should be close to cost of a reference per INode.





> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kurtis Heimerl updated HADOOP-1298:
-----------------------------------

    Attachment: hadoop-user-munncha.patch

this one has User.java

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12491854 ]

Kurtis Heimerl commented on HADOOP-1298:
----------------------------------------

Fixed the patch, now has user.java.

I am aware of the performance penalties. However, the common case does not have a ton of INodes anyhow, as a distributed file system tends to have a smaller amount of very large files.

Also, at this point it's more of a design specification anyhow. The optimizations can happen later, perhaps keeping a global string area to remove any duplicates. It wouldn't be hard to implement this in a hashtable.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12491900 ]

Owen O'Malley commented on HADOOP-1298:
---------------------------------------

This looks like a reasonable start. I think that in the ClientProtocol that the create method should have User rather than complete. My reasoning is that you want to stop the user quickly, before  they start writing to the file. Actually, all of the namespace operations should take User:
   create
   delete
   exists
   getHints
   getListing
   isDir
   mkDirs
   open
   rename
   setReplication

On a side note, you've got some spurious whitespace changes that should be reverted.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12491937 ]

Sameer Paranjpye commented on HADOOP-1298:
------------------------------------------

Should there be a User parameter added to each method?
Would it make more sense to have a ClientContext object that can be extended to include more fields as we need them?

It might even sense to pass the client context when the session is established and have it persist somewhere for the length of the session, this is trickier though.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492066 ]

Doug Cutting commented on HADOOP-1298:
--------------------------------------

> It might even sense to pass the client context when the session is established [ ...]

Yes, that would make sense, if we had sessions.  But the current RPC framework does not have any notion of session.  So we either need to add it to RPC, or make it a part of this protocol.  We're likely to want user validation for other protocols, so it does make sense to consider somehow adding it to the framework.  But, for now, explicitly passing a context as a parameter is probably right.  Longer-term, to implement sessions we might use a Proxy that wraps a raw RPC proxy, keeping a context with the client that's passed to the server on each call.  Does that sound reasonable?  Or should we add sessions before users?

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492084 ]

Kurtis Heimerl commented on HADOOP-1298:
----------------------------------------

I do plan to add user to all namespace functions, but the current task is simply getting a user associated with a file. So, i've only implemented the functions needed to accomplish that.

I'll look into the whitespace.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492085 ]

Kurtis Heimerl commented on HADOOP-1298:
----------------------------------------

as to the context, that's pretty much what User is. So, if we wanted that changed to ClientContext, i'd feel fine with that change.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kurtis Heimerl updated HADOOP-1298:
-----------------------------------

    Attachment: hadoop-user-munncha.patch

Next iteration.

This has seperated the client side user information from the stuff stored on the server.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492166 ]

Kurtis Heimerl commented on HADOOP-1298:
----------------------------------------

I've sent a new version up, as I'm sure everyone can see.

This implements groups and seperates the client side from the server side.

There are a few unresolved issues. the first of which is the mechanism for putting groups into the server. It uses config and is ugly.

Currently,

fs.groups -> list of groups

for each in this list, a property fs.group.GROUP_NAME should exist.
It has the associated users.

You should be able to call the init method at run time to add new users to the system. It's that or check this file every time a user connects.

Lastly, I need to implement Unix permissions. I don't see any obvious way to do this in java, the closest is 3 integers, one for user, one for groups and one for everyone. Does that sound reasonable?

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kurtis Heimerl updated HADOOP-1298:
-----------------------------------

    Attachment: hadoop-user-munncha.patch4

Nexty iteration, primitive POSIX permissions

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch4
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12492396 ]

Kurtis Heimerl commented on HADOOP-1298:
----------------------------------------

So in this patch I've implemented primitive POSIX permissions on files. The permissions object I inserted in the last patch now has three integer fields. The error checking is off, a code of 708 will still get througn, even though it's not valid.

So, this is nearly finished assuming people don't think of exciting new ideas. Once file permissions go through, then i'll work on implementing restrictions based on these permissions.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch4
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kurtis Heimerl updated HADOOP-1298:
-----------------------------------

    Attachment: hadoop-user-munncha.patch5

Next iteration. This fixed some of the bugs in my permissions implementation, and now implements a dfs -chmod command.

I'm very close to working on unit tests and documentation, so if anyone could take a look at the architectural changes, I'd really appreciated it.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch4, hadoop-user-munncha.patch5
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kurtis Heimerl updated HADOOP-1298:
-----------------------------------

    Attachment: hadoop-user-munncha.patch6

Added chgrp and more functionality to ls. We now have a programmatic way to discover if the file is readable, writable or executable.

there is a known bug with the groups as I don't want to send the server side groups over the wire but the permissions object uses the data structure. The probable fix is to send a FSPermissions object instead.

Feedback would be greatly appreciated on this. I plan to very shortly push this patch.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch4, hadoop-user-munncha.patch5, hadoop-user-munncha.patch6
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12493170 ]

Doug Cutting commented on HADOOP-1298:
--------------------------------------

1. New public methods on public classes must have informative javadoc comments.

2. There should always be a space before '{'.

3. I don't see a need for an "executable" permission.  Read & write should be sufficient for now, no?  We can add this later if we need it.

4. In FSPermissions you have both a 'String permissions' and 'boolean readable; boolean writable;'.  This looks redundant to me. I'd expect something more like:
 
  private static final int OWNER_WRITABLE = ...;
  ...
  private int permissions;
  public int getPermissions();
  public String getPermissionsAsString();
  public boolean isOwnerWritable() {
    return (permissions & OWNER_WRITABLE) != 0;
  }
  public boolean isOwnerReadable() { ... };
  public boolean isGroupWritable(); { ... };
  public boolean isGroupReadable(); { ... };
  public boolean isOtherWritable(); { ... };
  public boolean isOtherReadable(); { ... };

5. Let's rename FSPermissions to be FileStatus, by analogy with the stat() system call.  Then, in the future we can add modified date, etc. to this.  The FileSystem method to access this should then be getFileStatus(Path).

6. ClientContext should be in the dfs package, since it's not used in the generic API.

7. The default permission/status should not be null, but either be abstract or return a permission that permits everything.

8. Don't we need a 'chown' method too?

9. The cache of FileSystem instances (FileSystem.CACHE) should be keyed by Configuration in addition to scheme and authority, so that FileSystem.get(URI,Configuration) will return different instances for the same path if the configuration (i.e., user) differs.

I have not yet had a chance to look closely at the implementation of permissions in DFS.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch4, hadoop-user-munncha.patch5, hadoop-user-munncha.patch6
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kurtis Heimerl updated HADOOP-1298:
-----------------------------------

    Attachment: hadoop-user-munncha.patch7

This patch addresses many of those issues:
Formatting and javadocs are future issues, once the architecture debate is settled.

3-7 are implemented.

I've also implemented a groups method. This will allow a programmatic access to file permissions, if needed.

I believe that either the scheme or the authority includes the user name, allowing it to be multiplexed on that. I'll look over that again soon.

Are there any other concerns over this design? A review of the DFS side of things would be appreciated.

> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch4, hadoop-user-munncha.patch5, hadoop-user-munncha.patch6, hadoop-user-munncha.patch7
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1298) adding user info to file

Jan Høydahl (Jira)
In reply to this post by Jan Høydahl (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12493524 ]

Doug Cutting commented on HADOOP-1298:
--------------------------------------

1. Should FileStatus should also include length, blockSize and isDirectory?  The FileSystem methods getLength(), getBlockSize(), and isDirectory() could then be defined in terms of getStatus().  This is compatible with what unix does internally, but not with java.io.File, and is also probably compatible with most distributed filesystems, where RPCs are at a premium, and it is best to bundle more information into a single call.

2. The six booleans in the FileStatus constructor are error prone.  Rather a single int parameter and constants would be easier to use.  Perhaps something like:

public static final int OWNER_READ = 1;
public static final int OWNER_WRITE = 2;
public static final int OWNER_RW = 3;

public static final int GROUP_READ = 8;
public static final int GROUP_WRITE = 16;
public static final int GROUP_RW = 24;

public static final int OTHER_READ = 64;
public static final int OTHER_WRITE = 128;
public static final int OTHER_RW = 196;

public static final int PERMIT_ALL = 0xffffffff;

This way one can construct a FileStatus with something like:

FileStatus status = new FileStatus(Path, user, group, PERMIT_ALL);
FileStatus status = new FileStatus(Path, user, group, OWNER_RW | GROUP_READ);




> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>         Attachments: hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch, hadoop-user-munncha.patch4, hadoop-user-munncha.patch5, hadoop-user-munncha.patch6, hadoop-user-munncha.patch7
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is this change, which associates user info with files. Following this I'll assoicate permissions info, then block methods based on that user info, then authorization of the user info.
> So, right now i've implemented adding user info to files. I'm looking for feedback before I clean this up and make it offical.
> I wasn't sure what release, i'm working off trunk.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

1234 ... 15