[jira] Created: (HADOOP-1509) Open HRegionServer/HClient for extension

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
Open HRegionServer/HClient for extension
----------------------------------------

                 Key: HADOOP-1509
                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
             Project: Hadoop
          Issue Type: Improvement
          Components: contrib/hbase
    Affects Versions: 0.14.0
            Reporter: James Kennedy
            Priority: Minor


For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.

Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.

However, for extensibility it would be nice if the servers/client were easy to extend.

Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.


--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Kennedy updated HADOOP-1509:
----------------------------------

    Fix Version/s: 0.14.0
           Status: Patch Available  (was: Open)

Index: /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HClient.java
===================================================================
--- /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HClient.java (revision 549130)
+++ /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HClient.java (working copy)
@@ -62,7 +62,7 @@
   /*
    * Data structure that holds current location for a region and its info.
    */
-  static class RegionLocation {
+  protected static class RegionLocation {
     HRegionInfo regionInfo;
     HServerAddress serverAddress;
 
@@ -76,6 +76,22 @@
       return "address: " + this.serverAddress.toString() + ", regioninfo: " +
         this.regionInfo;
     }
+    
+    public HRegionInfo getRegionInfo(){
+      return regionInfo;
+    }
+
+    public void setRegionInfo(HRegionInfo regionInfo){
+      this.regionInfo = regionInfo;
+    }
+
+    public HServerAddress getServerAddress(){
+      return serverAddress;
+    }
+
+    public void setServerAddress(HServerAddress serverAddress){
+      this.serverAddress = serverAddress;
+    }
   }
   
   // Map tableName -> (Map startRow -> (HRegionInfo, HServerAddress)
@@ -116,7 +132,7 @@
     this.rand = new Random();
   }
   
-  private void handleRemoteException(RemoteException e) throws IOException {
+  protected void handleRemoteException(RemoteException e) throws IOException {
     String msg = e.getMessage();
     if(e.getClassName().equals("org.apache.hadoop.hbase.InvalidColumnNameException")) {
       throw new InvalidColumnNameException(msg);
@@ -143,7 +159,7 @@
   
   /* Find the address of the master and connect to it
    */
-  private void checkMaster() throws MasterNotRunningException {
+  protected void checkMaster() throws MasterNotRunningException {
     if (this.master != null) {
       return;
     }
@@ -531,7 +547,7 @@
    * @param tableName - the table name to be checked
    * @throws IllegalArgumentException - if the table name is reserved
    */
-  private void checkReservedTableName(Text tableName) {
+  protected void checkReservedTableName(Text tableName) {
     if(tableName.equals(ROOT_TABLE_NAME)
         || tableName.equals(META_TABLE_NAME)) {
       
@@ -547,7 +563,7 @@
   //////////////////////////////////////////////////////////////////////////////
   // Client API
   //////////////////////////////////////////////////////////////////////////////
-
+  
   /**
    * Loads information so that a table can be manipulated.
    *
@@ -558,8 +574,21 @@
     if(tableName == null || tableName.getLength() == 0) {
       throw new IllegalArgumentException("table name cannot be null or zero length");
     }
-    this.tableServers = tablesToServers.get(tableName);
-    if (this.tableServers == null ) {
+    this.tableServers = getTableServers(tableName);
+  }
+  
+  /**
+   * Gets the servers of the given table.
+   *
+   * @param tableName - the table to be located
+   * @throws IOException - if the table can not be located after retrying
+   */
+  protected synchronized SortedMap<Text, RegionLocation> getTableServers(Text tableName) throws IOException {
+    if(tableName == null || tableName.getLength() == 0) {
+      throw new IllegalArgumentException("table name cannot be null or zero length");
+    }
+    SortedMap<Text, RegionLocation> serverResult  = tablesToServers.get(tableName);
+    if (serverResult == null ) {
       if (LOG.isDebugEnabled()) {
         LOG.debug("No servers for " + tableName + ". Doing a find...");
       }
@@ -565,8 +594,9 @@
       }
       // We don't know where the table is.
       // Load the information from meta.
-      this.tableServers = findServersForTable(tableName);
+      serverResult = findServersForTable(tableName);
     }
+    return serverResult;
   }
 
   /*
@@ -836,7 +866,7 @@
    * @param regionServer - the server to connect to
    * @throws IOException
    */
-  synchronized HRegionInterface getHRegionConnection(HServerAddress regionServer)
+  protected synchronized HRegionInterface getHRegionConnection(HServerAddress regionServer)
       throws IOException {
 
       // See if we already have a connection
@@ -916,7 +946,7 @@
    * @param row Row to find.
    * @return Location of row.
    */
-  synchronized RegionLocation getRegionLocation(Text row) {
+  protected synchronized RegionLocation getRegionLocation(Text row) {
     if(row == null || row.getLength() == 0) {
       throw new IllegalArgumentException("row key cannot be null or zero length");
     }
@@ -1554,6 +1584,20 @@
     }
     
     return errCode;
+  }  
+
+  /**
+   * @return the map of opened servers
+   */
+  protected TreeMap<String, HRegionInterface> getOpenServers(){
+    return servers;
+  }
+
+  /**
+   * @return the configuration for this server
+   */
+  public Configuration getConf(){
+    return conf;
   }
   
   /**
@@ -1565,4 +1609,5 @@
     int errCode = (new HClient(c)).doCommandLine(args);
     System.exit(errCode);
   }
+
 }
Index: /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegion.java
===================================================================
--- /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegion.java (revision 549130)
+++ /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegion.java (working copy)
@@ -55,7 +55,7 @@
  * regionName is a unique identifier for this HRegion. (startKey, endKey]
  * defines the keyspace for this HRegion.
  */
-class HRegion implements HConstants {
+public class HRegion implements HConstants {
   static String SPLITDIR = "splits";
   static String MERGEDIR = "merges";
   static String TMPREGION_PREFIX = "tmpregion_";
@@ -298,7 +298,7 @@
    *
    * @throws IOException
    */
-  HRegion(Path rootDir, HLog log, FileSystem fs, Configuration conf,
+  public HRegion(Path rootDir, HLog log, FileSystem fs, Configuration conf,
       HRegionInfo regionInfo, Path initialFiles)
   throws IOException {
     
@@ -386,7 +386,7 @@
    * This method could take some time to execute, so don't call it from a
    * time-sensitive thread.
    */
-  Vector<HStoreFile> close() throws IOException {
+  public Vector<HStoreFile> close() throws IOException {
     lock.obtainWriteLock();
     try {
       boolean shouldClose = false;
@@ -548,43 +548,43 @@
   // HRegion accessors
   //////////////////////////////////////////////////////////////////////////////
 
-  Text getStartKey() {
+  public Text getStartKey() {
     return regionInfo.startKey;
   }
   
-  Text getEndKey() {
+  public Text getEndKey() {
     return regionInfo.endKey;
   }
   
-  long getRegionId() {
+  public long getRegionId() {
     return regionInfo.regionId;
   }
 
-  Text getRegionName() {
+  public Text getRegionName() {
     return regionInfo.regionName;
   }
   
-  Path getRootDir() {
+  public Path getRootDir() {
     return rootDir;
   }
 
-  HTableDescriptor getTableDesc() {
+  public HTableDescriptor getTableDesc() {
     return regionInfo.tableDesc;
   }
   
-  HLog getLog() {
+  public HLog getLog() {
     return log;
   }
   
-  Configuration getConf() {
+  public Configuration getConf() {
     return conf;
   }
   
-  Path getRegionDir() {
+  public Path getRegionDir() {
     return regiondir;
   }
   
-  FileSystem getFilesystem() {
+  public FileSystem getFilesystem() {
     return fs;
   }
 
@@ -973,7 +973,7 @@
    * Return an iterator that scans over the HRegion, returning the indicated
    * columns.  This Iterator must be closed by the caller.
    */
-  HInternalScannerInterface getScanner(Text[] cols, Text firstRow)
+  public HInternalScannerInterface getScanner(Text[] cols, Text firstRow)
   throws IOException {
     lock.obtainReadLock();
     try {
@@ -1011,7 +1011,7 @@
    * @return lockid
    * @see #put(long, Text, BytesWritable)
    */
-  long startUpdate(Text row) throws IOException {
+  public long startUpdate(Text row) throws IOException {
     // We obtain a per-row lock, so other clients will block while one client
     // performs an update.  The read lock is released by the client calling
     // #commit or #abort or if the HRegionServer lease on the lock expires.
@@ -1029,7 +1029,7 @@
    * This method really just tests the input, then calls an internal localput()
    * method.
    */
-  void put(long lockid, Text targetCol, byte [] val) throws IOException {
+  public void put(long lockid, Text targetCol, byte [] val) throws IOException {
     if (DELETE_BYTES.compareTo(val) == 0) {
       throw new IOException("Cannot insert value: " + val);
     }
@@ -1039,7 +1039,7 @@
   /**
    * Delete a value or write a value. This is a just a convenience method for put().
    */
-  void delete(long lockid, Text targetCol) throws IOException {
+  public void delete(long lockid, Text targetCol) throws IOException {
     localput(lockid, targetCol, DELETE_BYTES.get());
   }
 
@@ -1055,7 +1055,7 @@
    * @param val Value to enter into cell
    * @throws IOException
    */
-  void localput(final long lockid, final Text targetCol,
+  public void localput(final long lockid, final Text targetCol,
     final byte [] val)
   throws IOException {
     checkColumn(targetCol);
@@ -1090,7 +1090,7 @@
    * writes associated with the given row-lock.  These values have not yet
    * been placed in memcache or written to the log.
    */
-  void abort(long lockid) throws IOException {
+  public void abort(long lockid) throws IOException {
     Text row = getRowFromLock(lockid);
     if(row == null) {
       throw new LockException("No write lock for lockid " + lockid);
@@ -1124,7 +1124,7 @@
    * @param lockid Lock for row we're to commit.
    * @throws IOException
    */
-  void commit(final long lockid) throws IOException {
+  public void commit(final long lockid) throws IOException {
     // Remove the row from the pendingWrites list so
     // that repeated executions won't screw this up.
     Text row = getRowFromLock(lockid);
Index: /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionInfo.java
===================================================================
--- /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionInfo.java (revision 549130)
+++ /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionInfo.java (working copy)
@@ -139,6 +139,76 @@
     this.regionName.readFields(in);
     this.offLine = in.readBoolean();
   }
+  
+  /**
+   * @return the endKey
+   */
+  public Text getEndKey(){
+    return endKey;
+  }
+
+  /**
+   * @param endKey the endKey to set
+   */
+  public void setEndKey(Text endKey){
+    this.endKey = endKey;
+  }
+
+  /**
+   * @return the regionId
+   */
+  public long getRegionId(){
+    return regionId;
+  }
+
+  /**
+   * @param regionId the regionId to set
+   */
+  public void setRegionId(long regionId){
+    this.regionId = regionId;
+  }
+
+  /**
+   * @return the regionName
+   */
+  public Text getRegionName(){
+    return regionName;
+  }
+
+  /**
+   * @param regionName the regionName to set
+   */
+  public void setRegionName(Text regionName){
+    this.regionName = regionName;
+  }
+
+  /**
+   * @return the startKey
+   */
+  public Text getStartKey(){
+    return startKey;
+  }
+
+  /**
+   * @param startKey the startKey to set
+   */
+  public void setStartKey(Text startKey){
+    this.startKey = startKey;
+  }
+
+  /**
+   * @return the tableDesc
+   */
+  public HTableDescriptor getTableDesc(){
+    return tableDesc;
+  }
+
+  /**
+   * @param tableDesc the tableDesc to set
+   */
+  public void setTableDesc(HTableDescriptor tableDesc){
+    this.tableDesc = tableDesc;
+  }
 
   //////////////////////////////////////////////////////////////////////////////
   // Comparable
Index: /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionServer.java
===================================================================
--- /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionServer.java (revision 549130)
+++ /opt/eclipse/sandbox2/Hadoop/src/contrib/hbase/src/java/org/apache/hadoop/hbase/HRegionServer.java (working copy)
@@ -468,7 +468,7 @@
    * Sets a flag that will cause all the HRegionServer threads to shut down
    * in an orderly fashion.
    */
-  synchronized void stop() {
+  public synchronized void stop() {
     stopRequested = true;
     notifyAll();                        // Wakes run() if it is sleeping
   }
@@ -1079,7 +1079,7 @@
   }
 
   /**
-   * Private utility method for safely obtaining an HRegion handle.
+   * Protected utility method for safely obtaining an HRegion handle.
    * @param regionName Name of online {@link HRegion} to return
    * @return {@link HRegion} for <code>regionName</code>
    * @throws NotServingRegionException
@@ -1084,7 +1084,7 @@
    * @return {@link HRegion} for <code>regionName</code>
    * @throws NotServingRegionException
    */
-  private HRegion getRegion(final Text regionName)
+  protected HRegion getRegion(final Text regionName)
   throws NotServingRegionException {
     return getRegion(regionName, false);
   }
@@ -1090,7 +1090,7 @@
   }
   
   /**
-   * Private utility method for safely obtaining an HRegion handle.
+   * Protected utility method for safely obtaining an HRegion handle.
    * @param regionName Name of online {@link HRegion} to return
    * @param checkRetiringRegions Set true if we're to check retiring regions
    * as well as online regions.
@@ -1097,7 +1097,7 @@
    * @return {@link HRegion} for <code>regionName</code>
    * @throws NotServingRegionException
    */
-  private HRegion getRegion(final Text regionName,
+  protected HRegion getRegion(final Text regionName,
       final boolean checkRetiringRegions)
   throws NotServingRegionException {
     HRegion region = null;


> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Kennedy updated HADOOP-1509:
----------------------------------

    Comment: was deleted

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Kennedy updated HADOOP-1509:
----------------------------------

    Attachment: openForExtension.patch

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506636 ]

Hadoop QA commented on HADOOP-1509:
-----------------------------------

-1, could not apply patch.

The patch command could not apply the latest attachment http://issues.apache.org/jira/secure/attachment/12360222/openForExtension.patch as a patch to trunk revision r548794.

Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/313/console

Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Kennedy updated HADOOP-1509:
----------------------------------

    Status: Open  (was: Patch Available)

Oops, submitted patch prematurely.

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Kennedy updated HADOOP-1509:
----------------------------------

    Attachment: openForExtension-v2.patch

Followed procedure better this time. New patch includes relative paths and update to CHANGES.txt

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Assigned: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Cutting reassigned HADOOP-1509:
------------------------------------

    Assignee: James Kennedy

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506720 ]

stack commented on HADOOP-1509:
-------------------------------

James:

+ Why make HClient.getConf public rather than protected? (You refer to 'server' when you mean 'client' in the javadoc)
+ Regards HClient.getOpenServers, is it wise letting out the servers data structure?  Access probably needs to be synchronized.  Would wrapping the servers map returned by getOpenServers with an Arrays.unmodifiableMap be obnoxious?  (Probably)  Better might be adding (synchronized) 'servers' manipulating methods to the base HClient class?
+ Does HRegion need to be public?  It would be better if HRegions were only manipulated by HRegionServers since instantation of a HRegion will change the on-hdfs state, a problem if two instances of HRegion are pointed at same region  (But maybe your subclass of HRegionServer cannot use HRegion unless its public?)  Same for its internal scanner.

Otherwise patch looks good.  It applied cleanly and passed all tests.

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12506924 ]

James Kennedy commented on HADOOP-1509:
---------------------------------------

+HClient.getConf doesn't have to be public though I see no harm in it.  Supposedly the client user already has a handle to the conf object anyway though,  i'll make it protected.

+perhaps exposing HClient.getOpenServers was overzealous.  The only place where I use it in my extension is in where I override HClient.getHRegionConnection().  The only reason this override is necessary is because to make RPC work with added methods in my HRegionServer extension (call it QRegionServer), i need to inject the QRegionServerInterface.  

e.g.
server = (QRegionInterface) RPC.waitForProxy(
                                        QRegionInterface.class, QRegionInterface.versionID,
                                        regionServer.getInetSocketAddress(), getConf());

Incidentally that is also the only place where I use the, now protected, getConf() method.

If there was a configuration parameter that specified an HRegionServer extending interface, and the base impl of HClient.getHRegionConnection() used that to initialize the server proxy, then all that would become unnecessary.  What do you think of that idea? I guess a similar pattern should exist for injecting specific HClient impls into HMaster too.

+About public HRegion and its scanner... I'm not sure how to make it accessible only to HRegionServer subclasses without making it an inner class. I have a particular need for it where my QRegionServer perfroms scans on its own HRegions... put perhaps there isn't a general need for that and I could remove that from this patch. Or maybe HRegion manipulation is a likely need for HRegionServer extensions... what do you think?





> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507061 ]

stack commented on HADOOP-1509:
-------------------------------

"...perhaps exposing HClient.getOpenServers was overzealous....If there was a configuration parameter that specified an HRegionServer extending interface, and the base impl of HClient.getHRegionConnection() used that to initialize the server proxy, then all that would become unnecessary. What do you think of that idea?"

Sounds like a good idea.

"I guess a similar pattern should exist for injecting specific HClient impls into HMaster too."

Unfortunately there is no HClientInterface currently as there is a HRegionInterface (or HMasterInterface) so it would be a bit of work and HClient is still evolving so keeping up interfaces and their implementation will be a bit of a pain -- but its still a good idea.

"... I'm not sure how to make it accessible only to HRegionServer subclasses without making it an inner class. I have a particular need for it where my QRegionServer perfroms scans on its own HRegions... put perhaps there isn't a general need for that and I could remove that from this patch. Or maybe HRegion manipulation is a likely need for HRegionServer extensions... what do you think?"

A region server that can't manipulate regions is like a herder without his sheep.  Lets just make HRegion public (Any future table-repair tool would need to have public access to HRegions.)

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Kennedy updated HADOOP-1509:
----------------------------------

    Attachment: openForExtension-v3.patch

Ok. Attached patch v3.

HClient.getHRegionConnection() no longer needs to be extended since it is aware of the configuration property "hbase.regionserver.class" which is set in hbase-default.xml to HRegionInterface class name.  

I nixed getOpenServers().

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension-v3.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507522 ]

stack commented on HADOOP-1509:
-------------------------------

I like the way you convert comments to javadoc when method goes from private to protected and thanks for adding the missing version check when going to a new region server.  

One question.  Why make RegionLocation and HRegionInfo mutable?  And HRegion.localput probably shouldn't be public?

The patch didn't apply cleanly for me.  HClient has changed since your add of v3 to this issue.

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension-v3.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Kennedy updated HADOOP-1509:
----------------------------------

    Attachment: openForExtension-v4.patch

Sorry about that.  Mutabability not required, I used eclipse to generate getters and setters when I should have specified just getters.

New patch created after full update as of this morning. Removed those setters. HRegion.localput() now package visible only again.

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension-v3.patch, openForExtension-v4.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507930 ]

stack commented on HADOOP-1509:
-------------------------------

+1

Patch looks good to me and it built and passed all tests locally.  I'd suggest this issue is ripe for moving its state to 'patch available' so it gets picked up by hudson.

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension-v3.patch, openForExtension-v4.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Kennedy updated HADOOP-1509:
----------------------------------

    Status: Patch Available  (was: Open)

Cool, thanks.

So what's the process for that step? I figure I just do 'Submit Patch'...

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension-v3.patch, openForExtension-v4.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507940 ]

Hadoop QA commented on HADOOP-1509:
-----------------------------------

-1, new javadoc warnings

The javadoc tool appears to have generated warning messages when testing the latest attachment http://issues.apache.org/jira/secure/attachment/12360489/openForExtension-v4.patch against trunk revision r549977.

Test results:   http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/325/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/325/console

Please note that this message is automatically generated and may represent a problem with the automation system and not the patch.

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension-v3.patch, openForExtension-v4.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507941 ]

stack commented on HADOOP-1509:
-------------------------------

Yep (See 'Contributing your work' at end of this page http://wiki.apache.org/lucene-hadoop/HowToContribute).

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension-v3.patch, openForExtension-v4.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Kennedy updated HADOOP-1509:
----------------------------------

    Status: Open  (was: Patch Available)

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension-v3.patch, openForExtension-v4.patch, openForExtension-v5.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1509) Open HRegionServer/HClient for extension

Markus Jelsma (Jira)
In reply to this post by Markus Jelsma (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Kennedy updated HADOOP-1509:
----------------------------------

    Status: Patch Available  (was: Open)

> Open HRegionServer/HClient for extension
> ----------------------------------------
>
>                 Key: HADOOP-1509
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1509
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: contrib/hbase
>    Affects Versions: 0.14.0
>            Reporter: James Kennedy
>            Assignee: James Kennedy
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: openForExtension-v2.patch, openForExtension-v3.patch, openForExtension-v4.patch, openForExtension-v5.patch, openForExtension.patch
>
>
> For what i'm doing I found it necessary to extend HRegionServer/HRegion/HClient for some custom functionality.
> Following good Java practice I see that the HBase code as been programmed defensively, keeping stuff private as much as possible.
> However, for extensibility it would be nice if the servers/client were easy to extend.
> Attached is a patch that makes several methods protected instead of private, adds getters to fields of inner classes, and some other modifications i found were useful for some simple extension code.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

12