[jira] Created: (HADOOP-1511) hbase unit tests are slow

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

[jira] Created: (HADOOP-1511) hbase unit tests are slow

Tim Allison (Jira)
hbase unit tests are slow
-------------------------

                 Key: HADOOP-1511
                 URL: https://issues.apache.org/jira/browse/HADOOP-1511
             Project: Hadoop
          Issue Type: Bug
          Components: contrib/hbase
            Reporter: Doug Cutting
            Priority: Minor


The hbase unit tests take a *long* time to run, now dominating Hadoop unit test runs.  Is there any way to make them run a bit faster?

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1511) hbase unit tests are slow

Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack updated HADOOP-1511:
--------------------------

    Attachment: speedupunittests.patch

Patch makes hbase tests run in about a quarter of the time they used to take (<5mins rather than the ~19mins as measured here on my desktop).  Close the issue is this is enough of an improvement.

Here's the commit message:

Add a hbase-site.xml for use running unit tests.  Makes cluster more
responsive and retry less often. Defaults are too long for
single-process minihbasecluster.

* src/contrib/hbase/src/test/hbase-site.xml
    Added. Shorten pauses between message checks and lessen number
    of retries.
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/MiniHBaseCluster.java
    (startRegionServer): Added.  Make it accessible to unit tests so
    they can start region servers after test startup.
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestRegionServerAbort.java
    Removed its guts.  Test has been moved to the imperfectly named
    TestCleanRegionServerExit.java.  Later, when mr hbase-committer, jimk,
    gets back, will remove this file and rename TestCleanRegionServerExit.
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestCleanRegionServerExit.java
    Removed waits for conditions that are never checked.
    (testRegionServerAbort): Added.
* src/contrib/hbase/src/test/org/apache/hadoop/hbase/TestHClient.java
    Remove no-longer-relevant comment.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HConstants.java
    (COLUMN_FAMILY_ARRAY): Added.
* src/contrib/hbase/src/java/org/apache/hadoop/hbase/HClient.java
    Removed unnecessary setting of info to null.
    (getRegionLocation): Allow passing in null for row ('null' is
    the first row).
* src/contrib/hbase/build.xml
    Add the src/test directory to CLASSPATH so our test hadoop-site.xml
    gets picked up when unit tests run.


> hbase unit tests are slow
> -------------------------
>
>                 Key: HADOOP-1511
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1511
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/hbase
>            Reporter: Doug Cutting
>            Priority: Minor
>         Attachments: speedupunittests.patch
>
>
> The hbase unit tests take a *long* time to run, now dominating Hadoop unit test runs.  Is there any way to make them run a bit faster?

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1511) hbase unit tests are slow

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

stack updated HADOOP-1511:
--------------------------

    Status: Patch Available  (was: Open)

Trying patch against hudson.  Works locally but perhaps the test configuration is inappropriate for hadoop build box.

> hbase unit tests are slow
> -------------------------
>
>                 Key: HADOOP-1511
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1511
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/hbase
>            Reporter: Doug Cutting
>            Priority: Minor
>         Attachments: speedupunittests.patch
>
>
> The hbase unit tests take a *long* time to run, now dominating Hadoop unit test runs.  Is there any way to make them run a bit faster?

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1511) hbase unit tests are slow

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507016 ]

Hadoop QA commented on HADOOP-1511:
-----------------------------------

+1

http://issues.apache.org/jira/secure/attachment/12360316/speedupunittests.patch applied and successfully tested against trunk revision r549577.

Test results:   http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/318/testReport/
Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/318/console

> hbase unit tests are slow
> -------------------------
>
>                 Key: HADOOP-1511
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1511
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/hbase
>            Reporter: Doug Cutting
>            Priority: Minor
>         Attachments: speedupunittests.patch
>
>
> The hbase unit tests take a *long* time to run, now dominating Hadoop unit test runs.  Is there any way to make them run a bit faster?

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Updated: (HADOOP-1511) hbase unit tests are slow

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

     [ https://issues.apache.org/jira/browse/HADOOP-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Doug Cutting updated HADOOP-1511:
---------------------------------

       Resolution: Fixed
    Fix Version/s: 0.14.0
           Status: Resolved  (was: Patch Available)

I just committed this.  Thanks, Michael!

> hbase unit tests are slow
> -------------------------
>
>                 Key: HADOOP-1511
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1511
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/hbase
>            Reporter: Doug Cutting
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: speedupunittests.patch
>
>
> The hbase unit tests take a *long* time to run, now dominating Hadoop unit test runs.  Is there any way to make them run a bit faster?

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply | Threaded
Open this post in threaded view
|

[jira] Commented: (HADOOP-1511) hbase unit tests are slow

Tim Allison (Jira)
In reply to this post by Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/HADOOP-1511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12507210 ]

Hudson commented on HADOOP-1511:
--------------------------------

Integrated in Hadoop-Nightly #132 (See [http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/132/])

> hbase unit tests are slow
> -------------------------
>
>                 Key: HADOOP-1511
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1511
>             Project: Hadoop
>          Issue Type: Bug
>          Components: contrib/hbase
>            Reporter: Doug Cutting
>            Priority: Minor
>             Fix For: 0.14.0
>
>         Attachments: speedupunittests.patch
>
>
> The hbase unit tests take a *long* time to run, now dominating Hadoop unit test runs.  Is there any way to make them run a bit faster?

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.