shading killing trunk build times

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

shading killing trunk build times

Steve Loughran-3
a clean build with no tests, javadocs, is now taking 7+ min locally, with most of that time going into the packaging work

[INFO] Apache Hadoop Client API ........................... SUCCESS [01:50 min]
[INFO] Apache Hadoop Client Runtime ....................... SUCCESS [01:19 min]
[INFO] Apache Hadoop Client Packaging Invariants .......... SUCCESS [  0.332 s]
[INFO] Apache Hadoop Client Test Minicluster .............. SUCCESS [02:52 min]

This really hurts when you are trying to build and test across modules, where I want to do a change in say, a hadoop-common test, then run that test in hdfs and aws modules. I don't want or need this client stuff, and all it is doing is taking up time and forcing me into contrived bits where I have to explicitly only build the modules I want, which gets dangerous once you start switching branches


Can we make these a profile? Even if it's on by default and something that those of us try

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: shading killing trunk build times

Daniel Templeton
For most purposes, you can get by with "mvn clean process-resources
process-test-resources compile -DskipTests -T 1C".  (Hat tip to Karthik)

Daniel

On 1/18/17 6:27 AM, Steve Loughran wrote:

> a clean build with no tests, javadocs, is now taking 7+ min locally, with most of that time going into the packaging work
>
> [INFO] Apache Hadoop Client API ........................... SUCCESS [01:50 min]
> [INFO] Apache Hadoop Client Runtime ....................... SUCCESS [01:19 min]
> [INFO] Apache Hadoop Client Packaging Invariants .......... SUCCESS [  0.332 s]
> [INFO] Apache Hadoop Client Test Minicluster .............. SUCCESS [02:52 min]
>
> This really hurts when you are trying to build and test across modules, where I want to do a change in say, a hadoop-common test, then run that test in hdfs and aws modules. I don't want or need this client stuff, and all it is doing is taking up time and forcing me into contrived bits where I have to explicitly only build the modules I want, which gets dangerous once you start switching branches
>
>
> Can we make these a profile? Even if it's on by default and something that those of us try
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: shading killing trunk build times

Arun Suresh-3
Just filed https://issues.apache.org/jira/browse/HADOOP-13999
To add a mvn flag and profile that can disable shading

On Wed, Jan 18, 2017 at 9:46 AM, Daniel Templeton <[hidden email]>
wrote:

> For most purposes, you can get by with "mvn clean process-resources
> process-test-resources compile -DskipTests -T 1C".  (Hat tip to Karthik)
>
> Daniel
>
>
> On 1/18/17 6:27 AM, Steve Loughran wrote:
>
>> a clean build with no tests, javadocs, is now taking 7+ min locally, with
>> most of that time going into the packaging work
>>
>> [INFO] Apache Hadoop Client API ........................... SUCCESS
>> [01:50 min]
>> [INFO] Apache Hadoop Client Runtime ....................... SUCCESS
>> [01:19 min]
>> [INFO] Apache Hadoop Client Packaging Invariants .......... SUCCESS [
>> 0.332 s]
>> [INFO] Apache Hadoop Client Test Minicluster .............. SUCCESS
>> [02:52 min]
>>
>> This really hurts when you are trying to build and test across modules,
>> where I want to do a change in say, a hadoop-common test, then run that
>> test in hdfs and aws modules. I don't want or need this client stuff, and
>> all it is doing is taking up time and forcing me into contrived bits where
>> I have to explicitly only build the modules I want, which gets dangerous
>> once you start switching branches
>>
>>
>> Can we make these a profile? Even if it's on by default and something
>> that those of us try
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>