The Gradle train.

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

The Gradle train.

Mark Miller-3
https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

--
Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Erick Erickson
Go for it! I’ve been amazed by the effort you’ve put in it, and it’ll be good for some of the load to be distributed.

Of course I’ll need to learn new tricks (something about old dogs in here)…..

Erick

> On Aug 15, 2019, at 5:23 PM, Mark Miller <[hidden email]> wrote:
>
> https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
>
> Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.
>
> I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.
>
> This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).
>
> I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.
>
> I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.
>
> This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)
>
> --
> - Mark
>
> http://about.me/markrmiller


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Dawid Weiss-2
In reply to this post by Mark Miller-3
I'm +1. Gradle can be annoying like hell, but it's much faster and
flexible than any other build tool I've tried.
D.

On Thu, Aug 15, 2019 at 11:24 PM Mark Miller <[hidden email]> wrote:

>
> https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
>
> Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.
>
> I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.
>
> This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).
>
> I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.
>
> I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.
>
> This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)
>
> --
> - Mark
>
> http://about.me/markrmiller

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Andrzej Białecki-2
In reply to this post by Mark Miller-3
+1 for landing this in master soon. Thanks Mark for your relentless efforts to make this happen!

On 15 Aug 2019, at 23:23, Mark Miller <[hidden email]> wrote:

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

--

Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Jan Høydahl / Cominvent
In reply to this post by Mark Miller-3
+1

Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)
I'll probably also try to adapt releaseWidard.py on master to work with the new build..

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

--

Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Eric Pugh-4
Sooner is better!  I’m reading up on Gradle now ;-)


On Aug 16, 2019, at 5:33 AM, Jan Høydahl <[hidden email]> wrote:

+1

Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)
I'll probably also try to adapt releaseWidard.py on master to work with the new build..

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

--


_______________________
Eric Pugh Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy  
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.

Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Mark Miller-3
Perhaps better for me as well (I couldn’t help myself and started addressing compiler warnings to clean up build output since gradle keeps the output so compact when its not dumping warning output), my main hold off is that I want to be available and engaged when it goes in. I’d like to target early sept, but I don’t want to commit and then delay, so I gave a little room.

Mark

On Fri, Aug 16, 2019 at 6:54 AM Eric Pugh <[hidden email]> wrote:
Sooner is better!  I’m reading up on Gradle now ;-)


On Aug 16, 2019, at 5:33 AM, Jan Høydahl <[hidden email]> wrote:

+1

Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)
I'll probably also try to adapt releaseWidard.py on master to work with the new build..

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

--


_______________________
Eric Pugh Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy  
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.

--
Reply | Threaded
Open this post in threaded view
|

RE: The Gradle train.

Uwe Schindler
In reply to this post by Jan Høydahl / Cominvent

Hi,

 

I was talking with several people on berlinbuzzwords and we all agreed on one thing: Don’t keep the Ant, Maven and Gradle builds in parallel. IMHO, we should get rid of the Ant build ASAP, because it’s impossible to keep all three systems up to date at the same time.

 

I think the reason for the parallel builds was to have it easier to merge when the new branch was created. In addition, I think Mark was afraid that some people will complain. But I think people will complain more, iff they have to maintain 3 build systems in parallel.

 

In short: I’d like to get rid of the Ant/Maven build as soon as possible! I will also port over the Multirelease-JAR stuff in branch_8x, no worries!

 

I am on vacation the next 2 weeks, so if you switch now, I can’t help with changing Jenkins.

 

Uwe

 

-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: [hidden email]

 

From: Jan Høydahl <[hidden email]>
Sent: Friday, August 16, 2019 11:34 AM
To: [hidden email]; [hidden email]
Subject: Re: The Gradle train.

 

+1

 

Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)

I'll probably also try to adapt releaseWidard.py on master to work with the new build..

 

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com



15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:

 

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

 

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

 

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

 

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

 

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

 

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

 

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

 

--

 

Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Erick Erickson
In reply to this post by Mark Miller-3
Focus, Mark, focus ;)

I fully sympathize, I catch myself repeatedly thinking “Well, since I’m in this code already, why don’t I just change this completely unrelated thing that’s been bothering me for a long time”…..

Not to mention merge issues…..

Random thought: maybe a hack-a-thon on this at Activate? (he says but can’t help since he’ll be teaching Mon and Tue)...

> On Aug 16, 2019, at 8:30 AM, Mark Miller <[hidden email]> wrote:
>
> Perhaps better for me as well (I couldn’t help myself and started addressing compiler warnings to clean up build output since gradle keeps the output so compact when its not dumping warning output), my main hold off is that I want to be available and engaged when it goes in. I’d like to target early sept, but I don’t want to commit and then delay, so I gave a little room.
>
> Mark
>
> On Fri, Aug 16, 2019 at 6:54 AM Eric Pugh <[hidden email]> wrote:
> Sooner is better!  I’m reading up on Gradle now ;-)
>
>
>> On Aug 16, 2019, at 5:33 AM, Jan Høydahl <[hidden email]> wrote:
>>
>> +1
>>
>> Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)
>> I'll probably also try to adapt releaseWidard.py on master to work with the new build..
>>
>> --
>> Jan Høydahl, search solution architect
>> Cominvent AS - www.cominvent.com
>>
>>> 15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:
>>>
>>> https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.
>>>
>>> Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.
>>>
>>> I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.
>>>
>>> This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).
>>>
>>> I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.
>>>
>>> I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.
>>>
>>> This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)
>>>
>>> --
>>> - Mark
>>>
>>> http://about.me/markrmiller
>>
>
> _______________________
> Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy  
> Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
> This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.
>
> --
> - Mark
>
> http://about.me/markrmiller


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Eric Pugh-4
Hack-a-thon is a great thought…. We haven’t had a chance to publicize it widely yet, but Charlie Hull has set up another “pre Activate” hack-a-thon, similar to others he has organized.

We have a conference room for 20 at the American Geophysical Union building that is literally 5 minute walk from where Activate is being held on Tuesday September 10th.   

Getting lots of folks to try out the new build tools on various laptops and environments would be a great theme for the hack-a-thon.


Eric


On Aug 16, 2019, at 11:57 AM, Erick Erickson <[hidden email]> wrote:

Focus, Mark, focus ;)

I fully sympathize, I catch myself repeatedly thinking “Well, since I’m in this code already, why don’t I just change this completely unrelated thing that’s been bothering me for a long time”…..

Not to mention merge issues…..

Random thought: maybe a hack-a-thon on this at Activate? (he says but can’t help since he’ll be teaching Mon and Tue)...

On Aug 16, 2019, at 8:30 AM, Mark Miller <[hidden email]> wrote:

Perhaps better for me as well (I couldn’t help myself and started addressing compiler warnings to clean up build output since gradle keeps the output so compact when its not dumping warning output), my main hold off is that I want to be available and engaged when it goes in. I’d like to target early sept, but I don’t want to commit and then delay, so I gave a little room.

Mark

On Fri, Aug 16, 2019 at 6:54 AM Eric Pugh <[hidden email]> wrote:
Sooner is better!  I’m reading up on Gradle now ;-)


On Aug 16, 2019, at 5:33 AM, Jan Høydahl <[hidden email]> wrote:

+1

Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)
I'll probably also try to adapt releaseWidard.py on master to work with the new build..

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

--
- Mark

http://about.me/markrmiller


_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy  
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.

--
- Mark

http://about.me/markrmiller


---------------------------------------------------------------------
To unsubscribe, [hidden email]
For additional commands, [hidden email]


_______________________
Eric Pugh Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy  
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.

Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Tomás Fernández Löbbe
+1 to merge soon Mark. Thanks for working on this.

On Fri, Aug 16, 2019 at 10:03 AM Eric Pugh <[hidden email]> wrote:
Hack-a-thon is a great thought…. We haven’t had a chance to publicize it widely yet, but Charlie Hull has set up another “pre Activate” hack-a-thon, similar to others he has organized.

We have a conference room for 20 at the American Geophysical Union building that is literally 5 minute walk from where Activate is being held on Tuesday September 10th.   

Getting lots of folks to try out the new build tools on various laptops and environments would be a great theme for the hack-a-thon.


Eric


On Aug 16, 2019, at 11:57 AM, Erick Erickson <[hidden email]> wrote:

Focus, Mark, focus ;)

I fully sympathize, I catch myself repeatedly thinking “Well, since I’m in this code already, why don’t I just change this completely unrelated thing that’s been bothering me for a long time”…..

Not to mention merge issues…..

Random thought: maybe a hack-a-thon on this at Activate? (he says but can’t help since he’ll be teaching Mon and Tue)...

On Aug 16, 2019, at 8:30 AM, Mark Miller <[hidden email]> wrote:

Perhaps better for me as well (I couldn’t help myself and started addressing compiler warnings to clean up build output since gradle keeps the output so compact when its not dumping warning output), my main hold off is that I want to be available and engaged when it goes in. I’d like to target early sept, but I don’t want to commit and then delay, so I gave a little room.

Mark

On Fri, Aug 16, 2019 at 6:54 AM Eric Pugh <[hidden email]> wrote:
Sooner is better!  I’m reading up on Gradle now ;-)


On Aug 16, 2019, at 5:33 AM, Jan Høydahl <[hidden email]> wrote:

+1

Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)
I'll probably also try to adapt releaseWidard.py on master to work with the new build..

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com

15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

--
- Mark

http://about.me/markrmiller


_______________________
Eric Pugh | Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy  
Co-Author: Apache Solr Enterprise Search Server, 3rd Ed
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.

--
- Mark

http://about.me/markrmiller


---------------------------------------------------------------------
To unsubscribe, [hidden email]
For additional commands, [hidden email]


_______________________
Eric Pugh Founder & CEO | OpenSource Connections, LLC | 434.466.1467 | http://www.opensourceconnections.com | My Free/Busy  
This e-mail and all contents, including attachments, is considered to be Company Confidential unless explicitly stated otherwise, regardless of whether attachments are marked as such.

Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Anshum Gupta
In reply to this post by Mark Miller-3
Thanks for doing this Mark! 

+1 on merging this in soon.

On Thu, Aug 15, 2019 at 2:24 PM Mark Miller <[hidden email]> wrote:
https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

--


--
Anshum Gupta
Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Mark Miller-3
In reply to this post by Uwe Schindler
I also would like to keep this overlap to a minimum. Ideally it would be just enough time for devs to try things so we can fine tune any major rough edges before removing things. 

The only reason it’s not my concrete plan is that it’s a little out of my hands and will depend on when/if enough of us can handle churning though through the switch to the teams satisfaction. I don’t expect all smooth sailing.

You are definitely a key piece to that equation, so that’s a start if you are going to be ready.

Mark

On Fri, Aug 16, 2019 at 11:56 AM Uwe Schindler <[hidden email]> wrote:

Hi,

 

I was talking with several people on berlinbuzzwords and we all agreed on one thing: Don’t keep the Ant, Maven and Gradle builds in parallel. IMHO, we should get rid of the Ant build ASAP, because it’s impossible to keep all three systems up to date at the same time.

 

I think the reason for the parallel builds was to have it easier to merge when the new branch was created. In addition, I think Mark was afraid that some people will complain. But I think people will complain more, iff they have to maintain 3 build systems in parallel.

 

In short: I’d like to get rid of the Ant/Maven build as soon as possible! I will also port over the Multirelease-JAR stuff in branch_8x, no worries!

 

I am on vacation the next 2 weeks, so if you switch now, I can’t help with changing Jenkins.

 

Uwe

 

 

From: Jan Høydahl <[hidden email]>
Sent: Friday, August 16, 2019 11:34 AM
To: [hidden email]; [hidden email]
Subject: Re: The Gradle train.

 

+1

 

Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)

I'll probably also try to adapt releaseWidard.py on master to work with the new build..

 

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com



15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:

 

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

 

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

 

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

 

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

 

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

 

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

 

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

 

--

 

--
Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Dawid Weiss-2

I have some experience with Gradle too. I will try to help out if there is something I can do.

Dawid

On Sun, Aug 18, 2019, 09:40 Mark Miller <[hidden email]> wrote:
I also would like to keep this overlap to a minimum. Ideally it would be just enough time for devs to try things so we can fine tune any major rough edges before removing things. 

The only reason it’s not my concrete plan is that it’s a little out of my hands and will depend on when/if enough of us can handle churning though through the switch to the teams satisfaction. I don’t expect all smooth sailing.

You are definitely a key piece to that equation, so that’s a start if you are going to be ready.

Mark

On Fri, Aug 16, 2019 at 11:56 AM Uwe Schindler <[hidden email]> wrote:

Hi,

 

I was talking with several people on berlinbuzzwords and we all agreed on one thing: Don’t keep the Ant, Maven and Gradle builds in parallel. IMHO, we should get rid of the Ant build ASAP, because it’s impossible to keep all three systems up to date at the same time.

 

I think the reason for the parallel builds was to have it easier to merge when the new branch was created. In addition, I think Mark was afraid that some people will complain. But I think people will complain more, iff they have to maintain 3 build systems in parallel.

 

In short: I’d like to get rid of the Ant/Maven build as soon as possible! I will also port over the Multirelease-JAR stuff in branch_8x, no worries!

 

I am on vacation the next 2 weeks, so if you switch now, I can’t help with changing Jenkins.

 

Uwe

 

 

From: Jan Høydahl <[hidden email]>
Sent: Friday, August 16, 2019 11:34 AM
To: [hidden email]; [hidden email]
Subject: Re: The Gradle train.

 

+1

 

Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)

I'll probably also try to adapt releaseWidard.py on master to work with the new build..

 

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com



15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:

 

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

 

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

 

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

 

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

 

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

 

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

 

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

 

--

 

--
Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Mark Miller-3
No doubt there will be. Ensuring we have not lost too much in terms of testing and quantifying what we have lost may be an area you are comfortable with?

I've done a loose attempt at passing in the right system properties and what not, but some things have changed, some things may be missed (random seed and random seed? I do have a start on that though! https://github.com/apache/lucene-solr/blob/jira/SOLR-13452_gradle_5/buildSrc/common/configure-ext.gradle#L53), some things can't be easily configured yet.

We are using gradle to luanch tests and I love it cause it plays well with gradle's max worker setting and launches tests across modules in parallel nicely and works well with their pretty print output stuff, but they still only have round robin garbage for distribution of tests across jvms. Hopefully it catches up to what we had at some point - I'm not super keen on trying to plug the ant launcher in instead myself, though I have experimented with getting the junit 5 stuff working, thinking it's custom engine separation stuff could be cool, but probably not that useful here it turns out.

Anyway, you probably have more experience with that stuff than me, so just my current brain waves.



On Sat, Aug 17, 2019 at 8:55 PM Dawid Weiss <[hidden email]> wrote:

I have some experience with Gradle too. I will try to help out if there is something I can do.

Dawid

On Sun, Aug 18, 2019, 09:40 Mark Miller <[hidden email]> wrote:
I also would like to keep this overlap to a minimum. Ideally it would be just enough time for devs to try things so we can fine tune any major rough edges before removing things. 

The only reason it’s not my concrete plan is that it’s a little out of my hands and will depend on when/if enough of us can handle churning though through the switch to the teams satisfaction. I don’t expect all smooth sailing.

You are definitely a key piece to that equation, so that’s a start if you are going to be ready.

Mark

On Fri, Aug 16, 2019 at 11:56 AM Uwe Schindler <[hidden email]> wrote:

Hi,

 

I was talking with several people on berlinbuzzwords and we all agreed on one thing: Don’t keep the Ant, Maven and Gradle builds in parallel. IMHO, we should get rid of the Ant build ASAP, because it’s impossible to keep all three systems up to date at the same time.

 

I think the reason for the parallel builds was to have it easier to merge when the new branch was created. In addition, I think Mark was afraid that some people will complain. But I think people will complain more, iff they have to maintain 3 build systems in parallel.

 

In short: I’d like to get rid of the Ant/Maven build as soon as possible! I will also port over the Multirelease-JAR stuff in branch_8x, no worries!

 

I am on vacation the next 2 weeks, so if you switch now, I can’t help with changing Jenkins.

 

Uwe

 

 

From: Jan Høydahl <[hidden email]>
Sent: Friday, August 16, 2019 11:34 AM
To: [hidden email]; [hidden email]
Subject: Re: The Gradle train.

 

+1

 

Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)

I'll probably also try to adapt releaseWidard.py on master to work with the new build..

 

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com



15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:

 

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

 

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

 

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

 

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

 

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

 

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

 

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

 

--

 

--


--
Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Mark Miller-3
I'm also looking for someone who wants to help tackle javadoc and actually enabling our missing javadocs links check. I have some early stuff for that, some of it currently commented out:


Perhaps someone that knows a bit about our javadoc situation now?



On Sun, Aug 25, 2019 at 6:31 PM Mark Miller <[hidden email]> wrote:
No doubt there will be. Ensuring we have not lost too much in terms of testing and quantifying what we have lost may be an area you are comfortable with?

I've done a loose attempt at passing in the right system properties and what not, but some things have changed, some things may be missed (random seed and random seed? I do have a start on that though! https://github.com/apache/lucene-solr/blob/jira/SOLR-13452_gradle_5/buildSrc/common/configure-ext.gradle#L53), some things can't be easily configured yet.

We are using gradle to luanch tests and I love it cause it plays well with gradle's max worker setting and launches tests across modules in parallel nicely and works well with their pretty print output stuff, but they still only have round robin garbage for distribution of tests across jvms. Hopefully it catches up to what we had at some point - I'm not super keen on trying to plug the ant launcher in instead myself, though I have experimented with getting the junit 5 stuff working, thinking it's custom engine separation stuff could be cool, but probably not that useful here it turns out.

Anyway, you probably have more experience with that stuff than me, so just my current brain waves.



On Sat, Aug 17, 2019 at 8:55 PM Dawid Weiss <[hidden email]> wrote:

I have some experience with Gradle too. I will try to help out if there is something I can do.

Dawid

On Sun, Aug 18, 2019, 09:40 Mark Miller <[hidden email]> wrote:
I also would like to keep this overlap to a minimum. Ideally it would be just enough time for devs to try things so we can fine tune any major rough edges before removing things. 

The only reason it’s not my concrete plan is that it’s a little out of my hands and will depend on when/if enough of us can handle churning though through the switch to the teams satisfaction. I don’t expect all smooth sailing.

You are definitely a key piece to that equation, so that’s a start if you are going to be ready.

Mark

On Fri, Aug 16, 2019 at 11:56 AM Uwe Schindler <[hidden email]> wrote:

Hi,

 

I was talking with several people on berlinbuzzwords and we all agreed on one thing: Don’t keep the Ant, Maven and Gradle builds in parallel. IMHO, we should get rid of the Ant build ASAP, because it’s impossible to keep all three systems up to date at the same time.

 

I think the reason for the parallel builds was to have it easier to merge when the new branch was created. In addition, I think Mark was afraid that some people will complain. But I think people will complain more, iff they have to maintain 3 build systems in parallel.

 

In short: I’d like to get rid of the Ant/Maven build as soon as possible! I will also port over the Multirelease-JAR stuff in branch_8x, no worries!

 

I am on vacation the next 2 weeks, so if you switch now, I can’t help with changing Jenkins.

 

Uwe

 

 

From: Jan Høydahl <[hidden email]>
Sent: Friday, August 16, 2019 11:34 AM
To: [hidden email]; [hidden email]
Subject: Re: The Gradle train.

 

+1

 

Better to jump in now and have a few weeks of frustration and bug fixing from all of us than keeping this amazing improvement it a dark branch much longer :)

I'll probably also try to adapt releaseWidard.py on master to work with the new build..

 

--
Jan Høydahl, search solution architect
Cominvent AS - www.cominvent.com



15. aug. 2019 kl. 23:23 skrev Mark Miller <[hidden email]>:

 

https://issues.apache.org/jira/projects/SOLR/issues/SOLR-13452 Update the lucene-solr build from Ivy+Ant+Maven (shadow build) to Gradle.

 

Okay, we are at the point where either this thing lands soon and gains some contributors to help finish or it  overwhelms me and crashes & burns. That almost sounds negative, but it was actually the plan so far and I'm pretty excited after all this time invested. I need to punt this over to the community though - the final implications and ramifications of moving fully to gradle are just too big for me individually regardless of the time frame.

 

I've done about 95%+ of what I wanted to do before trying to land something - a few more hoops to jump around. We pull in more deps than we should right now, I'll deal with that shortly, and mvn publishing needs work (mostly around solr-server, but dist and publishing both prob need edge work at least). Those are the main things on my mind. There are probably a ton of other little things, but I'm thinking those that are important will rise up quickly and the rest can be handled over time.

 

This will be a large change. Some things will still take time to get up to par with what we have now. Many things will need to be sorted out (jenkins, releases, smoke tester type things, docs, etc).

 

I've also made all the decisions and trade-offs and what not. I'm pretty happy about that, but I'm sure some will want to discuss and debate some choices once things are in their face. I've spent a lot of time in my recent life on this stuff and I'm ready to battle for some of it :) And to be mistaken, ignorant, or convinced of other paths for some other parts of it. I'll only say, every time I go from working with the gradle build back to ant+ivy+mvn, it feels like a big backslide.

 

I'm thinking maybe in September/October? And only on master, hopefully living side by side with ant+ivy+mvn, but the goal would be for that period to be brief. They can't live in complete harmony - someone has to own the dependency view of the world for example, the one that actually gets committed (license, checksums, etc). Otherwise, I've done my best to do this in a way that doesn't break the current build. Will need to inspect that closer before landing though.

 

This is just another heads up. Once we are in a main branch, I'm hoping a few of you will either have to jump in and help this land or we will have to pull it back out I think. Be prepared :)

 

--

 

--


--


--
Reply | Threaded
Open this post in threaded view
|

Re: The Gradle train.

Dawid Weiss-2
In reply to this post by Mark Miller-3
> I've done a loose attempt at passing in the right system properties and what not, but some things have changed, some things may be missed (random seed and random seed?

This really shouldn't be too much of a problem since all the stuff
configuring the runner can be passed via system properties -- this was
designed this way so that tests could be run in isolation from the ant
runner (in the IDE, for example).

The ant runner doesn't have to be ported. The main advantages
(isolating test outputs, balancing test runtimes) are not worth the
complexity I think. We ourselves are using the built-in Gradle test
runner; ES has dropped their own wrapper for running the ant runner a
while ago (Ryan wrote that). I think we should stick to what's
available to minimize the configuration files.

> junit 5 stuff working, thinking it's custom engine separation stuff could be cool, but probably not that useful here it turns out.

JUnit5 support is another thing, I'd say -- it'd be nice to have it
but it'd require changes to the runner's code itself. I didn't have
time to investigate JUnit5's benefits though and I'm not using it
anywhere so I don't have any experience.

> Anyway, you probably have more experience with that stuff than me, so just my current brain waves.

I'll look at it, sure. Not before mid-September though -- need to dig
myself out from the stuff that has accumulated over holidays.

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]