Lucene / Solr Gradle Build Update

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Lucene / Solr Gradle Build Update

Mark Miller-3
Since I have heard no objection, I've continued working on moving the project from ant+ivy+maven to gradle.

At this point I've contributed significant time to this project. I hope everyone has taken the time to consider this change and their possible concerns. I don't want to beat a dead horse, but there is too much effort involved to get caught up at the end.

There is still a lot to do, it's not going to happen tomorrow, but many, many things are done.

The performance of the build in comparison to what we had will astonish you on good hardware.

Even comparing to your experience with the majority of maven builds, this will be *significantly* faster. This is without using the gradle build cache or paying proper attention to task uptodate properties.

There are considerable improvements and benefits we can reap from this change, but the sheer speed has made the development experience for me way more enjoyable.

My goal is to take us from a very powerful but complicated and slow and clunky and dense build to what is essentially a modern top tier build experience in power, performance, integrity and ease of use.

I've made significant progress over the past month or so, but it will likely take me another 2-4 before I plan on having something I'd considered ready for prime time 9x duties.

I'll take the time needed to get things right, hopefully everyone else will take the time to help with a transition when that time comes.

My hope is that version 9 is the first built with gradle. We can consider it being available on 8 as well, but I don't think it makes sense to release 8x versions with gradle. I think we should only consider the gradle build on 8x as a developer convenience and it would be on the users of it to address keeping it up to date with changes on the ant build as problems arise. Depending on the time, it may not even make sense to put effort here.

This weekend I'm wrapping up some work on making our dependency management headache more transparent. I think we can make a lot of improvements on understanding what is in our build and why and what is published or shipped where and why. 
Reply | Threaded
Open this post in threaded view
|

Re: Lucene / Solr Gradle Build Update

Ishan Chattopadhyaya
Looking forward to it, and thanks a lot for your effort. Excited!

On Sat, 8 Jun, 2019, 4:29 AM Mark Miller, <[hidden email]> wrote:
Since I have heard no objection, I've continued working on moving the project from ant+ivy+maven to gradle.

At this point I've contributed significant time to this project. I hope everyone has taken the time to consider this change and their possible concerns. I don't want to beat a dead horse, but there is too much effort involved to get caught up at the end.

There is still a lot to do, it's not going to happen tomorrow, but many, many things are done.

The performance of the build in comparison to what we had will astonish you on good hardware.

Even comparing to your experience with the majority of maven builds, this will be *significantly* faster. This is without using the gradle build cache or paying proper attention to task uptodate properties.

There are considerable improvements and benefits we can reap from this change, but the sheer speed has made the development experience for me way more enjoyable.

My goal is to take us from a very powerful but complicated and slow and clunky and dense build to what is essentially a modern top tier build experience in power, performance, integrity and ease of use.

I've made significant progress over the past month or so, but it will likely take me another 2-4 before I plan on having something I'd considered ready for prime time 9x duties.

I'll take the time needed to get things right, hopefully everyone else will take the time to help with a transition when that time comes.

My hope is that version 9 is the first built with gradle. We can consider it being available on 8 as well, but I don't think it makes sense to release 8x versions with gradle. I think we should only consider the gradle build on 8x as a developer convenience and it would be on the users of it to address keeping it up to date with changes on the ant build as problems arise. Depending on the time, it may not even make sense to put effort here.

This weekend I'm wrapping up some work on making our dependency management headache more transparent. I think we can make a lot of improvements on understanding what is in our build and why and what is published or shipped where and why. 
Reply | Threaded
Open this post in threaded view
|

Re: Lucene / Solr Gradle Build Update

Gus Heck
Also looking forward to it. :) especially if it speeds things up. Moving forward with it in 9x and not 8 sounds good to me. There are folks out there who build themselves custom builds of Solr, so build changes this big seem like a sort of back compatability concern, though obviously only for a minority.

On Fri, Jun 7, 2019 at 11:05 PM Ishan Chattopadhyaya <[hidden email]> wrote:
Looking forward to it, and thanks a lot for your effort. Excited!

On Sat, 8 Jun, 2019, 4:29 AM Mark Miller, <[hidden email]> wrote:
Since I have heard no objection, I've continued working on moving the project from ant+ivy+maven to gradle.

At this point I've contributed significant time to this project. I hope everyone has taken the time to consider this change and their possible concerns. I don't want to beat a dead horse, but there is too much effort involved to get caught up at the end.

There is still a lot to do, it's not going to happen tomorrow, but many, many things are done.

The performance of the build in comparison to what we had will astonish you on good hardware.

Even comparing to your experience with the majority of maven builds, this will be *significantly* faster. This is without using the gradle build cache or paying proper attention to task uptodate properties.

There are considerable improvements and benefits we can reap from this change, but the sheer speed has made the development experience for me way more enjoyable.

My goal is to take us from a very powerful but complicated and slow and clunky and dense build to what is essentially a modern top tier build experience in power, performance, integrity and ease of use.

I've made significant progress over the past month or so, but it will likely take me another 2-4 before I plan on having something I'd considered ready for prime time 9x duties.

I'll take the time needed to get things right, hopefully everyone else will take the time to help with a transition when that time comes.

My hope is that version 9 is the first built with gradle. We can consider it being available on 8 as well, but I don't think it makes sense to release 8x versions with gradle. I think we should only consider the gradle build on 8x as a developer convenience and it would be on the users of it to address keeping it up to date with changes on the ant build as problems arise. Depending on the time, it may not even make sense to put effort here.

This weekend I'm wrapping up some work on making our dependency management headache more transparent. I think we can make a lot of improvements on understanding what is in our build and why and what is published or shipped where and why. 


--
Reply | Threaded
Open this post in threaded view
|

Re: Lucene / Solr Gradle Build Update

Michael Sokolov-4
Please don't stop now! Many thanks for doing the work. Faster builds
will answer for any grumbling/transition pains I expect

On Sat, Jun 8, 2019 at 9:58 AM Gus Heck <[hidden email]> wrote:

>
> Also looking forward to it. :) especially if it speeds things up. Moving forward with it in 9x and not 8 sounds good to me. There are folks out there who build themselves custom builds of Solr, so build changes this big seem like a sort of back compatability concern, though obviously only for a minority.
>
> On Fri, Jun 7, 2019 at 11:05 PM Ishan Chattopadhyaya <[hidden email]> wrote:
>>
>> Looking forward to it, and thanks a lot for your effort. Excited!
>>
>> On Sat, 8 Jun, 2019, 4:29 AM Mark Miller, <[hidden email]> wrote:
>>>
>>> Since I have heard no objection, I've continued working on moving the project from ant+ivy+maven to gradle.
>>>
>>> At this point I've contributed significant time to this project. I hope everyone has taken the time to consider this change and their possible concerns. I don't want to beat a dead horse, but there is too much effort involved to get caught up at the end.
>>>
>>> There is still a lot to do, it's not going to happen tomorrow, but many, many things are done.
>>>
>>> The performance of the build in comparison to what we had will astonish you on good hardware.
>>>
>>> Even comparing to your experience with the majority of maven builds, this will be *significantly* faster. This is without using the gradle build cache or paying proper attention to task uptodate properties.
>>>
>>> There are considerable improvements and benefits we can reap from this change, but the sheer speed has made the development experience for me way more enjoyable.
>>>
>>> My goal is to take us from a very powerful but complicated and slow and clunky and dense build to what is essentially a modern top tier build experience in power, performance, integrity and ease of use.
>>>
>>> I've made significant progress over the past month or so, but it will likely take me another 2-4 before I plan on having something I'd considered ready for prime time 9x duties.
>>>
>>> I'll take the time needed to get things right, hopefully everyone else will take the time to help with a transition when that time comes.
>>>
>>> My hope is that version 9 is the first built with gradle. We can consider it being available on 8 as well, but I don't think it makes sense to release 8x versions with gradle. I think we should only consider the gradle build on 8x as a developer convenience and it would be on the users of it to address keeping it up to date with changes on the ant build as problems arise. Depending on the time, it may not even make sense to put effort here.
>>>
>>> This weekend I'm wrapping up some work on making our dependency management headache more transparent. I think we can make a lot of improvements on understanding what is in our build and why and what is published or shipped where and why.
>>>
>>> --
>>> - Mark
>>>
>>> http://about.me/markrmiller
>
>
>
> --
> http://www.needhamsoftware.com (work)
> http://www.the111shift.com (play)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene / Solr Gradle Build Update

Dawid Weiss-2
In reply to this post by Mark Miller-3
> [...] but it will likely take me another 2-4 before I plan on having something I'd considered ready for prime time 9x duties.

I'd say aim at integrating it sooner than later. I think everyone
realizes switching a build system isn't a minor deal; I'd expect some
things to not work right away. And even if something doesn't work,
there may be more hands willing to help out if it's a particular
person's itch. ;)

Gradle is a powerful toy and I personally find it fun to work with,
even if it drives me crazy sometimes when I can't figure out why
something doesn't work the way I think it should. :)

Dawid

On Sat, Jun 8, 2019 at 12:59 AM Mark Miller <[hidden email]> wrote:

>
> Since I have heard no objection, I've continued working on moving the project from ant+ivy+maven to gradle.
>
> At this point I've contributed significant time to this project. I hope everyone has taken the time to consider this change and their possible concerns. I don't want to beat a dead horse, but there is too much effort involved to get caught up at the end.
>
> There is still a lot to do, it's not going to happen tomorrow, but many, many things are done.
>
> The performance of the build in comparison to what we had will astonish you on good hardware.
>
> Even comparing to your experience with the majority of maven builds, this will be *significantly* faster. This is without using the gradle build cache or paying proper attention to task uptodate properties.
>
> There are considerable improvements and benefits we can reap from this change, but the sheer speed has made the development experience for me way more enjoyable.
>
> My goal is to take us from a very powerful but complicated and slow and clunky and dense build to what is essentially a modern top tier build experience in power, performance, integrity and ease of use.
>
> I've made significant progress over the past month or so, but it will likely take me another 2-4 before I plan on having something I'd considered ready for prime time 9x duties.
>
> I'll take the time needed to get things right, hopefully everyone else will take the time to help with a transition when that time comes.
>
> My hope is that version 9 is the first built with gradle. We can consider it being available on 8 as well, but I don't think it makes sense to release 8x versions with gradle. I think we should only consider the gradle build on 8x as a developer convenience and it would be on the users of it to address keeping it up to date with changes on the ant build as problems arise. Depending on the time, it may not even make sense to put effort here.
>
> This weekend I'm wrapping up some work on making our dependency management headache more transparent. I think we can make a lot of improvements on understanding what is in our build and why and what is published or shipped where and why.
>
> --
> - Mark
>
> http://about.me/markrmiller

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene / Solr Gradle Build Update

david.w.smiley@gmail.com
I'm very much looking forward to this too!  Thanks for your tireless efforts Mark.

What roadblocks remain before merging/committing this into master now-ish?  If the Ant build still works fine and if there is plenty of utility out of the current state of your work (which you indicate) then I'd say lets get this into master sooner rather than later.  I know it's hard psychologically to do this when there's always more things left to do.

~ David Smiley
Apache Lucene/Solr Search Developer


On Sat, Jun 8, 2019 at 4:52 PM Dawid Weiss <[hidden email]> wrote:
> [...] but it will likely take me another 2-4 before I plan on having something I'd considered ready for prime time 9x duties.

I'd say aim at integrating it sooner than later. I think everyone
realizes switching a build system isn't a minor deal; I'd expect some
things to not work right away. And even if something doesn't work,
there may be more hands willing to help out if it's a particular
person's itch. ;)

Gradle is a powerful toy and I personally find it fun to work with,
even if it drives me crazy sometimes when I can't figure out why
something doesn't work the way I think it should. :)

Dawid

On Sat, Jun 8, 2019 at 12:59 AM Mark Miller <[hidden email]> wrote:
>
> Since I have heard no objection, I've continued working on moving the project from ant+ivy+maven to gradle.
>
> At this point I've contributed significant time to this project. I hope everyone has taken the time to consider this change and their possible concerns. I don't want to beat a dead horse, but there is too much effort involved to get caught up at the end.
>
> There is still a lot to do, it's not going to happen tomorrow, but many, many things are done.
>
> The performance of the build in comparison to what we had will astonish you on good hardware.
>
> Even comparing to your experience with the majority of maven builds, this will be *significantly* faster. This is without using the gradle build cache or paying proper attention to task uptodate properties.
>
> There are considerable improvements and benefits we can reap from this change, but the sheer speed has made the development experience for me way more enjoyable.
>
> My goal is to take us from a very powerful but complicated and slow and clunky and dense build to what is essentially a modern top tier build experience in power, performance, integrity and ease of use.
>
> I've made significant progress over the past month or so, but it will likely take me another 2-4 before I plan on having something I'd considered ready for prime time 9x duties.
>
> I'll take the time needed to get things right, hopefully everyone else will take the time to help with a transition when that time comes.
>
> My hope is that version 9 is the first built with gradle. We can consider it being available on 8 as well, but I don't think it makes sense to release 8x versions with gradle. I think we should only consider the gradle build on 8x as a developer convenience and it would be on the users of it to address keeping it up to date with changes on the ant build as problems arise. Depending on the time, it may not even make sense to put effort here.
>
> This weekend I'm wrapping up some work on making our dependency management headache more transparent. I think we can make a lot of improvements on understanding what is in our build and why and what is published or shipped where and why.
>
> --
> - Mark
>
> http://about.me/markrmiller

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Lucene / Solr Gradle Build Update

Mark Miller-3
I think there will end up being plenty of final bits and plenty of work (Jenkins and release todo etc) left when it lands on master - I’d like to finish some core parts first though. Dependencies, publishing and distribution. There is not a to left to them, but given my time commitment I can count on for this, it will take me a bit. I think this fits with good timing myself though - I think it will be easiest if we land and start to transition mid way through the 8x releases. 

On Mon, Jun 10, 2019 at 2:07 PM David Smiley <[hidden email]> wrote:
I'm very much looking forward to this too!  Thanks for your tireless efforts Mark.

What roadblocks remain before merging/committing this into master now-ish?  If the Ant build still works fine and if there is plenty of utility out of the current state of your work (which you indicate) then I'd say lets get this into master sooner rather than later.  I know it's hard psychologically to do this when there's always more things left to do.

~ David Smiley
Apache Lucene/Solr Search Developer


On Sat, Jun 8, 2019 at 4:52 PM Dawid Weiss <[hidden email]> wrote:
> [...] but it will likely take me another 2-4 before I plan on having something I'd considered ready for prime time 9x duties.

I'd say aim at integrating it sooner than later. I think everyone
realizes switching a build system isn't a minor deal; I'd expect some
things to not work right away. And even if something doesn't work,
there may be more hands willing to help out if it's a particular
person's itch. ;)

Gradle is a powerful toy and I personally find it fun to work with,
even if it drives me crazy sometimes when I can't figure out why
something doesn't work the way I think it should. :)

Dawid

On Sat, Jun 8, 2019 at 12:59 AM Mark Miller <[hidden email]> wrote:
>
> Since I have heard no objection, I've continued working on moving the project from ant+ivy+maven to gradle.
>
> At this point I've contributed significant time to this project. I hope everyone has taken the time to consider this change and their possible concerns. I don't want to beat a dead horse, but there is too much effort involved to get caught up at the end.
>
> There is still a lot to do, it's not going to happen tomorrow, but many, many things are done.
>
> The performance of the build in comparison to what we had will astonish you on good hardware.
>
> Even comparing to your experience with the majority of maven builds, this will be *significantly* faster. This is without using the gradle build cache or paying proper attention to task uptodate properties.
>
> There are considerable improvements and benefits we can reap from this change, but the sheer speed has made the development experience for me way more enjoyable.
>
> My goal is to take us from a very powerful but complicated and slow and clunky and dense build to what is essentially a modern top tier build experience in power, performance, integrity and ease of use.
>
> I've made significant progress over the past month or so, but it will likely take me another 2-4 before I plan on having something I'd considered ready for prime time 9x duties.
>
> I'll take the time needed to get things right, hopefully everyone else will take the time to help with a transition when that time comes.
>
> My hope is that version 9 is the first built with gradle. We can consider it being available on 8 as well, but I don't think it makes sense to release 8x versions with gradle. I think we should only consider the gradle build on 8x as a developer convenience and it would be on the users of it to address keeping it up to date with changes on the ant build as problems arise. Depending on the time, it may not even make sense to put effort here.
>
> This weekend I'm wrapping up some work on making our dependency management headache more transparent. I think we can make a lot of improvements on understanding what is in our build and why and what is published or shipped where and why.
>
> --
> - Mark
>
> http://about.me/markrmiller

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

--