kicking tires of gradle build

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

kicking tires of gradle build

Michael Sokolov-4
Hi I am playing around with the gradle build. Overall looks great!
Thanks to everyone who has been pushing this forward. I have a few
questions; maybe just gradle noob questions, since I haven't used it
much (except as part of Android Studio, where all the details are kind
of taken care of for you).

1) I'm not sure which branch is the "current" one. Ideally I'd like to
be using a branch based off master. I see there are lots of branches:
jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
what is referenced in the wiki:
https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
but that seems to be a little out of date, so I switched to 8, because
you know, bigger is better. What are all these numbers? Just tracking
snapshots along the way? Which is the one based off 8x?

2) When I run any gradle command I get this warning:

> Configure project :
not user home user.gradle /home/sokolov/user.properties

Its grammar is throwing me: does it mean it expects to find these
files and can't find them? I have no user.properties file in my
homedir: should I?

3) I can run tests in a package using (eg) ./gradlew
lucene:lucene-core:test and see the test report output in an html file
- cool. Is it possible to get test output to stdout though? I am used
to running tests in emacs and have a script set up for parsing stack
traces in the output so emacs can jump there. I know I can use
intellij, and I often do, but I would like to also get the emacs
workflow going - definitely should not be a blocker for switching to
this - I am just looking for some hints as to getting errors logged to
stdout by gradle.

4) If I use the --tests option to specify a single test class to run,
and the class does not exist (I made a booboo, say), the build runs no
tests, and succeeds, but this is misleading: it should fail instead,
as the ant build does. Again, not a blocker, but if anybody knows how
to fix this, it would be great. I'll open an issue anyway.

again thanks, this looks awesome; with the daemon it runs so nicely :)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: kicking tires of gradle build

Michael Sokolov-4
ah as soon as I sent, I realized that failed test output goes to the
console, so all is well on that front. There are no dumb questions,
right?

On Tue, Nov 12, 2019 at 9:11 AM Michael Sokolov <[hidden email]> wrote:

>
> Hi I am playing around with the gradle build. Overall looks great!
> Thanks to everyone who has been pushing this forward. I have a few
> questions; maybe just gradle noob questions, since I haven't used it
> much (except as part of Android Studio, where all the details are kind
> of taken care of for you).
>
> 1) I'm not sure which branch is the "current" one. Ideally I'd like to
> be using a branch based off master. I see there are lots of branches:
> jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
> what is referenced in the wiki:
> https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
> but that seems to be a little out of date, so I switched to 8, because
> you know, bigger is better. What are all these numbers? Just tracking
> snapshots along the way? Which is the one based off 8x?
>
> 2) When I run any gradle command I get this warning:
>
> > Configure project :
> not user home user.gradle /home/sokolov/user.properties
>
> Its grammar is throwing me: does it mean it expects to find these
> files and can't find them? I have no user.properties file in my
> homedir: should I?
>
> 3) I can run tests in a package using (eg) ./gradlew
> lucene:lucene-core:test and see the test report output in an html file
> - cool. Is it possible to get test output to stdout though? I am used
> to running tests in emacs and have a script set up for parsing stack
> traces in the output so emacs can jump there. I know I can use
> intellij, and I often do, but I would like to also get the emacs
> workflow going - definitely should not be a blocker for switching to
> this - I am just looking for some hints as to getting errors logged to
> stdout by gradle.
>
> 4) If I use the --tests option to specify a single test class to run,
> and the class does not exist (I made a booboo, say), the build runs no
> tests, and succeeds, but this is misleading: it should fail instead,
> as the ant build does. Again, not a blocker, but if anybody knows how
> to fix this, it would be great. I'll open an issue anyway.
>
> again thanks, this looks awesome; with the daemon it runs so nicely :)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: kicking tires of gradle build

Mark Miller-3
In reply to this post by Michael Sokolov-4
I'm gonna help you here, cause im not sure anyone else fully knows.

On Tue, Nov 12, 2019 at 8:12 AM Michael Sokolov <[hidden email]> wrote:
Hi I am playing around with the gradle build. Overall looks great!
Thanks to everyone who has been pushing this forward. I have a few
questions; maybe just gradle noob questions, since I haven't used it
much (except as part of Android Studio, where all the details are kind
of taken care of for you).

1) I'm not sure which branch is the "current" one. Ideally I'd like to
be using a branch based off master. I see there are lots of branches:
jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
what is referenced in the wiki:
https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
but that seems to be a little out of date, so I switched to 8, because
you know, bigger is better. What are all these numbers? Just tracking
snapshots along the way? Which is the one based off 8x?


8 was the latest - why these numbers and branches? I made a new one when I brought things up to trunk, because then I could rebase my work and it kept things more sensible.

Highest number, always latest, I didn't update the JIRA, I kind of got in a little fight.

 
2) When I run any gradle command I get this warning:

> Configure project :
not user home user.gradle /home/sokolov/user.properties

Its grammar is throwing me: does it mean it expects to find these
files and can't find them? I have no user.properties file in my
homedir: should I?

I hadn't quite polished this. Everyone was pissed off I hadn't given them a way to configure things and recommended they use their ~/.gradle config file and people didnt like, so this is an attmpt to let you configure by adding a user.properties to the project folder or the you home dir and if its there it should be sucked up and if not it shoudlnt matter.
 

3) I can run tests in a package using (eg) ./gradlew
lucene:lucene-core:test and see the test report output in an html file
- cool. Is it possible to get test output to stdout though? I am used
to running tests in emacs and have a script set up for parsing stack
traces in the output so emacs can jump there. I know I can use
intellij, and I often do, but I would like to also get the emacs
workflow going - definitely should not be a blocker for switching to
this - I am just looking for some hints as to getting errors logged to
stdout by gradle.

Not sure, been a while, I only know stuff well when I work on it and then I forget.

The Gradle philosophy is kind of very very minimal output that does not reallyscroll down unless you specify --debug or --info --stacktrace orwhatever.
 

4) If I use the --tests option to specify a single test class to run,
and the class does not exist (I made a booboo, say), the build runs no
tests, and succeeds, but this is misleading: it should fail instead,
as the ant build does. Again, not a blocker, but if anybody knows how
to fix this, it would be great. I'll open an issue anyway.

Yup, tweak that crap, this is probably to deal with some other issue and we need to find the balance.
 

again thanks, this looks awesome; with the daemon it runs so nicely :)


Dude it's so amazing and I don't say that because I built it - it took me friggen forever, but I love it. Use the build cache, configure the right number of works and test jvms and monitor your system load ... oh man, I love that gradle build.
 

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--
Reply | Threaded
Open this post in threaded view
|

Re: kicking tires of gradle build

Mark Miller-3
You might want to pull out my jdeps stuff, probably that needs more work by someone up to speed - every project should be doing, its like java gave us this tool, it seems super underused.

You might also pull the Docker test that tests tests the build - its slow but I couldn't live without it, but this is the type of stuff you guys would have to decide to keep and maintain.

On Tue, Nov 12, 2019 at 8:23 AM Mark Miller <[hidden email]> wrote:
I'm gonna help you here, cause im not sure anyone else fully knows.

On Tue, Nov 12, 2019 at 8:12 AM Michael Sokolov <[hidden email]> wrote:
Hi I am playing around with the gradle build. Overall looks great!
Thanks to everyone who has been pushing this forward. I have a few
questions; maybe just gradle noob questions, since I haven't used it
much (except as part of Android Studio, where all the details are kind
of taken care of for you).

1) I'm not sure which branch is the "current" one. Ideally I'd like to
be using a branch based off master. I see there are lots of branches:
jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
what is referenced in the wiki:
https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
but that seems to be a little out of date, so I switched to 8, because
you know, bigger is better. What are all these numbers? Just tracking
snapshots along the way? Which is the one based off 8x?


8 was the latest - why these numbers and branches? I made a new one when I brought things up to trunk, because then I could rebase my work and it kept things more sensible.

Highest number, always latest, I didn't update the JIRA, I kind of got in a little fight.

 
2) When I run any gradle command I get this warning:

> Configure project :
not user home user.gradle /home/sokolov/user.properties

Its grammar is throwing me: does it mean it expects to find these
files and can't find them? I have no user.properties file in my
homedir: should I?

I hadn't quite polished this. Everyone was pissed off I hadn't given them a way to configure things and recommended they use their ~/.gradle config file and people didnt like, so this is an attmpt to let you configure by adding a user.properties to the project folder or the you home dir and if its there it should be sucked up and if not it shoudlnt matter.
 

3) I can run tests in a package using (eg) ./gradlew
lucene:lucene-core:test and see the test report output in an html file
- cool. Is it possible to get test output to stdout though? I am used
to running tests in emacs and have a script set up for parsing stack
traces in the output so emacs can jump there. I know I can use
intellij, and I often do, but I would like to also get the emacs
workflow going - definitely should not be a blocker for switching to
this - I am just looking for some hints as to getting errors logged to
stdout by gradle.

Not sure, been a while, I only know stuff well when I work on it and then I forget.

The Gradle philosophy is kind of very very minimal output that does not reallyscroll down unless you specify --debug or --info --stacktrace orwhatever.
 

4) If I use the --tests option to specify a single test class to run,
and the class does not exist (I made a booboo, say), the build runs no
tests, and succeeds, but this is misleading: it should fail instead,
as the ant build does. Again, not a blocker, but if anybody knows how
to fix this, it would be great. I'll open an issue anyway.

Yup, tweak that crap, this is probably to deal with some other issue and we need to find the balance.
 

again thanks, this looks awesome; with the daemon it runs so nicely :)


Dude it's so amazing and I don't say that because I built it - it took me friggen forever, but I love it. Use the build cache, configure the right number of works and test jvms and monitor your system load ... oh man, I love that gradle build.
 

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--


--
Reply | Threaded
Open this post in threaded view
|

Re: kicking tires of gradle build

Dawid Weiss-2
In reply to this post by Michael Sokolov-4
> 1) I'm not sure which branch is the "current" one. Ideally I'd like to
> be using a branch based off master. I see there are lots of branches:

> 4) If I use the --tests option to specify a single test class to run,
> and the class does not exist (I made a booboo, say), the build runs no
> tests, and succeeds, but this is misleading: it should fail instead,
> as the ant build does. Again, not a blocker, but if anybody knows how
> to fix this, it would be great. I'll open an issue anyway.

I think the reason is that when you run an unqualified task against a
multi-project module all tasks of the given name (from all
subprojects) will be ordered and executed. This means that when you
run:

gradlew test --tests Foo

then some projects would end up failing the build because of zero
tests. This could be fixed by not failing on individual test tasks,
adding a listener and check the number of tests executed (from all
projects) after the build completes.

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: kicking tires of gradle build

Mark Miller-3
In reply to this post by Mark Miller-3
and A cool trick for solr tests.

So gradle does not do well for running long and short tests in a mix in parallel - our ant stuff does much better after first run.

I run some cool gradle to work around this, but then remember Solr tests could be fast enough that it doesn't matter. But until you have Solr tests that fast, there is a trick that can help.

So if you set the max number of jvms for a module to use for tests very high - like say 64, 128 and you set the max grade works to what you actually want to allow to run, like for 16 cores, maybe 14-16. It's hard to say - when you remove more blocking and non buttery slow stuff, you can use more, in a bad state less.

But the idea, now you almost have like virtual JVMs and when run solr tests across all its modules, you can get the heavy SolrCloud tests to start much much sooner - which helps everything a lot. Just experiment.

On Tue, Nov 12, 2019 at 8:28 AM Mark Miller <[hidden email]> wrote:
You might want to pull out my jdeps stuff, probably that needs more work by someone up to speed - every project should be doing, its like java gave us this tool, it seems super underused.

You might also pull the Docker test that tests tests the build - its slow but I couldn't live without it, but this is the type of stuff you guys would have to decide to keep and maintain.

On Tue, Nov 12, 2019 at 8:23 AM Mark Miller <[hidden email]> wrote:
I'm gonna help you here, cause im not sure anyone else fully knows.

On Tue, Nov 12, 2019 at 8:12 AM Michael Sokolov <[hidden email]> wrote:
Hi I am playing around with the gradle build. Overall looks great!
Thanks to everyone who has been pushing this forward. I have a few
questions; maybe just gradle noob questions, since I haven't used it
much (except as part of Android Studio, where all the details are kind
of taken care of for you).

1) I'm not sure which branch is the "current" one. Ideally I'd like to
be using a branch based off master. I see there are lots of branches:
jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
what is referenced in the wiki:
https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
but that seems to be a little out of date, so I switched to 8, because
you know, bigger is better. What are all these numbers? Just tracking
snapshots along the way? Which is the one based off 8x?


8 was the latest - why these numbers and branches? I made a new one when I brought things up to trunk, because then I could rebase my work and it kept things more sensible.

Highest number, always latest, I didn't update the JIRA, I kind of got in a little fight.

 
2) When I run any gradle command I get this warning:

> Configure project :
not user home user.gradle /home/sokolov/user.properties

Its grammar is throwing me: does it mean it expects to find these
files and can't find them? I have no user.properties file in my
homedir: should I?

I hadn't quite polished this. Everyone was pissed off I hadn't given them a way to configure things and recommended they use their ~/.gradle config file and people didnt like, so this is an attmpt to let you configure by adding a user.properties to the project folder or the you home dir and if its there it should be sucked up and if not it shoudlnt matter.
 

3) I can run tests in a package using (eg) ./gradlew
lucene:lucene-core:test and see the test report output in an html file
- cool. Is it possible to get test output to stdout though? I am used
to running tests in emacs and have a script set up for parsing stack
traces in the output so emacs can jump there. I know I can use
intellij, and I often do, but I would like to also get the emacs
workflow going - definitely should not be a blocker for switching to
this - I am just looking for some hints as to getting errors logged to
stdout by gradle.

Not sure, been a while, I only know stuff well when I work on it and then I forget.

The Gradle philosophy is kind of very very minimal output that does not reallyscroll down unless you specify --debug or --info --stacktrace orwhatever.
 

4) If I use the --tests option to specify a single test class to run,
and the class does not exist (I made a booboo, say), the build runs no
tests, and succeeds, but this is misleading: it should fail instead,
as the ant build does. Again, not a blocker, but if anybody knows how
to fix this, it would be great. I'll open an issue anyway.

Yup, tweak that crap, this is probably to deal with some other issue and we need to find the balance.
 

again thanks, this looks awesome; with the daemon it runs so nicely :)


Dude it's so amazing and I don't say that because I built it - it took me friggen forever, but I love it. Use the build cache, configure the right number of works and test jvms and monitor your system load ... oh man, I love that gradle build.
 

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--


--


--
Reply | Threaded
Open this post in threaded view
|

Re: kicking tires of gradle build

Dawid Weiss-2
In reply to this post by Michael Sokolov-4
Also, I wrote a short help for running tests under Gradle branch -- if
it's of some help to you, please read it!

https://github.com/apache/lucene-solr/blob/jira/SOLR-13452_gradle_7/buildSrc/common/help-text/testHelp.txt

On Tue, Nov 12, 2019 at 3:12 PM Michael Sokolov <[hidden email]> wrote:

>
> Hi I am playing around with the gradle build. Overall looks great!
> Thanks to everyone who has been pushing this forward. I have a few
> questions; maybe just gradle noob questions, since I haven't used it
> much (except as part of Android Studio, where all the details are kind
> of taken care of for you).
>
> 1) I'm not sure which branch is the "current" one. Ideally I'd like to
> be using a branch based off master. I see there are lots of branches:
> jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
> what is referenced in the wiki:
> https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
> but that seems to be a little out of date, so I switched to 8, because
> you know, bigger is better. What are all these numbers? Just tracking
> snapshots along the way? Which is the one based off 8x?
>
> 2) When I run any gradle command I get this warning:
>
> > Configure project :
> not user home user.gradle /home/sokolov/user.properties
>
> Its grammar is throwing me: does it mean it expects to find these
> files and can't find them? I have no user.properties file in my
> homedir: should I?
>
> 3) I can run tests in a package using (eg) ./gradlew
> lucene:lucene-core:test and see the test report output in an html file
> - cool. Is it possible to get test output to stdout though? I am used
> to running tests in emacs and have a script set up for parsing stack
> traces in the output so emacs can jump there. I know I can use
> intellij, and I often do, but I would like to also get the emacs
> workflow going - definitely should not be a blocker for switching to
> this - I am just looking for some hints as to getting errors logged to
> stdout by gradle.
>
> 4) If I use the --tests option to specify a single test class to run,
> and the class does not exist (I made a booboo, say), the build runs no
> tests, and succeeds, but this is misleading: it should fail instead,
> as the ant build does. Again, not a blocker, but if anybody knows how
> to fix this, it would be great. I'll open an issue anyway.
>
> again thanks, this looks awesome; with the daemon it runs so nicely :)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: kicking tires of gradle build

Michael Sokolov-4
Thanks for the answers. I opened
https://issues.apache.org/jira/browse/SOLR-13923 for the minor issue
about failing when no tests run. I'm happily running tests. Now just
need to tweak my emacs-lisp compilation-error-regexp to handle stack
trace output from tests ...

On Tue, Nov 12, 2019 at 10:14 AM Dawid Weiss <[hidden email]> wrote:

>
> Also, I wrote a short help for running tests under Gradle branch -- if
> it's of some help to you, please read it!
>
> https://github.com/apache/lucene-solr/blob/jira/SOLR-13452_gradle_7/buildSrc/common/help-text/testHelp.txt
>
> On Tue, Nov 12, 2019 at 3:12 PM Michael Sokolov <[hidden email]> wrote:
> >
> > Hi I am playing around with the gradle build. Overall looks great!
> > Thanks to everyone who has been pushing this forward. I have a few
> > questions; maybe just gradle noob questions, since I haven't used it
> > much (except as part of Android Studio, where all the details are kind
> > of taken care of for you).
> >
> > 1) I'm not sure which branch is the "current" one. Ideally I'd like to
> > be using a branch based off master. I see there are lots of branches:
> > jira/SOLR-13452_gradle_2, 3, 4, ... I started with _7 since that is
> > what is referenced in the wiki:
> > https://cwiki.apache.org/confluence/display/SOLR/Intro+to+the+Gradle+build,
> > but that seems to be a little out of date, so I switched to 8, because
> > you know, bigger is better. What are all these numbers? Just tracking
> > snapshots along the way? Which is the one based off 8x?
> >
> > 2) When I run any gradle command I get this warning:
> >
> > > Configure project :
> > not user home user.gradle /home/sokolov/user.properties
> >
> > Its grammar is throwing me: does it mean it expects to find these
> > files and can't find them? I have no user.properties file in my
> > homedir: should I?
> >
> > 3) I can run tests in a package using (eg) ./gradlew
> > lucene:lucene-core:test and see the test report output in an html file
> > - cool. Is it possible to get test output to stdout though? I am used
> > to running tests in emacs and have a script set up for parsing stack
> > traces in the output so emacs can jump there. I know I can use
> > intellij, and I often do, but I would like to also get the emacs
> > workflow going - definitely should not be a blocker for switching to
> > this - I am just looking for some hints as to getting errors logged to
> > stdout by gradle.
> >
> > 4) If I use the --tests option to specify a single test class to run,
> > and the class does not exist (I made a booboo, say), the build runs no
> > tests, and succeeds, but this is misleading: it should fail instead,
> > as the ant build does. Again, not a blocker, but if anybody knows how
> > to fix this, it would be great. I'll open an issue anyway.
> >
> > again thanks, this looks awesome; with the daemon it runs so nicely :)
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]