Solr Gradle build (SOLR-13452)

classic Classic list List threaded Threaded
20 messages Options
Reply | Threaded
Open this post in threaded view
|

Solr Gradle build (SOLR-13452)

Erick Erickson
I’m trying to push this forward. I intend to keep working on the latest branch of Mark’s Gradle build (jira/SOLR-13452_gradle_8).

This is a plea for anyone interested to at least glance at any updates to that JIRA and chime in if you have any hints. Also, feel free to update that branch yourself as I don’t intend to create another branch, but merge the gradle_8 branch as soon as possible. We can treat it like any other branch that needs work before merging.

I’ll try to keep the gradle_8 branch up to date with master.

I’m concentrating at this point on making the gradle and ant builds play nice together. I saw one situation where after running the Gradle build, “Ant precommit” failed. Turned out to be a mismatch in the versions of some of the jars Solr depends on. What I want people to be able to do is:

> try to use Gradle to do whatever task they need to accomplish. If there’s no equivalent, then quickly switch to Ant. Or, at worst, execute some target in Gradle (pristineClean?) to reset to zero and then use Ant.

> get to the minimal point that allows us to merge into master. So far, the primary objection (private conversation) is that there’s no equivalent of “ant precommit”. I haven’t checked this out in detail yet.

Also, please feel free to add (or take) sub-tasks of SOLR-13914, where I’m trying to collect an organized list of issues so we can track them.

Best,
Erick
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Cassandra Targett
I was able to get the changes Dawid and I worked out in the gradle_7_refguide branch merged into the gradle_8 branch.

There’s one current problem with it, which may be something in my env: in the grade_7_refguide branch the HTML files were generated in ./solr/solr-ref-guide/build, but now they are in the top-level ./build directory.

I generally don’t really care where it gets built, but there are a few places in the docs where we pull in source code and those now throw an error because the path to the files is wrong. If the Ref Guide is going to get built there forever, it’s a tiny & simple fix to change the path, I just need to know if that’s the plan or if it’s a mistake. This problem also does not need to hold up merging that branch to master.

Cassandra
On Nov 11, 2019, 10:54 AM -0600, Erick Erickson <[hidden email]>, wrote:
I’m trying to push this forward. I intend to keep working on the latest branch of Mark’s Gradle build (jira/SOLR-13452_gradle_8).

This is a plea for anyone interested to at least glance at any updates to that JIRA and chime in if you have any hints. Also, feel free to update that branch yourself as I don’t intend to create another branch, but merge the gradle_8 branch as soon as possible. We can treat it like any other branch that needs work before merging.

I’ll try to keep the gradle_8 branch up to date with master.

I’m concentrating at this point on making the gradle and ant builds play nice together. I saw one situation where after running the Gradle build, “Ant precommit” failed. Turned out to be a mismatch in the versions of some of the jars Solr depends on. What I want people to be able to do is:

try to use Gradle to do whatever task they need to accomplish. If there’s no equivalent, then quickly switch to Ant. Or, at worst, execute some target in Gradle (pristineClean?) to reset to zero and then use Ant.

get to the minimal point that allows us to merge into master. So far, the primary objection (private conversation) is that there’s no equivalent of “ant precommit”. I haven’t checked this out in detail yet.

Also, please feel free to add (or take) sub-tasks of SOLR-13914, where I’m trying to collect an organized list of issues so we can track them.

Best,
Erick
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Dawid Weiss-2
> There’s one current problem with it, which may be something in my env: in the grade_7_refguide branch the HTML files were generated in ./solr/solr-ref-guide/build, but now they are in the top-level ./build directory.

I think Mark has configured gradle defaults to be specifically this way.

> I generally don’t really care where it gets built, but there are a few places in the docs where we pull in source code and those now throw an error because the path to the files is wrong. If the Ref Guide is going to get built there forever, it’s a tiny & simple fix to change the path, I just need to know if that’s the plan or if it’s a mistake. This problem also does not need to hold up merging that branch to master.

Ideally, Gradle tasks consume not explicit paths but configuration
artifacts or "outputs" of other tasks. This is elegant and
error-resilient. We can clean it up as we go -- feel free to create a
task in jira and assign it to me and I'll take a look.

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Erick Erickson
Dawid:

Cassandra is away for a bit, so I created SOLR-13922 and assigned it to you. I don’t have much real understanding of the problem so the JIRA statement may need amending.

Best,
Erick

> On Nov 12, 2019, at 4:05 AM, Dawid Weiss <[hidden email]> wrote:
>
>> There’s one current problem with it, which may be something in my env: in the grade_7_refguide branch the HTML files were generated in ./solr/solr-ref-guide/build, but now they are in the top-level ./build directory.
>
> I think Mark has configured gradle defaults to be specifically this way.
>
>> I generally don’t really care where it gets built, but there are a few places in the docs where we pull in source code and those now throw an error because the path to the files is wrong. If the Ref Guide is going to get built there forever, it’s a tiny & simple fix to change the path, I just need to know if that’s the plan or if it’s a mistake. This problem also does not need to hold up merging that branch to master.
>
> Ideally, Gradle tasks consume not explicit paths but configuration
> artifacts or "outputs" of other tasks. This is elegant and
> error-resilient. We can clean it up as we go -- feel free to create a
> task in jira and assign it to me and I'll take a look.
>
> D.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Erick Erickson
In reply to this post by Dawid Weiss-2
I just pulled down the new gradle_8 branch and got this error:

* What went wrong:
Execution failed for task ':solr:solr-ref-guide:compileToolsJava'.
> Could not resolve all files for configuration ':solr:solr-ref-guide:tools'.
   > Could not find org.asciidoctor:asciidoctorj:.
     Required by:
         project :solr:solr-ref-guide

I “fixed" it by adding the below to the versions.props file, but have no idea if that’s correct or not, it’s just the latest release of asciidoctorj I found on a quick google search.

org.asciidoctor:asciidoctorj=2.1.0

Is this OK?

> On Nov 12, 2019, at 4:05 AM, Dawid Weiss <[hidden email]> wrote:
>
>> There’s one current problem with it, which may be something in my env: in the grade_7_refguide branch the HTML files were generated in ./solr/solr-ref-guide/build, but now they are in the top-level ./build directory.
>
> I think Mark has configured gradle defaults to be specifically this way.
>
>> I generally don’t really care where it gets built, but there are a few places in the docs where we pull in source code and those now throw an error because the path to the files is wrong. If the Ref Guide is going to get built there forever, it’s a tiny & simple fix to change the path, I just need to know if that’s the plan or if it’s a mistake. This problem also does not need to hold up merging that branch to master.
>
> Ideally, Gradle tasks consume not explicit paths but configuration
> artifacts or "outputs" of other tasks. This is elegant and
> error-resilient. We can clean it up as we go -- feel free to create a
> task in jira and assign it to me and I'll take a look.
>
> D.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Dawid Weiss-2
What command did you use? Are you sure you're running the "wrapper"
script (gradlew) and not just any gradle version installed on the
system?

I ask because I just ran this and everything is fine:
./gradlew -p solr/solr-ref-guide buildSite

The exception you're getting indicates a missing dependency but this
dependency is present in versions.props on that branch:

org.asciidoctor:asciidoctorj=1.6.0-alpha.5

Dawid

On Tue, Nov 12, 2019 at 9:26 PM Erick Erickson <[hidden email]> wrote:

>
> I just pulled down the new gradle_8 branch and got this error:
>
> * What went wrong:
> Execution failed for task ':solr:solr-ref-guide:compileToolsJava'.
> > Could not resolve all files for configuration ':solr:solr-ref-guide:tools'.
>    > Could not find org.asciidoctor:asciidoctorj:.
>      Required by:
>          project :solr:solr-ref-guide
>
> I “fixed" it by adding the below to the versions.props file, but have no idea if that’s correct or not, it’s just the latest release of asciidoctorj I found on a quick google search.
>
> org.asciidoctor:asciidoctorj=2.1.0
>
> Is this OK?
>
> > On Nov 12, 2019, at 4:05 AM, Dawid Weiss <[hidden email]> wrote:
> >
> >> There’s one current problem with it, which may be something in my env: in the grade_7_refguide branch the HTML files were generated in ./solr/solr-ref-guide/build, but now they are in the top-level ./build directory.
> >
> > I think Mark has configured gradle defaults to be specifically this way.
> >
> >> I generally don’t really care where it gets built, but there are a few places in the docs where we pull in source code and those now throw an error because the path to the files is wrong. If the Ref Guide is going to get built there forever, it’s a tiny & simple fix to change the path, I just need to know if that’s the plan or if it’s a mistake. This problem also does not need to hold up merging that branch to master.
> >
> > Ideally, Gradle tasks consume not explicit paths but configuration
> > artifacts or "outputs" of other tasks. This is elegant and
> > error-resilient. We can clean it up as we go -- feel free to create a
> > task in jira and assign it to me and I'll take a look.
> >
> > D.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Anshum Gupta
In reply to this post by Erick Erickson
Looking at the following file:
solr/solr-ref-guide/build.gradle

I see the apparently missing dependency:

    dependencies {
        classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
    }

I also see this TODO:

// TODO 4: currently buildscript dependencies are hardcoded (asciidoctor) because they can't be resolved
// using Palantir's plugin. This is another reason to switch to gradle-based tools -- then
// only the build script dependencies would be needed and asciidoctorj would be removed from version
// properties entirely (it is only used locally in this build file).

I don't think we need to add this line, specially because I'm able to run the buildSite target fine.

Can you provide more information w.r.t. how you ran this?

-Anshum

On Tue, Nov 12, 2019 at 12:30 PM Erick Erickson <[hidden email]> wrote:
I just pulled down the new gradle_8 branch and got this error:

* What went wrong:
Execution failed for task ':solr:solr-ref-guide:compileToolsJava'.
> Could not resolve all files for configuration ':solr:solr-ref-guide:tools'.
   > Could not find org.asciidoctor:asciidoctorj:.
     Required by:
         project :solr:solr-ref-guide

I “fixed" it by adding the below to the versions.props file, but have no idea if that’s correct or not, it’s just the latest release of asciidoctorj I found on a quick google search.

org.asciidoctor:asciidoctorj=2.1.0

Is this OK?

> On Nov 12, 2019, at 4:05 AM, Dawid Weiss <[hidden email]> wrote:
>
>> There’s one current problem with it, which may be something in my env: in the grade_7_refguide branch the HTML files were generated in ./solr/solr-ref-guide/build, but now they are in the top-level ./build directory.
>
> I think Mark has configured gradle defaults to be specifically this way.
>
>> I generally don’t really care where it gets built, but there are a few places in the docs where we pull in source code and those now throw an error because the path to the files is wrong. If the Ref Guide is going to get built there forever, it’s a tiny & simple fix to change the path, I just need to know if that’s the plan or if it’s a mistake. This problem also does not need to hold up merging that branch to master.
>
> Ideally, Gradle tasks consume not explicit paths but configuration
> artifacts or "outputs" of other tasks. This is elegant and
> error-resilient. We can clean it up as we go -- feel free to create a
> task in jira and assign it to me and I'll take a look.
>
> D.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--
Anshum Gupta
Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Dawid Weiss-2
> Looking at the following file:
> solr/solr-ref-guide/build.gradle
>
> I see the apparently missing dependency:

The buildscript dependency is there because, well, we use it from the
script itself.

For the "tools" configuration classpath the dependency is not missing,
it's correct without the
version number (palantir's plugin resolves it):

    tools('org.asciidoctor:asciidoctorj')

>     dependencies {
>         classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
>     }

This isn't correct, don't add it. The existing config in that script
is fine (and works for me); I think Erik's problem is caused by
something else.

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Anshum Gupta
I didn't add this. This is already in the gradle_8 branch.

What branch are you looking at? Don't you see this too ?


On Tue, Nov 12, 2019 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
> Looking at the following file:
> solr/solr-ref-guide/build.gradle
>
> I see the apparently missing dependency:

The buildscript dependency is there because, well, we use it from the
script itself.

For the "tools" configuration classpath the dependency is not missing,
it's correct without the
version number (palantir's plugin resolves it):

    tools('org.asciidoctor:asciidoctorj')

>     dependencies {
>         classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
>     }

This isn't correct, don't add it. The existing config in that script
is fine (and works for me); I think Erik's problem is caused by
something else.

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--
Anshum Gupta
Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Dawid Weiss-2
Oh, sorry -- I thought you meant this line to be added to the main
dependencies section (not buildScript's).

D.

On Tue, Nov 12, 2019 at 10:14 PM Anshum Gupta <[hidden email]> wrote:

>
> I didn't add this. This is already in the gradle_8 branch.
>
> What branch are you looking at? Don't you see this too ?
>
>
> On Tue, Nov 12, 2019 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
>>
>> > Looking at the following file:
>> > solr/solr-ref-guide/build.gradle
>> >
>> > I see the apparently missing dependency:
>>
>> The buildscript dependency is there because, well, we use it from the
>> script itself.
>>
>> For the "tools" configuration classpath the dependency is not missing,
>> it's correct without the
>> version number (palantir's plugin resolves it):
>>
>>     tools('org.asciidoctor:asciidoctorj')
>>
>> >     dependencies {
>> >         classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
>> >     }
>>
>> This isn't correct, don't add it. The existing config in that script
>> is fine (and works for me); I think Erik's problem is caused by
>> something else.
>>
>> D.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
>
> --
> Anshum Gupta

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Erick Erickson
Dawid:

I took my change back out and I’m failing again. Both
gw jarChecksums
and
./gradlew build -x test

error out with:

Execution failed for task ':solr:jarChecksums'.
> Could not resolve all dependencies for configuration ':solr:solr-ref-guide:tools'.
   > Could not find org.asciidoctor:asciidoctorj:.
     Required by:
         project :solr:solr-ref-guide



> On Nov 12, 2019, at 4:35 PM, Dawid Weiss <[hidden email]> wrote:
>
> Oh, sorry -- I thought you meant this line to be added to the main
> dependencies section (not buildScript's).
>
> D.
>
> On Tue, Nov 12, 2019 at 10:14 PM Anshum Gupta <[hidden email]> wrote:
>>
>> I didn't add this. This is already in the gradle_8 branch.
>>
>> What branch are you looking at? Don't you see this too ?
>>
>>
>> On Tue, Nov 12, 2019 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
>>>
>>>> Looking at the following file:
>>>> solr/solr-ref-guide/build.gradle
>>>>
>>>> I see the apparently missing dependency:
>>>
>>> The buildscript dependency is there because, well, we use it from the
>>> script itself.
>>>
>>> For the "tools" configuration classpath the dependency is not missing,
>>> it's correct without the
>>> version number (palantir's plugin resolves it):
>>>
>>>    tools('org.asciidoctor:asciidoctorj')
>>>
>>>>    dependencies {
>>>>        classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
>>>>    }
>>>
>>> This isn't correct, don't add it. The existing config in that script
>>> is fine (and works for me); I think Erik's problem is caused by
>>> something else.
>>>
>>> D.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>> --
>> Anshum Gupta
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Erick Erickson
In reply to this post by Dawid Weiss-2
Apparently
org.asciidoctor:asciidoctorj=1.6.2

also cures my problem. Mark had put
org.asciidoctor:asciidoctorj=1.6.0-alpha.5

in the top level versions.props so I don’t quite know if it belongs there or not.

Erick

> On Nov 12, 2019, at 4:35 PM, Dawid Weiss <[hidden email]> wrote:
>
> Oh, sorry -- I thought you meant this line to be added to the main
> dependencies section (not buildScript's).
>
> D.
>
> On Tue, Nov 12, 2019 at 10:14 PM Anshum Gupta <[hidden email]> wrote:
>>
>> I didn't add this. This is already in the gradle_8 branch.
>>
>> What branch are you looking at? Don't you see this too ?
>>
>>
>> On Tue, Nov 12, 2019 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
>>>
>>>> Looking at the following file:
>>>> solr/solr-ref-guide/build.gradle
>>>>
>>>> I see the apparently missing dependency:
>>>
>>> The buildscript dependency is there because, well, we use it from the
>>> script itself.
>>>
>>> For the "tools" configuration classpath the dependency is not missing,
>>> it's correct without the
>>> version number (palantir's plugin resolves it):
>>>
>>>    tools('org.asciidoctor:asciidoctorj')
>>>
>>>>    dependencies {
>>>>        classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
>>>>    }
>>>
>>> This isn't correct, don't add it. The existing config in that script
>>> is fine (and works for me); I think Erik's problem is caused by
>>> something else.
>>>
>>> D.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>> --
>> Anshum Gupta
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Erick Erickson
In reply to this post by Dawid Weiss-2
Final note before I turn in. “ant -Dtestcase=TestKoreanTokenizer” fails OOB on the gradle_8 branch with:

java.lang.RuntimeException: Cannot find userdict.txt in test classpath!

I see on master that that file is copied into
lucene/build/analysis/nori/classes/test/org/apache/lucene/analysis/ko/userdict.txt

I’ll look at it more tomorrow.

After that gets fixed, and pending the bits about asciidoctorj, I think it’s close to ready to push to master and iterate there.

Erick

> On Nov 12, 2019, at 4:35 PM, Dawid Weiss <[hidden email]> wrote:
>
> Oh, sorry -- I thought you meant this line to be added to the main
> dependencies section (not buildScript's).
>
> D.
>
> On Tue, Nov 12, 2019 at 10:14 PM Anshum Gupta <[hidden email]> wrote:
>>
>> I didn't add this. This is already in the gradle_8 branch.
>>
>> What branch are you looking at? Don't you see this too ?
>>
>>
>> On Tue, Nov 12, 2019 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
>>>
>>>> Looking at the following file:
>>>> solr/solr-ref-guide/build.gradle
>>>>
>>>> I see the apparently missing dependency:
>>>
>>> The buildscript dependency is there because, well, we use it from the
>>> script itself.
>>>
>>> For the "tools" configuration classpath the dependency is not missing,
>>> it's correct without the
>>> version number (palantir's plugin resolves it):
>>>
>>>    tools('org.asciidoctor:asciidoctorj')
>>>
>>>>    dependencies {
>>>>        classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
>>>>    }
>>>
>>> This isn't correct, don't add it. The existing config in that script
>>> is fine (and works for me); I think Erik's problem is caused by
>>> something else.
>>>
>>> D.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>> --
>> Anshum Gupta
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Dawid Weiss-2
In reply to this post by Erick Erickson
I would really like you help out but you seem special. ;) Is your
checkout clean (git clean -xfd .), do you have any locally modified
files (git status)? I just ran ./gradlew jarChecksums on two different
machines (Windows, Linux) and everything passed, look:

:~/work-ssd/lucene-solr$ ./gradlew jarChecksums
Starting a Gradle Daemon, 1 incompatible Daemon could not be reused,
use --status for details

> Task :buildSrc:jar
:jar: No valid plugin descriptors were found in META-INF/gradle-plugins

> Configure project :
not user home user.gradle /home/dweiss/user.properties

BUILD SUCCESSFUL in 9m 19s
2 actionable tasks: 2 executed

> Execution failed for task ':solr:jarChecksums'.
> > Could not resolve all dependencies for configuration ':solr:solr-ref-guide:tools'.
>    > Could not find org.asciidoctor:asciidoctorj:.
>      Required by:
>          project :solr:solr-ref-guide

The dependency is defined in versions.props so I don't see any reason
why it'd be empty (log above). That changing version number to 1.6.2
makes the above pass for you is an even larger head-scratcher...

Make sure you have a clean unmodified checkout, stop any running
daemons (gradlew --stop) and try again with "--debug" switch passed to
gradlew -- send the output to me in private; maybe I'll be able to
figure it out (but I don't promise anything - palantir's plugin does
some black magic stuff around dependency resolution so it may as well
be a bug in their implementation somewhere).

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Anshum Gupta
In reply to this post by Erick Erickson
Hey Erick,

Any updates on this? Just checking in and though I'm not a gradle guru, I'd be happy to help as much as I can :)

-Anshum

On Tue, Nov 12, 2019 at 7:32 PM Erick Erickson <[hidden email]> wrote:
Final note before I turn in. “ant -Dtestcase=TestKoreanTokenizer” fails OOB on the gradle_8 branch with:

java.lang.RuntimeException: Cannot find userdict.txt in test classpath!

I see on master that that file is copied into
lucene/build/analysis/nori/classes/test/org/apache/lucene/analysis/ko/userdict.txt

I’ll look at it more tomorrow.

After that gets fixed, and pending the bits about asciidoctorj, I think it’s close to ready to push to master and iterate there.

Erick

> On Nov 12, 2019, at 4:35 PM, Dawid Weiss <[hidden email]> wrote:
>
> Oh, sorry -- I thought you meant this line to be added to the main
> dependencies section (not buildScript's).
>
> D.
>
> On Tue, Nov 12, 2019 at 10:14 PM Anshum Gupta <[hidden email]> wrote:
>>
>> I didn't add this. This is already in the gradle_8 branch.
>>
>> What branch are you looking at? Don't you see this too ?
>>
>>
>> On Tue, Nov 12, 2019 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
>>>
>>>> Looking at the following file:
>>>> solr/solr-ref-guide/build.gradle
>>>>
>>>> I see the apparently missing dependency:
>>>
>>> The buildscript dependency is there because, well, we use it from the
>>> script itself.
>>>
>>> For the "tools" configuration classpath the dependency is not missing,
>>> it's correct without the
>>> version number (palantir's plugin resolves it):
>>>
>>>    tools('org.asciidoctor:asciidoctorj')
>>>
>>>>    dependencies {
>>>>        classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
>>>>    }
>>>
>>> This isn't correct, don't add it. The existing config in that script
>>> is fine (and works for me); I think Erik's problem is caused by
>>> something else.
>>>
>>> D.
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>> --
>> Anshum Gupta
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--
Anshum Gupta
Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Erick Erickson
Anshum et. al. :

It’s been a busy day, never got back to it. I’ll try to push my changes, mostly updating the dependencies tonight.

It’s not ready to merge into master yet, there’s this problem:

ant -Dtestcase=TestKoreanTokenizer test

fails with:

java.lang.RuntimeException: Cannot find userdict.txt in test classpath!

userdict.txt gets copied when I test on the trunk branch to (at least I think this is the corresponding one):

./lucene/build/analysis/nori/classes/test/org/apache/lucene/analysis/ko/userdict.txt

So my presumption is that the ant build takes care of this and somehow the classpath is set to include it.

This is on a clean checkout of the current gradle_8 branch, _without_ trying to do anything with Gradle.

I’m thrashing by trying to both update the dependencies and fix this problem at once, so I’ll concentrate on the dependencies tonight an (hopefully) get that pushed.

If you have time and energy to tackle the TestKoreanTokenizer bit, that’d be great. Otherwise I may get back to it Friday or this weekend.

Thanks!
Erick

> On Nov 13, 2019, at 6:52 PM, Anshum Gupta <[hidden email]> wrote:
>
> Hey Erick,
>
> Any updates on this? Just checking in and though I'm not a gradle guru, I'd be happy to help as much as I can :)
>
> -Anshum
>
> On Tue, Nov 12, 2019 at 7:32 PM Erick Erickson <[hidden email]> wrote:
> Final note before I turn in. “ant -Dtestcase=TestKoreanTokenizer” fails OOB on the gradle_8 branch with:
>
> java.lang.RuntimeException: Cannot find userdict.txt in test classpath!
>
> I see on master that that file is copied into
> lucene/build/analysis/nori/classes/test/org/apache/lucene/analysis/ko/userdict.txt
>
> I’ll look at it more tomorrow.
>
> After that gets fixed, and pending the bits about asciidoctorj, I think it’s close to ready to push to master and iterate there.
>
> Erick
>
> > On Nov 12, 2019, at 4:35 PM, Dawid Weiss <[hidden email]> wrote:
> >
> > Oh, sorry -- I thought you meant this line to be added to the main
> > dependencies section (not buildScript's).
> >
> > D.
> >
> > On Tue, Nov 12, 2019 at 10:14 PM Anshum Gupta <[hidden email]> wrote:
> >>
> >> I didn't add this. This is already in the gradle_8 branch.
> >>
> >> What branch are you looking at? Don't you see this too ?
> >>
> >>
> >> On Tue, Nov 12, 2019 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
> >>>
> >>>> Looking at the following file:
> >>>> solr/solr-ref-guide/build.gradle
> >>>>
> >>>> I see the apparently missing dependency:
> >>>
> >>> The buildscript dependency is there because, well, we use it from the
> >>> script itself.
> >>>
> >>> For the "tools" configuration classpath the dependency is not missing,
> >>> it's correct without the
> >>> version number (palantir's plugin resolves it):
> >>>
> >>>    tools('org.asciidoctor:asciidoctorj')
> >>>
> >>>>    dependencies {
> >>>>        classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
> >>>>    }
> >>>
> >>> This isn't correct, don't add it. The existing config in that script
> >>> is fine (and works for me); I think Erik's problem is caused by
> >>> something else.
> >>>
> >>> D.
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: [hidden email]
> >>> For additional commands, e-mail: [hidden email]
> >>>
> >>
> >>
> >> --
> >> Anshum Gupta
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>
> --
> Anshum Gupta


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Erick Erickson
It looks like it’ll be very difficult to have the “jar checksum” stuff work seamlessly between Ant and Gradle, they tend to overwrite each other resulting in a zillion changes depending on which one you ran last. I see three options:

1> tell people NOT to run the Gradle jarChecksums. I doubt this’ll really work since some ops may require it.
2> Deal with a zillion checkins between the two
3> since these are all (?) in license directories, make separate ones for the gradle build and the lucene build.

I’ll work on <3> for a while tonight unless people have objections.

Erick

> On Nov 13, 2019, at 7:26 PM, Erick Erickson <[hidden email]> wrote:
>
> Anshum et. al. :
>
> It’s been a busy day, never got back to it. I’ll try to push my changes, mostly updating the dependencies tonight.
>
> It’s not ready to merge into master yet, there’s this problem:
>
> ant -Dtestcase=TestKoreanTokenizer test
>
> fails with:
>
> java.lang.RuntimeException: Cannot find userdict.txt in test classpath!
>
> userdict.txt gets copied when I test on the trunk branch to (at least I think this is the corresponding one):
>
> ./lucene/build/analysis/nori/classes/test/org/apache/lucene/analysis/ko/userdict.txt
>
> So my presumption is that the ant build takes care of this and somehow the classpath is set to include it.
>
> This is on a clean checkout of the current gradle_8 branch, _without_ trying to do anything with Gradle.
>
> I’m thrashing by trying to both update the dependencies and fix this problem at once, so I’ll concentrate on the dependencies tonight an (hopefully) get that pushed.
>
> If you have time and energy to tackle the TestKoreanTokenizer bit, that’d be great. Otherwise I may get back to it Friday or this weekend.
>
> Thanks!
> Erick
>
>> On Nov 13, 2019, at 6:52 PM, Anshum Gupta <[hidden email]> wrote:
>>
>> Hey Erick,
>>
>> Any updates on this? Just checking in and though I'm not a gradle guru, I'd be happy to help as much as I can :)
>>
>> -Anshum
>>
>> On Tue, Nov 12, 2019 at 7:32 PM Erick Erickson <[hidden email]> wrote:
>> Final note before I turn in. “ant -Dtestcase=TestKoreanTokenizer” fails OOB on the gradle_8 branch with:
>>
>> java.lang.RuntimeException: Cannot find userdict.txt in test classpath!
>>
>> I see on master that that file is copied into
>> lucene/build/analysis/nori/classes/test/org/apache/lucene/analysis/ko/userdict.txt
>>
>> I’ll look at it more tomorrow.
>>
>> After that gets fixed, and pending the bits about asciidoctorj, I think it’s close to ready to push to master and iterate there.
>>
>> Erick
>>
>>> On Nov 12, 2019, at 4:35 PM, Dawid Weiss <[hidden email]> wrote:
>>>
>>> Oh, sorry -- I thought you meant this line to be added to the main
>>> dependencies section (not buildScript's).
>>>
>>> D.
>>>
>>> On Tue, Nov 12, 2019 at 10:14 PM Anshum Gupta <[hidden email]> wrote:
>>>>
>>>> I didn't add this. This is already in the gradle_8 branch.
>>>>
>>>> What branch are you looking at? Don't you see this too ?
>>>>
>>>>
>>>> On Tue, Nov 12, 2019 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
>>>>>
>>>>>> Looking at the following file:
>>>>>> solr/solr-ref-guide/build.gradle
>>>>>>
>>>>>> I see the apparently missing dependency:
>>>>>
>>>>> The buildscript dependency is there because, well, we use it from the
>>>>> script itself.
>>>>>
>>>>> For the "tools" configuration classpath the dependency is not missing,
>>>>> it's correct without the
>>>>> version number (palantir's plugin resolves it):
>>>>>
>>>>>   tools('org.asciidoctor:asciidoctorj')
>>>>>
>>>>>>   dependencies {
>>>>>>       classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
>>>>>>   }
>>>>>
>>>>> This isn't correct, don't add it. The existing config in that script
>>>>> is fine (and works for me); I think Erik's problem is caused by
>>>>> something else.
>>>>>
>>>>> D.
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> For additional commands, e-mail: [hidden email]
>>>>>
>>>>
>>>>
>>>> --
>>>> Anshum Gupta
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>>
>> --
>> Anshum Gupta
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Erick Erickson
In reply to this post by Anshum Gupta
OK, so I went ahead and added my changes to make parallel license directories for the nonce (Lucene and Solr).

Dawid: If you’d like to see if this now breaks for you like it did for me, feel free ;). It works for me _AFTER_ I replaced the line in the top level versions.props file (this is what I just pushed):

org.asciidoctor:asciidoctorj=1.6.0-alpha.5

with

org.asciidoctor:asciidoctor-ant=1.6.2
org.asciidoctor:asciidoctorj=1.6.2

If I remove one or both, and try “gw jarChecksums” Bad Things Happen. Whether that’s because I have something polluted in my env is an open question.

Let me know if you’d like me to take them out and send you the debug info, I’m all confused about what should be happening here.

> On Nov 13, 2019, at 6:52 PM, Anshum Gupta <[hidden email]> wrote:
>
> Hey Erick,
>
> Any updates on this? Just checking in and though I'm not a gradle guru, I'd be happy to help as much as I can :)
>
> -Anshum
>
> On Tue, Nov 12, 2019 at 7:32 PM Erick Erickson <[hidden email]> wrote:
> Final note before I turn in. “ant -Dtestcase=TestKoreanTokenizer” fails OOB on the gradle_8 branch with:
>
> java.lang.RuntimeException: Cannot find userdict.txt in test classpath!
>
> I see on master that that file is copied into
> lucene/build/analysis/nori/classes/test/org/apache/lucene/analysis/ko/userdict.txt
>
> I’ll look at it more tomorrow.
>
> After that gets fixed, and pending the bits about asciidoctorj, I think it’s close to ready to push to master and iterate there.
>
> Erick
>
> > On Nov 12, 2019, at 4:35 PM, Dawid Weiss <[hidden email]> wrote:
> >
> > Oh, sorry -- I thought you meant this line to be added to the main
> > dependencies section (not buildScript's).
> >
> > D.
> >
> > On Tue, Nov 12, 2019 at 10:14 PM Anshum Gupta <[hidden email]> wrote:
> >>
> >> I didn't add this. This is already in the gradle_8 branch.
> >>
> >> What branch are you looking at? Don't you see this too ?
> >>
> >>
> >> On Tue, Nov 12, 2019 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
> >>>
> >>>> Looking at the following file:
> >>>> solr/solr-ref-guide/build.gradle
> >>>>
> >>>> I see the apparently missing dependency:
> >>>
> >>> The buildscript dependency is there because, well, we use it from the
> >>> script itself.
> >>>
> >>> For the "tools" configuration classpath the dependency is not missing,
> >>> it's correct without the
> >>> version number (palantir's plugin resolves it):
> >>>
> >>>    tools('org.asciidoctor:asciidoctorj')
> >>>
> >>>>    dependencies {
> >>>>        classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
> >>>>    }
> >>>
> >>> This isn't correct, don't add it. The existing config in that script
> >>> is fine (and works for me); I think Erik's problem is caused by
> >>> something else.
> >>>
> >>> D.
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: [hidden email]
> >>> For additional commands, e-mail: [hidden email]
> >>>
> >>
> >>
> >> --
> >> Anshum Gupta
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>
> --
> Anshum Gupta


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Anshum Gupta
In reply to this post by Erick Erickson
Sorry, I am just looking at this now and about to call it a night soon. 

I'll try and get to it tomorrow/todayi.e Friday but if you get to it before I do, please move forward.

On Wed, Nov 13, 2019 at 4:26 PM Erick Erickson <[hidden email]> wrote:
Anshum et. al. :

It’s been a busy day, never got back to it. I’ll try to push my changes, mostly updating the dependencies tonight.

It’s not ready to merge into master yet, there’s this problem:

ant -Dtestcase=TestKoreanTokenizer test

fails with:

java.lang.RuntimeException: Cannot find userdict.txt in test classpath!

userdict.txt gets copied when I test on the trunk branch to (at least I think this is the corresponding one):

./lucene/build/analysis/nori/classes/test/org/apache/lucene/analysis/ko/userdict.txt

So my presumption is that the ant build takes care of this and somehow the classpath is set to include it.

This is on a clean checkout of the current gradle_8 branch, _without_ trying to do anything with Gradle.

I’m thrashing by trying to both update the dependencies and fix this problem at once, so I’ll concentrate on the dependencies tonight an (hopefully) get that pushed.

If you have time and energy to tackle the TestKoreanTokenizer bit, that’d be great. Otherwise I may get back to it Friday or this weekend.

Thanks!
Erick

> On Nov 13, 2019, at 6:52 PM, Anshum Gupta <[hidden email]> wrote:
>
> Hey Erick,
>
> Any updates on this? Just checking in and though I'm not a gradle guru, I'd be happy to help as much as I can :)
>
> -Anshum
>
> On Tue, Nov 12, 2019 at 7:32 PM Erick Erickson <[hidden email]> wrote:
> Final note before I turn in. “ant -Dtestcase=TestKoreanTokenizer” fails OOB on the gradle_8 branch with:
>
> java.lang.RuntimeException: Cannot find userdict.txt in test classpath!
>
> I see on master that that file is copied into
> lucene/build/analysis/nori/classes/test/org/apache/lucene/analysis/ko/userdict.txt
>
> I’ll look at it more tomorrow.
>
> After that gets fixed, and pending the bits about asciidoctorj, I think it’s close to ready to push to master and iterate there.
>
> Erick
>
> > On Nov 12, 2019, at 4:35 PM, Dawid Weiss <[hidden email]> wrote:
> >
> > Oh, sorry -- I thought you meant this line to be added to the main
> > dependencies section (not buildScript's).
> >
> > D.
> >
> > On Tue, Nov 12, 2019 at 10:14 PM Anshum Gupta <[hidden email]> wrote:
> >>
> >> I didn't add this. This is already in the gradle_8 branch.
> >>
> >> What branch are you looking at? Don't you see this too ?
> >>
> >>
> >> On Tue, Nov 12, 2019 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
> >>>
> >>>> Looking at the following file:
> >>>> solr/solr-ref-guide/build.gradle
> >>>>
> >>>> I see the apparently missing dependency:
> >>>
> >>> The buildscript dependency is there because, well, we use it from the
> >>> script itself.
> >>>
> >>> For the "tools" configuration classpath the dependency is not missing,
> >>> it's correct without the
> >>> version number (palantir's plugin resolves it):
> >>>
> >>>    tools('org.asciidoctor:asciidoctorj')
> >>>
> >>>>    dependencies {
> >>>>        classpath "org.asciidoctor:asciidoctorj:1.6.0-alpha.5"
> >>>>    }
> >>>
> >>> This isn't correct, don't add it. The existing config in that script
> >>> is fine (and works for me); I think Erik's problem is caused by
> >>> something else.
> >>>
> >>> D.
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: [hidden email]
> >>> For additional commands, e-mail: [hidden email]
> >>>
> >>
> >>
> >> --
> >> Anshum Gupta
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>
> --
> Anshum Gupta


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--
Anshum Gupta
Reply | Threaded
Open this post in threaded view
|

Re: Solr Gradle build (SOLR-13452)

Dawid Weiss-2
In reply to this post by Erick Erickson
> org.asciidoctor:asciidoctorj=1.6.0-alpha.5
>
> with
>
> org.asciidoctor:asciidoctor-ant=1.6.2
> org.asciidoctor:asciidoctorj=1.6.2

You don't need to add asciidoctor-ant - it isn't used anywhere as an
explicit dependency.

> If I remove one or both, and try “gw jarChecksums” Bad Things Happen.

If you remove the "org.asciidoctor:asciidoctorj" then indeed the
dependency in solr-ref-guide/build.gradle won't be resolvable -- this
one:

    tools('org.asciidoctor:asciidoctorj')

the other line is not required. I checked gw jarChecksums and
everything works fine.

> Whether that’s because I have something polluted in my env is an open question.

Maybe. That's why I asked for the debug log.

Jar checksums work for Lucene for me now.

.\gradlew -p lucene checkLicenses --rerun-tasks

although the content of those folders differs for gradle and ant
(gradle has morfologik-fsa-2.1.6.jar.sha1 while ant has
morfologik-fsa-2.1.5.jar.sha1, for example).

Jar checksums for Solr don't work:

.\gradlew -p solr checkLicenses --rerun-tasks

this is because

1) the task that resolves them in gradle collects dependencies from
all configurations, regardless of whether they're used internally at
build time or truly exported at runtime. In ant we don't have (or
verify) all of these JARs. I don't know if we must (or should). An
example is the jruby dependency used for compiling the refguide. We
could add sha files, licenses, etc. to all of these but it'll be a
pain to upgrade anything after that:

Scanned 321 JAR file(s) for licenses (in 1.09s.), 129 error(s).

2) some exclusions of transitive dependencies would be required in the
gradle build to match ant's configs.

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]