analytics.adoc validation does not pass

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

analytics.adoc validation does not pass

Dawid Weiss-2
Hi Cassandra!

I get this when running precommit on gradle build (gradlew precommit):

> Task :validateSourcePatterns
[ant:groovy] Unescaped symbol "->" on line #43:
solr/solr-ref-guide/src/analytics.adoc
[ant:groovy] Unescaped symbol "->" on line #52:
solr/solr-ref-guide/src/analytics.adoc

I don't know if these are legitimate errors though or something that
needs to be improved in validation code.

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Dawid Weiss-2
Ok, I think it's the validation code at fault here. Mike -- would you
take a look at why the validation of solr-ref-guide doesn't pass?
There has to be a difference in infrastructure somewhere because we
use the same validation script...

Dawid

On Sun, Jan 26, 2020 at 10:55 AM Dawid Weiss <[hidden email]> wrote:

>
> Hi Cassandra!
>
> I get this when running precommit on gradle build (gradlew precommit):
>
> > Task :validateSourcePatterns
> [ant:groovy] Unescaped symbol "->" on line #43:
> solr/solr-ref-guide/src/analytics.adoc
> [ant:groovy] Unescaped symbol "->" on line #52:
> solr/solr-ref-guide/src/analytics.adoc
>
> I don't know if these are legitimate errors though or something that
> needs to be improved in validation code.
>
> Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Uwe Schindler
On Jenkins (Ant build) it only seems to happen on Windows. Which is strange.

Uwe

Am January 26, 2020 10:13:44 AM UTC schrieb Dawid Weiss <[hidden email]>:
Ok, I think it's the validation code at fault here. Mike -- would you
take a look at why the validation of solr-ref-guide doesn't pass?
There has to be a difference in infrastructure somewhere because we
use the same validation script...

Dawid

On Sun, Jan 26, 2020 at 10:55 AM Dawid Weiss <[hidden email]> wrote:

Hi Cassandra!

I get this when running precommit on gradle build (gradlew precommit):

Task :validateSourcePatterns
[ant:groovy] Unescaped symbol "->" on line #43:
solr/solr-ref-guide/src/analytics.adoc
[ant:groovy] Unescaped symbol "->" on line #52:
solr/solr-ref-guide/src/analytics.adoc

I don't know if these are legitimate errors though or something that
needs to be improved in validation code.

Dawid

To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de
Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Dawid Weiss-2
Sigh... It's because of the recent change to solr/.gitattributes
(SOLR-10713)... End of lines differ for adoc files. Jason -- please
change this one from auto to "don't touch line endings on any of other
files", whatever the magic is.

# Handles all files not treated particularly below
*          text=auto

Dawid

On Sun, Jan 26, 2020 at 11:20 AM Uwe Schindler <[hidden email]> wrote:

>
> On Jenkins (Ant build) it only seems to happen on Windows. Which is strange.
>
> Uwe
>
> Am January 26, 2020 10:13:44 AM UTC schrieb Dawid Weiss <[hidden email]>:
>>
>> Ok, I think it's the validation code at fault here. Mike -- would you
>> take a look at why the validation of solr-ref-guide doesn't pass?
>> There has to be a difference in infrastructure somewhere because we
>> use the same validation script...
>>
>> Dawid
>>
>> On Sun, Jan 26, 2020 at 10:55 AM Dawid Weiss <[hidden email]> wrote:
>>>
>>>
>>>  Hi Cassandra!
>>>
>>>  I get this when running precommit on gradle build (gradlew precommit):
>>>
>>>> Task :validateSourcePatterns
>>>
>>>  [ant:groovy] Unescaped symbol "->" on line #43:
>>>  solr/solr-ref-guide/src/analytics.adoc
>>>  [ant:groovy] Unescaped symbol "->" on line #52:
>>>  solr/solr-ref-guide/src/analytics.adoc
>>>
>>>  I don't know if these are legitimate errors though or something that
>>>  needs to be improved in validation code.
>>>
>>>  Dawid
>>
>> ________________________________
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Uwe Schindler
But we should nevertheless change the validator script to not fail on that. I think the split by lines regex is problematic.

Uwe

Am January 26, 2020 10:31:11 AM UTC schrieb Dawid Weiss <[hidden email]>:
Sigh... It's because of the recent change to solr/.gitattributes
(SOLR-10713)... End of lines differ for adoc files. Jason -- please
change this one from auto to "don't touch line endings on any of other
files", whatever the magic is.

# Handles all files not treated particularly below
* text=auto

Dawid

On Sun, Jan 26, 2020 at 11:20 AM Uwe Schindler <[hidden email]> wrote:

On Jenkins (Ant build) it only seems to happen on Windows. Which is strange.

Uwe

Am January 26, 2020 10:13:44 AM UTC schrieb Dawid Weiss <[hidden email]>:

Ok, I think it's the validation code at fault here. Mike -- would you
take a look at why the validation of solr-ref-guide doesn't pass?
There has to be a difference in infrastructure somewhere because we
use the same validation script...

Dawid

On Sun, Jan 26, 2020 at 10:55 AM Dawid Weiss <[hidden email]> wrote:


Hi Cassandra!

I get this when running precommit on gradle build (gradlew precommit):

Task :validateSourcePatterns

[ant:groovy] Unescaped symbol "->" on line #43:
solr/solr-ref-guide/src/analytics.adoc
[ant:groovy] Unescaped symbol "->" on line #52:
solr/solr-ref-guide/src/analytics.adoc

I don't know if these are legitimate errors though or something that
needs to be improved in validation code.

Dawid

To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de

--
Uwe Schindler
Achterdiek 19, 28357 Bremen
https://www.thetaphi.de
Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Dawid Weiss-2
Maybe. But I truly hate when git tries to alter those line endings for
me... this hasn't been the first time I spent an hour looking at what
the hell is wrong between Windows/ Linux on the same checkout...

D.

On Sun, Jan 26, 2020 at 11:34 AM Uwe Schindler <[hidden email]> wrote:

>
> But we should nevertheless change the validator script to not fail on that. I think the split by lines regex is problematic.
>
> Uwe
>
> Am January 26, 2020 10:31:11 AM UTC schrieb Dawid Weiss <[hidden email]>:
>>
>> Sigh... It's because of the recent change to solr/.gitattributes
>> (SOLR-10713)... End of lines differ for adoc files. Jason -- please
>> change this one from auto to "don't touch line endings on any of other
>> files", whatever the magic is.
>>
>> # Handles all files not treated particularly below
>> *          text=auto
>>
>> Dawid
>>
>> On Sun, Jan 26, 2020 at 11:20 AM Uwe Schindler <[hidden email]> wrote:
>>>
>>>
>>>  On Jenkins (Ant build) it only seems to happen on Windows. Which is strange.
>>>
>>>  Uwe
>>>
>>>  Am January 26, 2020 10:13:44 AM UTC schrieb Dawid Weiss <[hidden email]>:
>>>>
>>>>
>>>>  Ok, I think it's the validation code at fault here. Mike -- would you
>>>>  take a look at why the validation of solr-ref-guide doesn't pass?
>>>>  There has to be a difference in infrastructure somewhere because we
>>>>  use the same validation script...
>>>>
>>>>  Dawid
>>>>
>>>>  On Sun, Jan 26, 2020 at 10:55 AM Dawid Weiss <[hidden email]> wrote:
>>>>>
>>>>>
>>>>>
>>>>>   Hi Cassandra!
>>>>>
>>>>>   I get this when running precommit on gradle build (gradlew precommit):
>>>>>
>>>>>> Task :validateSourcePatterns
>>>>>
>>>>>
>>>>>   [ant:groovy] Unescaped symbol "->" on line #43:
>>>>>   solr/solr-ref-guide/src/analytics.adoc
>>>>>   [ant:groovy] Unescaped symbol "->" on line #52:
>>>>>   solr/solr-ref-guide/src/analytics.adoc
>>>>>
>>>>>   I don't know if these are legitimate errors though or something that
>>>>>   needs to be improved in validation code.
>>>>>
>>>>>   Dawid
>>>>
>>>> ________________________________
>>>>  To unsubscribe, e-mail: [hidden email]
>>>>  For additional commands, e-mail: [hidden email]
>>>>
>>>
>>>  --
>>>  Uwe Schindler
>>>  Achterdiek 19, 28357 Bremen
>>>  https://www.thetaphi.de
>
>
> --
> Uwe Schindler
> Achterdiek 19, 28357 Bremen
> https://www.thetaphi.de

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Mikhail Khludnev-2
In reply to this post by Dawid Weiss-2
Looking into.

On Sun, Jan 26, 2020 at 12:56 PM Dawid Weiss <[hidden email]> wrote:
Hi Cassandra!

I get this when running precommit on gradle build (gradlew precommit):

> Task :validateSourcePatterns
[ant:groovy] Unescaped symbol "->" on line #43:
solr/solr-ref-guide/src/analytics.adoc
[ant:groovy] Unescaped symbol "->" on line #52:
solr/solr-ref-guide/src/analytics.adoc

I don't know if these are legitimate errors though or something that
needs to be improved in validation code.

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



--
Sincerely yours
Mikhail Khludnev
Reply | Threaded
Open this post in threaded view
|

RE: analytics.adoc validation does not pass

Uwe Schindler
In reply to this post by Dawid Weiss-2
I found the bug, it's a stupid one:

The line splitter is:
def singleLineSplitter = ~$/\n\r?/$;

But should be (as it's \r\n on windows not the other way round):
def singleLineSplitter = ~$/\r?\n/$;

I'll commit the fix. It's just plain wrong!

Uwe

-----
Uwe Schindler
Achterdiek 19, D-28357 Bremen
https://www.thetaphi.de
eMail: [hidden email]

> -----Original Message-----
> From: Dawid Weiss <[hidden email]>
> Sent: Sunday, January 26, 2020 11:35 AM
> To: Uwe Schindler <[hidden email]>
> Cc: [hidden email]; Lucene Dev <[hidden email]>; Cassandra
> Targett <[hidden email]>; [hidden email]
> Subject: Re: analytics.adoc validation does not pass
>
> Maybe. But I truly hate when git tries to alter those line endings for
> me... this hasn't been the first time I spent an hour looking at what
> the hell is wrong between Windows/ Linux on the same checkout...
>
> D.
>
> On Sun, Jan 26, 2020 at 11:34 AM Uwe Schindler <[hidden email]> wrote:
> >
> > But we should nevertheless change the validator script to not fail on that. I
> think the split by lines regex is problematic.
> >
> > Uwe
> >
> > Am January 26, 2020 10:31:11 AM UTC schrieb Dawid Weiss
> <[hidden email]>:
> >>
> >> Sigh... It's because of the recent change to solr/.gitattributes
> >> (SOLR-10713)... End of lines differ for adoc files. Jason -- please
> >> change this one from auto to "don't touch line endings on any of other
> >> files", whatever the magic is.
> >>
> >> # Handles all files not treated particularly below
> >> *          text=auto
> >>
> >> Dawid
> >>
> >> On Sun, Jan 26, 2020 at 11:20 AM Uwe Schindler <[hidden email]>
> wrote:
> >>>
> >>>
> >>>  On Jenkins (Ant build) it only seems to happen on Windows. Which is
> strange.
> >>>
> >>>  Uwe
> >>>
> >>>  Am January 26, 2020 10:13:44 AM UTC schrieb Dawid Weiss
> <[hidden email]>:
> >>>>
> >>>>
> >>>>  Ok, I think it's the validation code at fault here. Mike -- would you
> >>>>  take a look at why the validation of solr-ref-guide doesn't pass?
> >>>>  There has to be a difference in infrastructure somewhere because we
> >>>>  use the same validation script...
> >>>>
> >>>>  Dawid
> >>>>
> >>>>  On Sun, Jan 26, 2020 at 10:55 AM Dawid Weiss
> <[hidden email]> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>   Hi Cassandra!
> >>>>>
> >>>>>   I get this when running precommit on gradle build (gradlew
> precommit):
> >>>>>
> >>>>>> Task :validateSourcePatterns
> >>>>>
> >>>>>
> >>>>>   [ant:groovy] Unescaped symbol "->" on line #43:
> >>>>>   solr/solr-ref-guide/src/analytics.adoc
> >>>>>   [ant:groovy] Unescaped symbol "->" on line #52:
> >>>>>   solr/solr-ref-guide/src/analytics.adoc
> >>>>>
> >>>>>   I don't know if these are legitimate errors though or something that
> >>>>>   needs to be improved in validation code.
> >>>>>
> >>>>>   Dawid
> >>>>
> >>>> ________________________________
> >>>>  To unsubscribe, e-mail: [hidden email]
> >>>>  For additional commands, e-mail: [hidden email]
> >>>>
> >>>
> >>>  --
> >>>  Uwe Schindler
> >>>  Achterdiek 19, 28357 Bremen
> >>>  https://www.thetaphi.de
> >
> >
> > --
> > Uwe Schindler
> > Achterdiek 19, 28357 Bremen
> > https://www.thetaphi.de


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: analytics.adoc validation does not pass

Uwe Schindler
In reply to this post by Mikhail Khludnev-2

I fixed it, It was a bug in the checker script. Somebody defined Windows Line endings in a backwards way:

 

diff --git a/lucene/tools/src/groovy/check-source-patterns.groovy b/lucene/tools/src/groovy/check-source-patterns.groovy

index 62565e6..4688584 100644

--- a/lucene/tools/src/groovy/check-source-patterns.groovy

+++ b/lucene/tools/src/groovy/check-source-patterns.groovy

@@ -71,7 +71,7 @@ def javadocsPattern = ~$/(?sm)^\Q/**\E(.*?)\Q*/\E/$;

def javaCommentPattern = ~$/(?sm)^\Q/*\E(.*?)\Q*/\E/$;

def xmlCommentPattern = ~$/(?sm)\Q<!--\E(.*?)\Q-->\E/$;

def lineSplitter = ~$/[\r\n]+/$;

-def singleLineSplitter = ~$/\n\r?/$;

+def singleLineSplitter = ~$/\r?\n/$;

def licenseMatcher = Defaults.createDefaultMatcher();

def validLoggerPattern = ~$/(?

 

Uwe

 

-----

Uwe Schindler

Achterdiek 19, D-28357 Bremen

https://www.thetaphi.de

eMail: [hidden email]

 

From: Mikhail Khludnev <[hidden email]>
Sent: Sunday, January 26, 2020 11:46 AM
To: [hidden email]
Subject: Re: analytics.adoc validation does not pass

 

Looking into.

 

On Sun, Jan 26, 2020 at 12:56 PM Dawid Weiss <[hidden email]> wrote:

Hi Cassandra!

I get this when running precommit on gradle build (gradlew precommit):

> Task :validateSourcePatterns
[ant:groovy] Unescaped symbol "->" on line #43:
solr/solr-ref-guide/src/analytics.adoc
[ant:groovy] Unescaped symbol "->" on line #52:
solr/solr-ref-guide/src/analytics.adoc

I don't know if these are legitimate errors though or something that
needs to be improved in validation code.

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail:
[hidden email]
For additional commands, e-mail:
[hidden email]


 

--

Sincerely yours
Mikhail Khludnev

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Jason Gerlowski
> It's because of the recent change to solr/.gitattributes
> (SOLR-10713) [sic - SOLR-14186] End of lines differ for adoc files. Jason -- please
> change this one from auto to "don't touch line endings on any of other
> files", whatever the magic is.

Since the issue ended up being a bad regex in the build task, do you
still want me to exclude adoc files specifically from gitattributes
Dawid?  I'll make the change if there's some particular reason why
adoc files should be exempted here.  But at a glance it seems like
gitattributes wasn't the core problem here.

Jason

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Dawid Weiss-2
> Since the issue ended up being a bad regex in the build task, do you
> still want me to exclude adoc files specifically from gitattributes
> Dawid?  I'll make the change if there's some particular reason why
> adoc files should be exempted here.  But at a glance it seems like
> gitattributes wasn't the core problem here.

Please read what Uwe wrote; the regexp was incorrect (didn't work for
Windows) but those
files were LF only in the repository. So to me gitattributes is/was
the problem as it changes
repository content on checkout/ update for Windows users. This must
*not* happen.
The current "auto" setting should default to "don't touch".

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Jan Høydahl / Cominvent
auto for .adoc files will guard us from a Windows user checking in an .adoc file with CRLF ending in the future, won’t it? But if you leave it as «don’t touch» such a mistake may happen, which was the root cause triggered .gitattributes in the first place? Or do you mean that it now creates issues for Windows users in some way?

Jan

> 27. jan. 2020 kl. 12:49 skrev Dawid Weiss <[hidden email]>:
>
>> Since the issue ended up being a bad regex in the build task, do you
>> still want me to exclude adoc files specifically from gitattributes
>> Dawid?  I'll make the change if there's some particular reason why
>> adoc files should be exempted here.  But at a glance it seems like
>> gitattributes wasn't the core problem here.
>
> Please read what Uwe wrote; the regexp was incorrect (didn't work for
> Windows) but those
> files were LF only in the repository. So to me gitattributes is/was
> the problem as it changes
> repository content on checkout/ update for Windows users. This must
> *not* happen.
> The current "auto" setting should default to "don't touch".
>
> D.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Dawid Weiss-2
> auto for .adoc files will guard us from a Windows user checking in an .adoc file with CRLF ending in the future, won’t it?

The current rule applies to all files not explicitly listed -- I think
this is wrong. You can add adoc explicitly but don't apply it to the
world at large...

> Or do you mean that it now creates issues for Windows users in some way?

It created an issue for me in that I spent two hours looking for a bug
in the wrong place... and the same problems with auto-magical git
conversions happen *over and over* for me in multiple projects -- it's
not the first time.

It is really hard to figure out what's going on when you stumble upon
a problem like this. Also, like I said previously, I don't trust all
git implementations and versions to handle those directives
identically (jgit in particular). I don't mind those rules as long as
they create identical repository on local clone/reset. This means all
hashes of all files should be identical: windows or linux. If they're
not the same it's potentially trouble.

If we're so concerned about CRLF then I'd say add such rules to
precommit check (I can do it, no problem) but those automatic
conversions are/will be a constant trouble.

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Jason Gerlowski
> It created an issue for me in that I spent two hours looking for a bug in the wrong place...

I regret that it made it harder for you to find the issue.  But at the
end of the day the issue wasn't with gitattributes, it was a slip-up
in custom precommit code.

I agree with Jan - "auto" prevents CRLF from being injected into the
files that don't need it, it's a Good Thing.  Now, if gitattributes
breaks your normal workflow in some way - if your diffs start looking
noisy and weird locally, if Eclipse or IntelliJ behave weirdly, if
builds fail, etc. - then absolutely we need to scale back what
gitattributes affects.  But with our custom precommit code fixed, is
that the case?

If you feel strongly enough, I'll scale back what gitattributes
affects.  But as Jan pointed out, this is a concrete benefit to us,
and it doesn't feel right to walk that back just because there might
be some workflow issue out there that we don't know about.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Robert Muir
In reply to this post by Dawid Weiss-2
There does seem to be a bit of a mess here. I noticed lots of ^Ms in the gradle files for example (I'm really not trying to put blame on anyone, just explain what I see)

I don't complain about it, my vim setup detects the file as DOS and then just continues with the trend, but for sure its strange to have some source files with DOS line endings and other ones with UNIX line endings, depending on who created the files. I see it when reviewing my stuff with 'git diff'...

For example:
gradle/testing/defaults-tests.gradle: UNIX line endings
gradle/testing/failed-tests-at-end.gradle: DOS line endings

I also think its confusing to have gitattributes at the root and then also under solr/
can we just have one gitattributes file at the root?

There shouldn't be that many files that require explicit line endings settings (e.g. .cmd files come to mind).

On Mon, Jan 27, 2020 at 7:48 AM Dawid Weiss <[hidden email]> wrote:
> auto for .adoc files will guard us from a Windows user checking in an .adoc file with CRLF ending in the future, won’t it?

The current rule applies to all files not explicitly listed -- I think
this is wrong. You can add adoc explicitly but don't apply it to the
world at large...

> Or do you mean that it now creates issues for Windows users in some way?

It created an issue for me in that I spent two hours looking for a bug
in the wrong place... and the same problems with auto-magical git
conversions happen *over and over* for me in multiple projects -- it's
not the first time.

It is really hard to figure out what's going on when you stumble upon
a problem like this. Also, like I said previously, I don't trust all
git implementations and versions to handle those directives
identically (jgit in particular). I don't mind those rules as long as
they create identical repository on local clone/reset. This means all
hashes of all files should be identical: windows or linux. If they're
not the same it's potentially trouble.

If we're so concerned about CRLF then I'd say add such rules to
precommit check (I can do it, no problem) but those automatic
conversions are/will be a constant trouble.

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Robert Muir
Here is a list of all non-binary files in master with DOS endings:

$ git grep -I -l '^M'
.gitattributes
dev-tools/git/HELP.txt
gradle/help.gradle
gradle/testing/failed-tests-at-end.gradle
gradle/testing/per-project-summary.gradle
gradle/testing/runtime-jvm-support.gradle
gradle/testing/slowest-tests-at-end.gradle
gradle/validation/check-environment.gradle
gradle/validation/forbidden-apis.gradle
gradle/validation/git-status.gradle
gradle/validation/jar-checks.gradle
gradle/validation/precommit.gradle
gradlew.bat
help/dependencies.txt
help/git.txt
help/tests.txt
lucene/licenses/hamcrest-core-LICENSE-BSD.txt
solr/bin/solr.cmd
solr/bin/solr.in.cmd
solr/licenses/hamcrest-core-LICENSE-BSD.txt
solr/server/scripts/cloud-scripts/zkcli.bat
solr/solr-ref-guide/src/fonts/Inconsolata/OFL.txt

On Mon, Jan 27, 2020 at 10:08 AM Robert Muir <[hidden email]> wrote:
There does seem to be a bit of a mess here. I noticed lots of ^Ms in the gradle files for example (I'm really not trying to put blame on anyone, just explain what I see)

I don't complain about it, my vim setup detects the file as DOS and then just continues with the trend, but for sure its strange to have some source files with DOS line endings and other ones with UNIX line endings, depending on who created the files. I see it when reviewing my stuff with 'git diff'...

For example:
gradle/testing/defaults-tests.gradle: UNIX line endings
gradle/testing/failed-tests-at-end.gradle: DOS line endings

I also think its confusing to have gitattributes at the root and then also under solr/
can we just have one gitattributes file at the root?

There shouldn't be that many files that require explicit line endings settings (e.g. .cmd files come to mind).

On Mon, Jan 27, 2020 at 7:48 AM Dawid Weiss <[hidden email]> wrote:
> auto for .adoc files will guard us from a Windows user checking in an .adoc file with CRLF ending in the future, won’t it?

The current rule applies to all files not explicitly listed -- I think
this is wrong. You can add adoc explicitly but don't apply it to the
world at large...

> Or do you mean that it now creates issues for Windows users in some way?

It created an issue for me in that I spent two hours looking for a bug
in the wrong place... and the same problems with auto-magical git
conversions happen *over and over* for me in multiple projects -- it's
not the first time.

It is really hard to figure out what's going on when you stumble upon
a problem like this. Also, like I said previously, I don't trust all
git implementations and versions to handle those directives
identically (jgit in particular). I don't mind those rules as long as
they create identical repository on local clone/reset. This means all
hashes of all files should be identical: windows or linux. If they're
not the same it's potentially trouble.

If we're so concerned about CRLF then I'd say add such rules to
precommit check (I can do it, no problem) but those automatic
conversions are/will be a constant trouble.

D.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Jason Gerlowski
> I also think its confusing to have gitattributes at the root and then also under solr/

Oh, that's new.  There wasn't a root gitattributes when I wrote the
solr/ one.  Must have been added around the same time through the
gradle work?

Anyway, there's no technical reason afaik to keep the files separate.
I put my recent addition under solr/ because I guessed the Lucene code
wasn't nearly as affected by the OS-specific script issue we've hit a
few times in Solr.  I didn't want to stick my nose in and affect the
Lucene folks without a concrete benefit to point to.  But if that's
something Lucene guys would want Robert, feel free to pursue a
combination.

Though it probably makes sense to first see where we land on Dawid's
point about narrowing the files affected.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Dawid Weiss-2
In reply to this post by Jason Gerlowski
> I agree with Jan - "auto" prevents CRLF from being injected into the
> files that don't need it, it's a Good Thing.

First of all, this isn't a personal attack, Jason. It's a personal sore wound
caused by gitattributes that hurts every time I happen to come across it.

> Now, if gitattributes breaks your normal workflow in some way - if your diffs start looking
> noisy and weird locally, if Eclipse or IntelliJ behave weirdly, if
> builds fail, etc. - then absolutely we need to scale back what
> gitattributes affects.

This is exactly what should have happened and didn't. git diff
wouldn't show any
local changes on those files that had altered newlines compared to
repository version
(\r\n from repository's \n). I really like to keep the ability for git diff to
show those differences (you can always run git diff -w if you don't
care about white space).

> If you feel strongly enough, I'll scale back what gitattributes
> affects.  But as Jan pointed out, this is a concrete benefit to us,

Here are the problems I've had in the past if you're curious:

1) any checksum computation, length-dependence or something like this
computed at build-time would compute different values on Windows vs. Linux
leading to different archives being created, different file signatures, etc.

2) sometimes extension collisions do happen, especially on esoteric projects
which store binaries. So a ".cmd" or a ".bat" can be a file that is
not a Windows
batch file... When git converts newlines upon commit not only it
ended up messing up those files locally but also in the repository (if they were
binary).

3) when you work on Windows and have global gitattributes things get
even more weird but I don't want to go into that.

> I noticed lots of ^Ms in the gradle files for example (I'm really not trying to put
> blame on anyone, just explain what I see)

...and they should be cleaned up!

Let me be honest -- I personally remove .gitattributes from any
repository. I've had enough
problems with those conversions to just avoid them anywhere I can, especially
globs. I don't mind if you guys feel like it's something really needed
and helpful (which I
don't quite agree with) but at least let's avoid auto globs for files
not explicitly named.

bq. Must have been added around the same time through the gradle work?

Correct. This particular one holds lfs for versions lock which is
generated and line endings differ
depending on which system you use for updating (sigh). It is explicit
though; not even a file extension:

# Ignore all differences in line endings for the lock file.
versions.lock text eol=lf

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: analytics.adoc validation does not pass

Robert Muir

I don't mind doing the cleanup work once, but I'd like us to prevent new files with inconsistent newlines from being introduced in the future, because I don't want to do it again. I don't care how that is accomplished. It doesn't have to be "auto".

On Mon, Jan 27, 2020 at 10:51 AM Dawid Weiss <[hidden email]> wrote:
> I agree with Jan - "auto" prevents CRLF from being injected into the
> files that don't need it, it's a Good Thing.

First of all, this isn't a personal attack, Jason. It's a personal sore wound
caused by gitattributes that hurts every time I happen to come across it.

> Now, if gitattributes breaks your normal workflow in some way - if your diffs start looking
> noisy and weird locally, if Eclipse or IntelliJ behave weirdly, if
> builds fail, etc. - then absolutely we need to scale back what
> gitattributes affects.

This is exactly what should have happened and didn't. git diff
wouldn't show any
local changes on those files that had altered newlines compared to
repository version
(\r\n from repository's \n). I really like to keep the ability for git diff to
show those differences (you can always run git diff -w if you don't
care about white space).

> If you feel strongly enough, I'll scale back what gitattributes
> affects.  But as Jan pointed out, this is a concrete benefit to us,

Here are the problems I've had in the past if you're curious:

1) any checksum computation, length-dependence or something like this
computed at build-time would compute different values on Windows vs. Linux
leading to different archives being created, different file signatures, etc.

2) sometimes extension collisions do happen, especially on esoteric projects
which store binaries. So a ".cmd" or a ".bat" can be a file that is
not a Windows
batch file... When git converts newlines upon commit not only it
ended up messing up those files locally but also in the repository (if they were
binary).

3) when you work on Windows and have global gitattributes things get
even more weird but I don't want to go into that.

> I noticed lots of ^Ms in the gradle files for example (I'm really not trying to put
> blame on anyone, just explain what I see)

...and they should be cleaned up!

Let me be honest -- I personally remove .gitattributes from any
repository. I've had enough
problems with those conversions to just avoid them anywhere I can, especially
globs. I don't mind if you guys feel like it's something really needed
and helpful (which I
don't quite agree with) but at least let's avoid auto globs for files
not explicitly named.

bq. Must have been added around the same time through the gradle work?

Correct. This particular one holds lfs for versions lock which is
generated and line endings differ
depending on which system you use for updating (sigh). It is explicit
though; not even a file extension:

# Ignore all differences in line endings for the lock file.
versions.lock text eol=lf

Dawid

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]