gitattributes and line-endings

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

gitattributes and line-endings

Jason Gerlowski
Hi all,

Wanted to sent out a notice about a change I plan on making this week
regarding how line endings are maintained in some of our OS-specific
files (bin/solr, zkcli, etc.).

Previously it's been on committers/contributors individually to put
the correct line endings in their files.  This has bitten us a few
times recently.  solr.cmd often ends up missing the CRLF EOLs that it
needs, which breaks the code surrounding those edits. See SOLR-13977
for a recent example.

To remedy this, I plan on adding a .gitattributes file to normalize
EOL's across Solr code.  File editing is unchanged.  But on certain
git commands (add, commit, checkout), git will rewrite the line
endings of changed files to match the desired EOL for that file.  For
most files, this is LF.  For Windows specific files, this is CRLF.
See https://github.com/apache/lucene-solr/pull/1163/files#diff-c781eab6689a2956b8e2d082fdecbe38
for the specific rules.

This change should be entirely invisible to developers, with two caveats:

1. Of necessity, the gitattributes PR also corrects a few files that
have the wrong line endings currently.  Developers applying old
patches that touch one of these files will encounter merge conflicts
due to the EOL changes.

2. If developers have incorrect line endings in files unique to a
feature branch, and they merge master into that feature branch, they
may see unexpected changes in git-status as git normalizes the EOL's
in those files.

Given how rare incorrect EOLs are (outside of solr.cmd), I expect this
change will be invisible to 99% of developers.  If anyone has any
questions or concerns, let me know or comment on the PR (#1163)
directly.  I plan on merging the change tomorrow or the day after.

Best,

Jason

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: gitattributes and line-endings

Erick Erickson
Jason:

I’ve been poking around some of the regenerate code. I should say
up-front that I have no idea whether this is relevant or not, but…

Here’s a command to scare you:

"find . -name build.xml | xargs grep -i fixcrlf”

To be frank, I’m not totally sure what the intent for those is. From
what I can tell they operate on java files that are the output from
things like jflex and just modify line endings for the Java output,
presumably the output may have windows-style line endings, but
that’s a guess. Originally I had a vague fear that it operated on
binary output, but this made me look more deeply and I don’t see that.

So since they appear to be working on generated Java code, and
I assume this is happening before the git operations you
outlined, I expect it to be totally invisible.

This is mostly FYI, I happened to see the “fixcrlf” in the build files
and thought I’d pass it along. I don’t think this should hold up the
commit at all.

Best,
Erick

> On Jan 14, 2020, at 10:18 AM, Jason Gerlowski <[hidden email]> wrote:
>
> Hi all,
>
> Wanted to sent out a notice about a change I plan on making this week
> regarding how line endings are maintained in some of our OS-specific
> files (bin/solr, zkcli, etc.).
>
> Previously it's been on committers/contributors individually to put
> the correct line endings in their files.  This has bitten us a few
> times recently.  solr.cmd often ends up missing the CRLF EOLs that it
> needs, which breaks the code surrounding those edits. See SOLR-13977
> for a recent example.
>
> To remedy this, I plan on adding a .gitattributes file to normalize
> EOL's across Solr code.  File editing is unchanged.  But on certain
> git commands (add, commit, checkout), git will rewrite the line
> endings of changed files to match the desired EOL for that file.  For
> most files, this is LF.  For Windows specific files, this is CRLF.
> See https://github.com/apache/lucene-solr/pull/1163/files#diff-c781eab6689a2956b8e2d082fdecbe38
> for the specific rules.
>
> This change should be entirely invisible to developers, with two caveats:
>
> 1. Of necessity, the gitattributes PR also corrects a few files that
> have the wrong line endings currently.  Developers applying old
> patches that touch one of these files will encounter merge conflicts
> due to the EOL changes.
>
> 2. If developers have incorrect line endings in files unique to a
> feature branch, and they merge master into that feature branch, they
> may see unexpected changes in git-status as git normalizes the EOL's
> in those files.
>
> Given how rare incorrect EOLs are (outside of solr.cmd), I expect this
> change will be invisible to 99% of developers.  If anyone has any
> questions or concerns, let me know or comment on the PR (#1163)
> directly.  I plan on merging the change tomorrow or the day after.
>
> Best,
>
> Jason
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: gitattributes and line-endings

Jason Gerlowski
Thanks for the heads up Erick.

I poked around a little bit there and agree that this shouldn't affect
the gitattributes change.  I'm also curious why it was needed.  Maybe
Yonik or someone else involved with putting it in could clarify for
curiosity's sake.

Jason

On Tue, Jan 14, 2020 at 11:22 AM Erick Erickson <[hidden email]> wrote:

>
> Jason:
>
> I’ve been poking around some of the regenerate code. I should say
> up-front that I have no idea whether this is relevant or not, but…
>
> Here’s a command to scare you:
>
> "find . -name build.xml | xargs grep -i fixcrlf”
>
> To be frank, I’m not totally sure what the intent for those is. From
> what I can tell they operate on java files that are the output from
> things like jflex and just modify line endings for the Java output,
> presumably the output may have windows-style line endings, but
> that’s a guess. Originally I had a vague fear that it operated on
> binary output, but this made me look more deeply and I don’t see that.
>
> So since they appear to be working on generated Java code, and
> I assume this is happening before the git operations you
> outlined, I expect it to be totally invisible.
>
> This is mostly FYI, I happened to see the “fixcrlf” in the build files
> and thought I’d pass it along. I don’t think this should hold up the
> commit at all.
>
> Best,
> Erick
>
> > On Jan 14, 2020, at 10:18 AM, Jason Gerlowski <[hidden email]> wrote:
> >
> > Hi all,
> >
> > Wanted to sent out a notice about a change I plan on making this week
> > regarding how line endings are maintained in some of our OS-specific
> > files (bin/solr, zkcli, etc.).
> >
> > Previously it's been on committers/contributors individually to put
> > the correct line endings in their files.  This has bitten us a few
> > times recently.  solr.cmd often ends up missing the CRLF EOLs that it
> > needs, which breaks the code surrounding those edits. See SOLR-13977
> > for a recent example.
> >
> > To remedy this, I plan on adding a .gitattributes file to normalize
> > EOL's across Solr code.  File editing is unchanged.  But on certain
> > git commands (add, commit, checkout), git will rewrite the line
> > endings of changed files to match the desired EOL for that file.  For
> > most files, this is LF.  For Windows specific files, this is CRLF.
> > See https://github.com/apache/lucene-solr/pull/1163/files#diff-c781eab6689a2956b8e2d082fdecbe38
> > for the specific rules.
> >
> > This change should be entirely invisible to developers, with two caveats:
> >
> > 1. Of necessity, the gitattributes PR also corrects a few files that
> > have the wrong line endings currently.  Developers applying old
> > patches that touch one of these files will encounter merge conflicts
> > due to the EOL changes.
> >
> > 2. If developers have incorrect line endings in files unique to a
> > feature branch, and they merge master into that feature branch, they
> > may see unexpected changes in git-status as git normalizes the EOL's
> > in those files.
> >
> > Given how rare incorrect EOLs are (outside of solr.cmd), I expect this
> > change will be invisible to 99% of developers.  If anyone has any
> > questions or concerns, let me know or comment on the PR (#1163)
> > directly.  I plan on merging the change tomorrow or the day after.
> >
> > Best,
> >
> > Jason
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]