llvm dev - Jan 2020 - Git autocrlf for Windows - why does the Getting Started guide say to use false?

Right, my understanding is that files that specifically need CRLF should say so
via .gitattributes somewhere.
autocrlf=false ought to work if your Windows editor doesn’t (or can be persuaded
not to) introduce CRLFs.  I’ve observed that the Visual Studio editor will
detect the line-ending style of the file you’re editing, and imitate it.  This
works great until you create a new file. 😝 So I’ve learned to “create” a file by
copying an existing file and then editing it.
I’m inclined to think the llvm.org docs should say to use “input” on Windows,
we’ve been happy with that in the Sony repo as James mentioned.
But I’d also like to hear from a Windows user from outside Sony first.
--paulr

From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Greg
Bedwell via llvm-dev
Sent: Thursday, January 16, 2020 12:27 PM
To: jh7370.2008 at my.bristol.ac.uk
Cc: llvm-dev <llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Git autocrlf for Windows - why does the Getting Started
guide say to use false?

The previous discussion about it was here:  https://reviews.llvm.org/D48494

Personally I'd rather see if we could solve it with some clever use of
.gitattributes rather than make it a user-setting but sadly I don't have
time right now to commit to trying out a solution.
I notice that there is one case already of using .gitattributes to make sure
autocrlf doesn't change line endings, FWIW:
https://github.com/llvm/llvm-project/blob/master/llvm/.gitattributes#L19



On Thu, 16 Jan 2020 at 15:19, James Henderson via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
Hi all,

TL;DR - should we recommend core.autocrlf=input instead of core.autocrlf=false
on Windows?

I recently switched from doing pre-commit builds and tests from my Ubuntu VM, to
my Windows machine where I do all my development. I was aware that line endings
is an issue, so made sure to set core.autocrlf=false as directed in the Getting
Started guide (see
https://llvm.org/docs/GettingStarted.html#getting-the-source-code-and-building-llvm
and https://llvm.org/docs/GettingStarted.html#checkout-llvm-from-git which
explicitly say to use "false"). This seemed a little odd to me, as
that would mean that if I made any edits, I could potentially end up with
Windows line endings, by my understanding, but I assumed that I had
misunderstood something, so just went along with it.

However, at some point in one of my commits (at least I assume it was one of
mine, since I was working on the file), I somehow managed to change an existing
file to using Windows line endings, which someone else fixed for me (see commit
aca3e70d2bc0dd89b7d486c2a8eac70d8a89e790).

Having double-checked against my settings on my downstream clone, I noticed that
I had core.autocrlf set to input there, and as far as I know, I've never had
any line-ending problems with working with LLVM in that repo. My understanding
is "input" should avoid problems such as mine, i.e. by ensuring the
checked-in files have LF line endings, whilst everybody, regardless of system
will get native line endings in their actual clones. I am aware that there are
some files that deliberately use Windows line endings - I'm guessing that
these are not widespread and that they don't need regularly modifying, so
presumably those users who need to edit those files can change their setting as
needed.

I'm happy to update the docs, but I want to make sure that any update I make
is correct!

James
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200116/3d8e3cef/attachment.html>

I thought the handful of test files that require specific line endings were
marked as "binary" so that git won't treat them as text.

I agree that, in theory, you don't want your source control to change your
source code, and that's a complaint I've had about older source control
systems that replace tokens like `$Version$` upon check-in.  That's not the
code you tested.

I've been getting by with `false` on Windows.  You can tell Visual Studio
(at least in 2017+) to warn you if you load a file with inconsistent line
endings.  I believe the VS 2019 editor has options to select which type of
line endings you want be default.  It will also show you the files line
ending type in the status bar.

On Thu, Jan 16, 2020 at 9:36 AM Robinson, Paul via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Right, my understanding is that files that specifically need CRLF should
> say so via .gitattributes somewhere.
>
> autocrlf=false ought to work if your Windows editor doesn’t (or can be
> persuaded not to) introduce CRLFs.  I’ve observed that the Visual Studio
> editor will detect the line-ending style of the file you’re editing, and
> imitate it.  This works great until you create a new file. 😝 So I’ve
> learned to “create” a file by copying an existing file and then editing it.
>
> I’m inclined to think the llvm.org docs should say to use “input” on
> Windows, we’ve been happy with that in the Sony repo as James mentioned.
>
> But I’d also like to hear from a Windows user from outside Sony first.
>
> --paulr
>
>
>
> *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> *On Behalf Of
*Greg
> Bedwell via llvm-dev
> *Sent:* Thursday, January 16, 2020 12:27 PM
> *To:* jh7370.2008 at my.bristol.ac.uk
> *Cc:* llvm-dev <llvm-dev at lists.llvm.org>
> *Subject:* Re: [llvm-dev] Git autocrlf for Windows - why does the Getting
> Started guide say to use false?
>
>
>
> The previous discussion about it was here:
> https://reviews.llvm.org/D48494
>
>
>
> Personally I'd rather see if we could solve it with some clever use of
> .gitattributes rather than make it a user-setting but sadly I don't
have
> time right now to commit to trying out a solution.
>
> I notice that there is one case already of using .gitattributes to make
> sure autocrlf doesn't change line endings, FWIW:
>
> https://github.com/llvm/llvm-project/blob/master/llvm/.gitattributes#L19
>
>
>
>
>
>
>
> On Thu, 16 Jan 2020 at 15:19, James Henderson via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Hi all,
>
>
>
> TL;DR - should we recommend core.autocrlf=input instead of
> core.autocrlf=false on Windows?
>
>
>
> I recently switched from doing pre-commit builds and tests from my Ubuntu
> VM, to my Windows machine where I do all my development. I was aware that
> line endings is an issue, so made sure to set core.autocrlf=false as
> directed in the Getting Started guide (see
>
https://llvm.org/docs/GettingStarted.html#getting-the-source-code-and-building-llvm
> and https://llvm.org/docs/GettingStarted.html#checkout-llvm-from-git
> which explicitly say to use "false"). This seemed a little odd to
me, as
> that would mean that if I made any edits, I could potentially end up with
> Windows line endings, by my understanding, but I assumed that I had
> misunderstood something, so just went along with it.
>
>
>
> However, at some point in one of my commits (at least I assume it was one
> of mine, since I was working on the file), I somehow managed to change an
> existing file to using Windows line endings, which someone else fixed for
> me (see commit aca3e70d2bc0dd89b7d486c2a8eac70d8a89e790).
>
>
>
> Having double-checked against my settings on my downstream clone, I
> noticed that I had core.autocrlf set to input there, and as far as I know,
> I've never had any line-ending problems with working with LLVM in that
> repo. My understanding is "input" should avoid problems such as
mine, i.e.
> by ensuring the checked-in files have LF line endings, whilst everybody,
> regardless of system will get native line endings in their actual clones. I
> am aware that there are some files that deliberately use Windows line
> endings - I'm guessing that these are not widespread and that they
don't
> need regularly modifying, so presumably those users who need to edit those
> files can change their setting as needed.
>
>
>
> I'm happy to update the docs, but I want to make sure that any update I
> make is correct!
>
>
>
> James
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200116/a6131ae0/attachment.html>

On Thu, Jan 16, 2020 at 9:36 AM Robinson, Paul via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I’m inclined to think the llvm.org docs should say to use “input” on
> Windows, we’ve been happy with that in the Sony repo as James mentioned.
>
> But I’d also like to hear from a Windows user from outside Sony first.
>
I also think we should recommend `input`, if it does what I think it does:
convert from CRLF to LF, and never back.

This can break a few cases:
- adding binary files for a test that you forgot to mark binary
- adding CRLF text files for a test that you forgot to mark binary

These seem like more common exceptions than adding a new source file, which
are often mistakenly committed with CRLF. Any screwups, reverts, and bot
breakages from that seem less costly than accidentally adding line ending
changes, which live forever as a tax on `git annotate`.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200116/d0087352/attachment.html>

I use almost exclusively 4 text editors (for some meaning of "editor")
in
my usual workflow: Sublime Text (for tests), and Visual Studio 2017 (for
basically everything else). The third one is Beyond Compare which I use for
diffs, and often make minor edits in there prior to commit. Finally, I use
the p4merge tool for resolving merge conflicts. On further investigation,
it looks like the last of these is the culprit for messing up my line
endings (when I save the merge resolution, it saves with Windows line
endings), but it has an explicit setting I can change to use LF only
endings, so I have fixed this. Hopefully this should prevent me messing up
existing files from now on. It doesn't solve the new file issue though, nor
does it solve the issue for new people to the project, since it is an
obvious counter example to the "all reasonable text editors do the right
thing" comment.

I do use dos2unix, but only for generating new patches for uploading to
Phabricator. I did experiment with it once when I thought something looked
fishy with the line endings in the code I was looking at, but it seemed to
touch my whole file. Probably it was in a bad state already though, and I
just hadn't realised.

I do think fixing .gitattributes sounds like the best solution, and I
certainly would prefer input be the specified preference (I might well
change my settings to do that, to solve things for my own workflow), as
this solves the new file issue. I note also that git warns if your line
endings would be changed automatically when you add a file. I'd propose a
note to add to the documentation as a possible compromise, or possibly a
pair of options, listing the caveats of each, but I can't think of a
concise coherent way of saying it. Suggestions welcome!

On Thu, 16 Jan 2020 at 18:38, Reid Kleckner <rnk at google.com> wrote:
> On Thu, Jan 16, 2020 at 9:36 AM Robinson, Paul via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> I’m inclined to think the llvm.org docs should say to use “input” on
>> Windows, we’ve been happy with that in the Sony repo as James
mentioned.
>>
>> But I’d also like to hear from a Windows user from outside Sony first.
>>
>
> I also think we should recommend `input`, if it does what I think it does:
> convert from CRLF to LF, and never back.
>
> This can break a few cases:
> - adding binary files for a test that you forgot to mark binary
> - adding CRLF text files for a test that you forgot to mark binary
>
> These seem like more common exceptions than adding a new source file,
> which are often mistakenly committed with CRLF. Any screwups, reverts, and
> bot breakages from that seem less costly than accidentally adding line
> ending changes, which live forever as a tax on `git annotate`.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200117/24c584d1/attachment.html>