llvm dev - Jan 2019 - MSBuild incremental builds are broken with LLVM 6 (and beyond)

Hi,

Starting with LLVM 6, MSBuild incremental builds stopped working. I've
tracked this down to a CL that modified how file renaming was done on
Windows. It appears that FileTracker does not recognize renaming a file
with SetFileInformationByHandle.

https://github.com/llvm-mirror/llvm/commit/1b6a51a1425cc0180359ecd64733edb965f65d7c#diff-70db51c38d748dc9debf5f309643fcd8

You can repro this fairly easily by:
1) Downloading the prebuilt binaries of LLVM 6 or 7 for Windows.
2) Creating a new C++ project (I just chose the default console
application) in VS 2017.
3) Mess with the project settings to disable all the features clang doesn't
support (Conformance mode, Just my code, etc...)
4) Build

Every time you build it will rebuild everything.

I'm curious if this is a known issue? (I couldn't find any open bugs
relating to this)
Is MSBuild incremental builds something that is officially supported? Or
did it just work in the past by chance?

Thanks,
Scott
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190125/35b73cf8/attachment.html>

I had no idea they were broken, that seems like a major regression. Come to
think of it, I think someone may have reported this behavior, but it was
never root caused, so thanks for that.

However, we spent a lot of time coming up with this particular code pattern
to:
1. (mostly) atomically write the whole output file or nothing at all on
failure
2. not leak temporary files when the compiler crashes

Before our use of SetInformationByFileHandle we leaked temporary object
files in build directories like crazy. All you had to do was basically
interrupt a build and clang would leave behind files. The linker suffered
from similar problems.

So, I would be very careful about going back to our old code patterns. I
wonder if Microsoft would accept this as a bug in FileTracker.dll instead.

On Fri, Jan 25, 2019 at 7:00 AM Scott Funkenhauser via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hi,
>
> Starting with LLVM 6, MSBuild incremental builds stopped working. I've
> tracked this down to a CL that modified how file renaming was done on
> Windows. It appears that FileTracker does not recognize renaming a file
> with SetFileInformationByHandle.
>
>
>
https://github.com/llvm-mirror/llvm/commit/1b6a51a1425cc0180359ecd64733edb965f65d7c#diff-70db51c38d748dc9debf5f309643fcd8
>
> You can repro this fairly easily by:
> 1) Downloading the prebuilt binaries of LLVM 6 or 7 for Windows.
> 2) Creating a new C++ project (I just chose the default console
> application) in VS 2017.
> 3) Mess with the project settings to disable all the features clang
> doesn't support (Conformance mode, Just my code, etc...)
> 4) Build
>
> Every time you build it will rebuild everything.
>
> I'm curious if this is a known issue? (I couldn't find any open
bugs
> relating to this)
> Is MSBuild incremental builds something that is officially supported? Or
> did it just work in the past by chance?
>
> Thanks,
> Scott
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190125/a8bfa81f/attachment-0001.html>

You mention that you disabled features clang doesn’t support, so I take
this to mean you’re using the LLVM VS integration.

If so, what version of the VS integration is being used? Is it the version
that’s on the Marketplace and installs from a vsix, or the version that you
run a batch file to install?
On Fri, Jan 25, 2019 at 10:23 AM Reid Kleckner via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I had no idea they were broken, that seems like a major regression. Come
> to think of it, I think someone may have reported this behavior, but it was
> never root caused, so thanks for that.
>
> However, we spent a lot of time coming up with this particular code
> pattern to:
> 1. (mostly) atomically write the whole output file or nothing at all on
> failure
> 2. not leak temporary files when the compiler crashes
>
> Before our use of SetInformationByFileHandle we leaked temporary object
> files in build directories like crazy. All you had to do was basically
> interrupt a build and clang would leave behind files. The linker suffered
> from similar problems.
>
> So, I would be very careful about going back to our old code patterns. I
> wonder if Microsoft would accept this as a bug in FileTracker.dll instead.
>
> On Fri, Jan 25, 2019 at 7:00 AM Scott Funkenhauser via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hi,
>>
>> Starting with LLVM 6, MSBuild incremental builds stopped working.
I've
>> tracked this down to a CL that modified how file renaming was done on
>> Windows. It appears that FileTracker does not recognize renaming a file
>> with SetFileInformationByHandle.
>>
>>
>>
https://github.com/llvm-mirror/llvm/commit/1b6a51a1425cc0180359ecd64733edb965f65d7c#diff-70db51c38d748dc9debf5f309643fcd8
>>
>> You can repro this fairly easily by:
>> 1) Downloading the prebuilt binaries of LLVM 6 or 7 for Windows.
>> 2) Creating a new C++ project (I just chose the default console
>> application) in VS 2017.
>> 3) Mess with the project settings to disable all the features clang
>> doesn't support (Conformance mode, Just my code, etc...)
>> 4) Build
>>
>> Every time you build it will rebuild everything.
>>
>> I'm curious if this is a known issue? (I couldn't find any open
bugs
>> relating to this)
>> Is MSBuild incremental builds something that is officially supported?
Or
>> did it just work in the past by chance?
>>
>> Thanks,
>> Scott
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190125/bc338259/attachment.html>