llvm dev - Nov 2018 - Ninja build (on Windows anyway) may be doing redundant work

Do you still see this if you use lld-link for linking?

The "corrupt obj file" is something we saw on chrome's bots every
now and
then before we switched to lld.

On Mon, Nov 19, 2018 at 5:27 PM Zachary Turner <zturner at google.com>
wrote:
> +Nico Weber <thakis at google.com>
>
> On Mon, Nov 19, 2018 at 12:25 PM via llvm-dev <llvm-dev at
lists.llvm.org>
> wrote:
>
>> This afternoon I did a clean build of llvm using ninja and MSVC 2015 on
>> Windows 10. I saw something curious and wondered if any of the ninja or
>> CMake experts out there can explain/fix it.
>>
>> [2663/3121] Linking CXX shared library bin\LTO.dll
>>    Creating library lib\LTO.dll and object lib\LTO.exp
>>    Creating library lib\LTO.dll and object lib\LTO.exp
>> [3120/3121] Running the LLVM regression tests
>> [[[ etc ]]]
>>
>> I'm curious about the "creating library" message popping
out twice.
>> Sometimes ninja builds will die on me, complaining about a corrupted
>> object file, usually related to LTO. Rerunning ninja usually completes
>> successfully.  So I wonder if there's some mishap in the
dependencies
>> somewhere that actually causes the build to try to link the DLL twice,
>> which of course might corrupt a file if the links are overlapping in
>> time.
>>
>> Thought I'd throw this out there in case someone wants to take a
look.
>> --paulr
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181119/db2d96c3/attachment.html>

I'm more concerned about seeing the message come out twice, which implies
Ninja is executing a rule twice when it doesn't need to.
--paulr

From: Nico Weber [mailto:thakis at chromium.org]
Sent: Monday, November 19, 2018 6:09 PM
To: Zachary Turner
Cc: Robinson, Paul; llvm-dev
Subject: Re: [llvm-dev] Ninja build (on Windows anyway) may be doing redundant
work

Do you still see this if you use lld-link for linking?

The "corrupt obj file" is something we saw on chrome's bots every
now and then before we switched to lld.

On Mon, Nov 19, 2018 at 5:27 PM Zachary Turner <zturner at
google.com<mailto:zturner at google.com>> wrote:
+Nico Weber<mailto:thakis at google.com>
On Mon, Nov 19, 2018 at 12:25 PM via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
This afternoon I did a clean build of llvm using ninja and MSVC 2015 on
Windows 10. I saw something curious and wondered if any of the ninja or
CMake experts out there can explain/fix it.

[2663/3121] Linking CXX shared library bin\LTO.dll
   Creating library lib\LTO.dll and object lib\LTO.exp
   Creating library lib\LTO.dll and object lib\LTO.exp
[3120/3121] Running the LLVM regression tests
[[[ etc ]]]

I'm curious about the "creating library" message popping out
twice.
Sometimes ninja builds will die on me, complaining about a corrupted
object file, usually related to LTO. Rerunning ninja usually completes
successfully.  So I wonder if there's some mishap in the dependencies
somewhere that actually causes the build to try to link the DLL twice,
which of course might corrupt a file if the links are overlapping in
time.

Thought I'd throw this out there in case someone wants to take a look.
--paulr

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181119/c8d8bacb/attachment.html>

Since there's no "[2663/3121] " line between the two messages, the
two
lines are from the same link.exe invocation. I don't know why link.exe
thinks it needs to print this line twice, ninja doesn't have anything to do
with it.

On Mon, Nov 19, 2018 at 6:57 PM <paul.robinson at sony.com> wrote:
> I'm more concerned about seeing the message come out twice, which
implies
> Ninja is executing a rule twice when it doesn't need to.
>
> --paulr
>
>
>
> *From:* Nico Weber [mailto:thakis at chromium.org]
> *Sent:* Monday, November 19, 2018 6:09 PM
> *To:* Zachary Turner
> *Cc:* Robinson, Paul; llvm-dev
> *Subject:* Re: [llvm-dev] Ninja build (on Windows anyway) may be doing
> redundant work
>
>
>
> Do you still see this if you use lld-link for linking?
>
>
>
> The "corrupt obj file" is something we saw on chrome's bots
every now and
> then before we switched to lld.
>
>
>
> On Mon, Nov 19, 2018 at 5:27 PM Zachary Turner <zturner at
google.com> wrote:
>
> +Nico Weber <thakis at google.com>
>
> On Mon, Nov 19, 2018 at 12:25 PM via llvm-dev <llvm-dev at
lists.llvm.org>
> wrote:
>
> This afternoon I did a clean build of llvm using ninja and MSVC 2015 on
> Windows 10. I saw something curious and wondered if any of the ninja or
> CMake experts out there can explain/fix it.
>
> [2663/3121] Linking CXX shared library bin\LTO.dll
>    Creating library lib\LTO.dll and object lib\LTO.exp
>    Creating library lib\LTO.dll and object lib\LTO.exp
> [3120/3121] Running the LLVM regression tests
> [[[ etc ]]]
>
> I'm curious about the "creating library" message popping out
twice.
> Sometimes ninja builds will die on me, complaining about a corrupted
> object file, usually related to LTO. Rerunning ninja usually completes
> successfully.  So I wonder if there's some mishap in the dependencies
> somewhere that actually causes the build to try to link the DLL twice,
> which of course might corrupt a file if the links are overlapping in
> time.
>
> Thought I'd throw this out there in case someone wants to take a look.
> --paulr
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20181119/7c54e895/attachment-0001.html>