Martin J. O'Riordan via llvm-dev
2018-May-16 21:07 UTC
[llvm-dev] Windows build strangeness
With VS2013 I found that editing a CMakeLists.txt file caused CMake to be re-run successfully and the build to also continue successfully, but since I switched to VS2015 the CMake re-run occurs - apparently successfully, but more often than not the build failed afterwards from either the IDE and from MSBuild. Since I seldom change the CMakeLists.txt files, I simply do a clean CMake configuration anytime I do change them. I also use the "Build" rather than "Rebuild" for incremental development. MartinO -----Original Message----- From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of Michael Kruse via llvm-dev Sent: 16 May 2018 21:26 To: Reid Kleckner <rnk at google.com> Cc: llvm-dev <llvm-dev at lists.llvm.org>; Valentin Churavy <v.churavy at gmail.com> Subject: Re: [llvm-dev] Windows build strangeness msbuild is is able to re-run cmake if a CMakeLists.txt changes. CMake adds a special project "ZERO_CHECK" that does this. However, I am not sure it runs when invoking on the individual projects instead of the solution. Try the cmake --build command, which should output the following:> cmake --build . --target optCMake is re-running because C:/Users/Meinersbur/build/llvm/vc15_debug_x86/tools/polly/CMakeFiles/generate.stamp is out-of-date. [...] Michael 2018-05-16 15:08 GMT-05:00 Reid Kleckner via llvm-dev <llvm-dev at lists.llvm.org>:> I think MSBuild isn't capable of re-running cmake and then reloading > the project files when CMakeLists.txt changes. It re-runs cmake, but > then continues the build with the stale projects. That probably > explains the "PipSqueek.cxx doesn't exist" errors. As for the link > error, it could also be caused by things like a file rename not getting picked up by MSBuild. > > The fix is probably to always run cmake before MSBuild in the nightly job. > This isn't necessary if you are using ninja or make, if those are options. > > On Wed, May 16, 2018 at 12:59 PM via llvm-dev > <llvm-dev at lists.llvm.org> > wrote: >> >> Here are a couple of representative errors. C:\Dev\upstream\gitmono >> is where I keep my clone. >> >> >> >> "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild >> target) >> (1) -> >> >> >> "C:\Dev\upstream\gitmono\wbuild\unittests\Support\DynamicLibrary\SecondLib.vcxproj" >> (default target) (170:2) -> >> >> c1xx : fatal error C1083: Cannot open source file: >> 'C:\Dev\upstream\gitmono\llvm-project\llvm\unittests\Suppor >> >> t\DynamicLibrary\PipSqueak.cxx': No such file or directory >> [C:\Dev\upstream\gitmono\wbuild\unittests\Support\Dyn >> >> amicLibrary\SecondLib.vcxproj] >> >> >> >> "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild >> target) >> (1) -> >> >> "C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj" >> (default target) (339:2) -> >> >> (Link target) -> >> >> Backend.obj : error LNK2019: unresolved external symbol "public: >> virtual bool __cdecl mca::FetchStage::isReady >> >> (void)const " (?isReady at FetchStage@mca@@UEBA_NXZ) referenced >> in function "public: void __cdecl mca::Backend::run >> >> (void)" (?run at Backend@mca@@QEAAXXZ) >> [C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj] >> >> >> >> >> >> >> >> From: Valentin Churavy [mailto:v.churavy at gmail.com] >> Sent: Wednesday, May 16, 2018 1:54 PM >> To: Robinson, Paul >> Cc: llvm-dev at lists.llvm.org >> Subject: Re: [llvm-dev] Windows build strangeness >> >> >> >> What kind of missing symbols are you getting? >> >> I had to work around dependencies for a Mingw32 build. See >> https://reviews.llvm.org/D44650 >> >> On Wed, May 16, 2018, 13:13 via llvm-dev <llvm-dev at lists.llvm.org> wrote: >> >> I have the git monorepo, and Visual Studio 2015. I am finding that >> running a build from the command line with msbuild (as a nightly job) >> invariably fails on the first try, and succeeds on a retry. >> >> The first msbuild command looks like this: >> >> msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6 >> /t:Rebuild >> >> This appears to compile everything okay, but invariably fails with >> some set of link-time errors, reporting missing symbols. Then I run >> msbuild again, but without the /t:Rebuild (thus it's a dirty build >> the second time) and this succeeds. >> >> Is there some inadequacy in the CMake dependencies somewhere, so that >> some links are trying to happen before the libraries are completely >> finished? Or just an msbuild issue? >> >> I made my nightly script check for errors and run the second round, >> because it has been happening so consistently, but a proper build >> system shouldn't require such a workaround. >> >> For reference, I generated the sln/project files like this: >> >> cmake -G "Visual Studio 14 Win64" -Thost=x64 ^ >> -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^ >> -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^ >> ..\llvm-project\llvm >> >> (Yes I know the optimized tablegen comes from a separate tree, that's >> built separately and always works. It's just the main build that >> fails.) >> >> Thanks, >> --paulr >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >_______________________________________________ LLVM Developers mailing list llvm-dev at lists.llvm.org http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>From my own experience this is what I think happens when building the wholesolution through Visual Studio's UI. This also happens for building individual projects. I assume something similar happens when building via the command-line, but I rarely do that, so I can't be certain. 1) Visual Studio/MSBuild (I don't know which, but probably MSBuild) determines the dependency graph of the projects that are to be built. 2) The "ZeroCheck" project runs and cmake is rerun if any of the CMakeLists.txt files have been updated. 3) The build continues for each of the projects, using the dependency graph from 1). If the projects from 1) were updated due to 2) then the updated projects will be built in the updated way (e.g. new files will be included in that build). 4) Since any new projects created by re-running cmake in 2) were not in the dependency graph in 1), projects that now rely on them will likely get link errors. There are two solutions I've used in my development: 1) always run cmake explicitly after doing an svn/git fetch before building or 2) build twice. The second build is usually a lot faster, so it doesn't really matter. James On 16 May 2018 at 22:07, Martin J. O'Riordan via llvm-dev < llvm-dev at lists.llvm.org> wrote:> With VS2013 I found that editing a CMakeLists.txt file caused CMake to be > re-run successfully and the build to also continue successfully, but since > I switched to VS2015 the CMake re-run occurs - apparently successfully, but > more often than not the build failed afterwards from either the IDE and > from MSBuild. Since I seldom change the CMakeLists.txt files, I simply do > a clean CMake configuration anytime I do change them. I also use the > "Build" rather than "Rebuild" for incremental development. > > MartinO > > -----Original Message----- > From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of > Michael Kruse via llvm-dev > Sent: 16 May 2018 21:26 > To: Reid Kleckner <rnk at google.com> > Cc: llvm-dev <llvm-dev at lists.llvm.org>; Valentin Churavy < > v.churavy at gmail.com> > Subject: Re: [llvm-dev] Windows build strangeness > > msbuild is is able to re-run cmake if a CMakeLists.txt changes. CMake adds > a special project "ZERO_CHECK" that does this. However, I am not sure it > runs when invoking on the individual projects instead of the solution. > > Try the cmake --build command, which should output the following: > > > cmake --build . --target opt > CMake is re-running because > C:/Users/Meinersbur/build/llvm/vc15_debug_x86/tools/ > polly/CMakeFiles/generate.stamp > is out-of-date. > [...] > > Michael > > > > 2018-05-16 15:08 GMT-05:00 Reid Kleckner via llvm-dev < > llvm-dev at lists.llvm.org>: > > I think MSBuild isn't capable of re-running cmake and then reloading > > the project files when CMakeLists.txt changes. It re-runs cmake, but > > then continues the build with the stale projects. That probably > > explains the "PipSqueek.cxx doesn't exist" errors. As for the link > > error, it could also be caused by things like a file rename not getting > picked up by MSBuild. > > > > The fix is probably to always run cmake before MSBuild in the nightly > job. > > This isn't necessary if you are using ninja or make, if those are > options. > > > > On Wed, May 16, 2018 at 12:59 PM via llvm-dev > > <llvm-dev at lists.llvm.org> > > wrote: > >> > >> Here are a couple of representative errors. C:\Dev\upstream\gitmono > >> is where I keep my clone. > >> > >> > >> > >> "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild > >> target) > >> (1) -> > >> > >> > >> "C:\Dev\upstream\gitmono\wbuild\unittests\Support\ > DynamicLibrary\SecondLib.vcxproj" > >> (default target) (170:2) -> > >> > >> c1xx : fatal error C1083: Cannot open source file: > >> 'C:\Dev\upstream\gitmono\llvm-project\llvm\unittests\Suppor > >> > >> t\DynamicLibrary\PipSqueak.cxx': No such file or directory > >> [C:\Dev\upstream\gitmono\wbuild\unittests\Support\Dyn > >> > >> amicLibrary\SecondLib.vcxproj] > >> > >> > >> > >> "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj" (Rebuild > >> target) > >> (1) -> > >> > >> "C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj" > >> (default target) (339:2) -> > >> > >> (Link target) -> > >> > >> Backend.obj : error LNK2019: unresolved external symbol > "public: > >> virtual bool __cdecl mca::FetchStage::isReady > >> > >> (void)const " (?isReady at FetchStage@mca@@UEBA_NXZ) referenced > >> in function "public: void __cdecl mca::Backend::run > >> > >> (void)" (?run at Backend@mca@@QEAAXXZ) > >> [C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj] > >> > >> > >> > >> > >> > >> > >> > >> From: Valentin Churavy [mailto:v.churavy at gmail.com] > >> Sent: Wednesday, May 16, 2018 1:54 PM > >> To: Robinson, Paul > >> Cc: llvm-dev at lists.llvm.org > >> Subject: Re: [llvm-dev] Windows build strangeness > >> > >> > >> > >> What kind of missing symbols are you getting? > >> > >> I had to work around dependencies for a Mingw32 build. See > >> https://reviews.llvm.org/D44650 > >> > >> On Wed, May 16, 2018, 13:13 via llvm-dev <llvm-dev at lists.llvm.org> > wrote: > >> > >> I have the git monorepo, and Visual Studio 2015. I am finding that > >> running a build from the command line with msbuild (as a nightly job) > >> invariably fails on the first try, and succeeds on a retry. > >> > >> The first msbuild command looks like this: > >> > >> msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo" /m:6 > >> /t:Rebuild > >> > >> This appears to compile everything okay, but invariably fails with > >> some set of link-time errors, reporting missing symbols. Then I run > >> msbuild again, but without the /t:Rebuild (thus it's a dirty build > >> the second time) and this succeeds. > >> > >> Is there some inadequacy in the CMake dependencies somewhere, so that > >> some links are trying to happen before the libraries are completely > >> finished? Or just an msbuild issue? > >> > >> I made my nightly script check for errors and run the second round, > >> because it has been happening so consistently, but a proper build > >> system shouldn't require such a workaround. > >> > >> For reference, I generated the sln/project files like this: > >> > >> cmake -G "Visual Studio 14 Win64" -Thost=x64 ^ > >> -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^ > >> -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^ > >> ..\llvm-project\llvm > >> > >> (Yes I know the optimized tablegen comes from a separate tree, that's > >> built separately and always works. It's just the main build that > >> fails.) > >> > >> Thanks, > >> --paulr > >> > >> _______________________________________________ > >> LLVM Developers mailing list > >> llvm-dev at lists.llvm.org > >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >> > >> _______________________________________________ > >> LLVM Developers mailing list > >> llvm-dev at lists.llvm.org > >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > > > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180517/0f033986/attachment.html>
It looks like building ZERO_CHECK first to reconstruct the project files as
needed, then running msbuild a second time to do the actual build, has solved
the problem. At least, last night's run didn't take the usual two
tries. Running msbuild twice is a little bit simpler than running cmake
explicitly, the way I have my scripts set up, but I'm sure that would work
as well.
Thanks everybody!
--paulr
From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] On Behalf Of James
Henderson via llvm-dev
Sent: Thursday, May 17, 2018 5:54 AM
To: Martin J. O'Riordan
Cc: LLVM Developers; Valentin Churavy
Subject: Re: [llvm-dev] Windows build strangeness
From my own experience this is what I think happens when building the whole
solution through Visual Studio's UI. This also happens for building
individual projects. I assume something similar happens when building via the
command-line, but I rarely do that, so I can't be certain.
1) Visual Studio/MSBuild (I don't know which, but probably MSBuild)
determines the dependency graph of the projects that are to be built.
2) The "ZeroCheck" project runs and cmake is rerun if any of the
CMakeLists.txt files have been updated.
3) The build continues for each of the projects, using the dependency graph from
1). If the projects from 1) were updated due to 2) then the updated projects
will be built in the updated way (e.g. new files will be included in that
build).
4) Since any new projects created by re-running cmake in 2) were not in the
dependency graph in 1), projects that now rely on them will likely get link
errors.
There are two solutions I've used in my development: 1) always run cmake
explicitly after doing an svn/git fetch before building or 2) build twice. The
second build is usually a lot faster, so it doesn't really matter.
James
On 16 May 2018 at 22:07, Martin J. O'Riordan via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
With VS2013 I found that editing a CMakeLists.txt file caused CMake to be re-run
successfully and the build to also continue successfully, but since I switched
to VS2015 the CMake re-run occurs - apparently successfully, but more often than
not the build failed afterwards from either the IDE and from MSBuild. Since I
seldom change the CMakeLists.txt files, I simply do a clean CMake configuration
anytime I do change them. I also use the "Build" rather than
"Rebuild" for incremental development.
MartinO
-----Original Message-----
From: llvm-dev [mailto:llvm-dev-bounces at
lists.llvm.org<mailto:llvm-dev-bounces at lists.llvm.org>] On Behalf Of
Michael Kruse via llvm-dev
Sent: 16 May 2018 21:26
To: Reid Kleckner <rnk at google.com<mailto:rnk at google.com>>
Cc: llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at
lists.llvm.org>>; Valentin Churavy <v.churavy at
gmail.com<mailto:v.churavy at gmail.com>>
Subject: Re: [llvm-dev] Windows build strangeness
msbuild is is able to re-run cmake if a CMakeLists.txt changes. CMake adds a
special project "ZERO_CHECK" that does this. However, I am not sure it
runs when invoking on the individual projects instead of the solution.
Try the cmake --build command, which should output the following:
> cmake --build . --target opt
CMake is re-running because
C:/Users/Meinersbur/build/llvm/vc15_debug_x86/tools/polly/CMakeFiles/generate.stamp
is out-of-date.
[...]
Michael
2018-05-16 15:08 GMT-05:00 Reid Kleckner via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at
lists.llvm.org>>:> I think MSBuild isn't capable of re-running cmake and then reloading
> the project files when CMakeLists.txt changes. It re-runs cmake, but
> then continues the build with the stale projects. That probably
> explains the "PipSqueek.cxx doesn't exist" errors. As for the
link
> error, it could also be caused by things like a file rename not getting
picked up by MSBuild.
>
> The fix is probably to always run cmake before MSBuild in the nightly job.
> This isn't necessary if you are using ninja or make, if those are
options.
>
> On Wed, May 16, 2018 at 12:59 PM via llvm-dev
> <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>>
> wrote:
>>
>> Here are a couple of representative errors. C:\Dev\upstream\gitmono
>> is where I keep my clone.
>>
>>
>>
>> "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj"
(Rebuild
>> target)
>> (1) ->
>>
>>
>>
"C:\Dev\upstream\gitmono\wbuild\unittests\Support\DynamicLibrary\SecondLib.vcxproj"
>> (default target) (170:2) ->
>>
>> c1xx : fatal error C1083: Cannot open source file:
>> 'C:\Dev\upstream\gitmono\llvm-project\llvm\unittests\Suppor
>>
>> t\DynamicLibrary\PipSqueak.cxx': No such file or directory
>> [C:\Dev\upstream\gitmono\wbuild\unittests\Support\Dyn
>>
>> amicLibrary\SecondLib.vcxproj]
>>
>>
>>
>> "C:\Dev\upstream\gitmono\wbuild\ALL_BUILD.vcxproj"
(Rebuild
>> target)
>> (1) ->
>>
>>
"C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj"
>> (default target) (339:2) ->
>>
>> (Link target) ->
>>
>> Backend.obj : error LNK2019: unresolved external symbol
"public:
>> virtual bool __cdecl mca::FetchStage::isReady
>>
>> (void)const " (?isReady at FetchStage@mca@@UEBA_NXZ)
referenced
>> in function "public: void __cdecl mca::Backend::run
>>
>> (void)" (?run at Backend@mca@@QEAAXXZ)
>> [C:\Dev\upstream\gitmono\wbuild\tools\llvm-mca\llvm-mca.vcxproj]
>>
>>
>>
>>
>>
>>
>>
>> From: Valentin Churavy [mailto:v.churavy at
gmail.com<mailto:v.churavy at gmail.com>]
>> Sent: Wednesday, May 16, 2018 1:54 PM
>> To: Robinson, Paul
>> Cc: llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
>> Subject: Re: [llvm-dev] Windows build strangeness
>>
>>
>>
>> What kind of missing symbols are you getting?
>>
>> I had to work around dependencies for a Mingw32 build. See
>> https://reviews.llvm.org/D44650
>>
>> On Wed, May 16, 2018, 13:13 via llvm-dev <llvm-dev at
lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
>>
>> I have the git monorepo, and Visual Studio 2015. I am finding that
>> running a build from the command line with msbuild (as a nightly job)
>> invariably fails on the first try, and succeeds on a retry.
>>
>> The first msbuild command looks like this:
>>
>> msbuild ALL_BUILD.vcxproj /p:Configuration="RelWithDebInfo"
/m:6
>> /t:Rebuild
>>
>> This appears to compile everything okay, but invariably fails with
>> some set of link-time errors, reporting missing symbols. Then I run
>> msbuild again, but without the /t:Rebuild (thus it's a dirty build
>> the second time) and this succeeds.
>>
>> Is there some inadequacy in the CMake dependencies somewhere, so that
>> some links are trying to happen before the libraries are completely
>> finished? Or just an msbuild issue?
>>
>> I made my nightly script check for errors and run the second round,
>> because it has been happening so consistently, but a proper build
>> system shouldn't require such a workaround.
>>
>> For reference, I generated the sln/project files like this:
>>
>> cmake -G "Visual Studio 14 Win64" -Thost=x64 ^
>> -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_OPTIMIZED_TABLEGEN=ON ^
>> -DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" ^
>> ..\llvm-project\llvm
>>
>> (Yes I know the optimized tablegen comes from a separate tree,
that's
>> built separately and always works. It's just the main build that
>> fails.)
>>
>> Thanks,
>> --paulr
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180517/f801d973/attachment-0001.html>