llvm dev - Dec 2020 - Ninja hangs when I try to build

Sorry, I'm not familiar with "the usual message about switching from
the
32-bit to the 64-bit linker."  Can you elaborate?
> Is there an easy way to turn off PDB creation?
Probably, but I don't know it offhand.  The MS linker is going to produce a
PDB by default if /DEBUG is specified, and it's pretty common to generate
them for "release" builds as well.

My guess is that you're building multiple configurations at once, and
different configurations (e.g., 32- and 64-bit, or debug and release) are
trying to write their own PDBs to the same file name.  I generally only
build one configuration at a time, and I use ninja (even with MSVC) rather
than the Visual Studio solution.  Off the top of my head, exactly where the
Cmake-generated build configurations try to place the PDBs.

On Tue, Dec 8, 2020 at 1:17 PM Paul C. Anagnostopoulos <paul at
windfall.com>
wrote:
> Thank you! I killed mspdbsrv and things got much further. Now I'm
getting
> the usual message about switching from the 32-bit to the 64-bit linker, and
> then an "Unexpected PDB error; RPC (23)" message. This has been
going on
> for awhile.
>
> Is there an easy way to turn off PDB creation?
>
> At 12/8/2020 02:17 PM, Adrian McCarthy wrote:
> >The program database manager is a separate process VS uses to compile
> debug info into a PDB.  The error happens when it's trying to update a
PDB
> that was created (or last modified) by a newer version of the PDB manager.
> >
> ><
>
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-errors-1/fatal-error-c1902?view=msvc-160
> >
>
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-errors-1/fatal-error-c1902?view=msvc-160
> >
> >I suspect you have more than one version of VS installed.
> >
> >I would close all instances of Visual Studio, check your environment to
> ensure that your build is set up to use the specific version you're
> expecting, and make sure there are no lingering processes runningÂ
> mspdbsrv.exe, and then try again.  If the problem recurs after that,
I'd
> repeat but also delete the PDB files in question and force the compiler to
> rebuild them from scratch.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201208/9f9a6aed/attachment.html>

I think the linker message occurs because Paul is compiling a 32-bit target
(x86). When the 32-bit linker is out-of-space it automatically falls back to the
64-bit linker (x64), thus the message.

Paul, there are several options here. You could perhaps use the 64-bit target
instead? Such as:
> mkdir buildVS && cd buildVS
>  cmake -G"Visual Studio 16 2019" -Thost=x64 ../llvm (your other
cmake options)
The default scenario you’re seeing is `cl.exe` using /Zi which offloads the PDB
generation to `mspdbsrv.exe`. Since the LLVM libs are quite big, something in
there might be running out of memory in a 32-bit process. You should be fine if
switching to 64-bit host processes.


Another option is to build with Clang instead of MSVC in which case /Zi is like
/Z7.
There’s already LLVM 10.0 embedded into Visual Studio (ensure it is installed
through the VS Installer, click Modify):
> cmake -G "Visual Studio 16 2019" -TClangCL ../llvm (your other
cmake options)
Or download & install the LLVM 11.0 release (and install the “llvm2019”
extension in Visual Studio: Extensions menu, Manage Extensions, click “Online”
on the left, and use the search box):
> cmake -G "Visual Studio 16 2019" -Tllvm ../llvm (your other cmake
options)

Another other option is to create a secondary build folder and use ninja for
building, and VS for editing & debugging. This is the best in terms of
iteration times I would say.
To use MSVC & Ninja you would do:
> mkdir buildninja && cd buildninja
> cmake -G Ninja ../llvm -DCMAKE_BUILD_TYPE=Debug
-DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_ENABLE_PROJECTS="clang;lld;llvm" 
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_LIBXML2=OFF
> ninja check-all
Or if you want to use LLVM 11.0 & Ninja:
> cmake -G Ninja ../llvm -DCMAKE_BUILD_TYPE=Debug
-DLLVM_OPTIMIZED_TABLEGEN=ON -DCMAKE_C_COMPILER=”C:/Program
Files/LLVM/bin/clang-cl.exe” -DCMAKE_CXX_COMPILER=”C:/Program
Files/LLVM/bin/clang-cl.exe” -DCMAKE_LINKER=”C:/Program
Files/LLVM/bin/lld-link.exe” -DLLVM_ENABLE_PROJECTS="clang;lld;llvm" 
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_LIBXML2=OFF
Important - ensure all the above commands run in a Visual Studio cmd prompt, by
first launching “x64 Native Tools Command Prompt for VS 2019” from your start
menu.

Alex.

De : llvm-dev <llvm-dev-bounces at lists.llvm.org> De la part de Adrian
McCarthy via llvm-dev
Envoyé : December 8, 2020 5:14 PM
À : Paul C. Anagnostopoulos <paul at windfall.com>
Cc : llvm-dev <llvm-dev at lists.llvm.org>
Objet : Re: [llvm-dev] Ninja hangs when I try to build

Sorry, I'm not familiar with "the usual message about switching from
the 32-bit to the 64-bit linker."  Can you elaborate?
> Is there an easy way to turn off PDB creation?
Probably, but I don't know it offhand.  The MS linker is going to produce a
PDB by default if /DEBUG is specified, and it's pretty common to generate
them for "release" builds as well.

My guess is that you're building multiple configurations at once, and
different configurations (e.g., 32- and 64-bit, or debug and release) are trying
to write their own PDBs to the same file name.  I generally only build one
configuration at a time, and I use ninja (even with MSVC) rather than the Visual
Studio solution.  Off the top of my head, exactly where the Cmake-generated
build configurations try to place the PDBs.

On Tue, Dec 8, 2020 at 1:17 PM Paul C. Anagnostopoulos <paul at
windfall.com<mailto:paul at windfall.com>> wrote:
Thank you! I killed mspdbsrv and things got much further. Now I'm getting
the usual message about switching from the 32-bit to the 64-bit linker, and then
an "Unexpected PDB error; RPC (23)" message. This has been going on
for awhile.

Is there an easy way to turn off PDB creation?

At 12/8/2020 02:17 PM, Adrian McCarthy wrote:
>The program database manager is a separate process VS uses to compile debug
info into a PDB.  The error happens when it's trying to update a PDB that
was created (or last modified) by a newer version of the PDB manager.
>
><https://docs.microsoft.com/en-us/cpp/error-messages/compiler-errors-1/fatal-error-c1902?view=msvc-160>https://docs.microsoft.com/en-us/cpp/error-messages/compiler-errors-1/fatal-error-c1902?view=msvc-160
>
>I suspect you have more than one version of VS installed.
>
>I would close all instances of Visual Studio, check your environment to
ensure that your build is set up to use the specific version you're
expecting, and make sure there are no lingering processes running mspdbsrv.exe,
and then try again.  If the problem recurs after that, I'd repeat but also
delete the PDB files in question and force the compiler to rebuild them from
scratch.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201208/d0ca9534/attachment.html>

The 32-bit linker runs out of heap and something (Ninja?) switches to the 64-bit
linker.

I'm not sure what you mean by building multiple configurations at once.
I'm new here. ;-) I am using Ninja to do the build, which builds just one
configuration, I assume.

I deleted all the PDB files and the build finished. I need to do some more
research.

At 12/8/2020 05:13 PM, Adrian McCarthy wrote:
>Sorry, I'm not familiar with "the usual message about switching
from the 32-bit to the 64-bit linker."Â  Can you elaborate?
>
>> Is there an easy way to turn off PDB creation?
>
>Probably, but I don't know it offhand.  The MS linker is going to
produce a PDB by default if /DEBUG is specified, and it's pretty common to
generate them for "release" builds as well.
>
>My guess is that you're building multiple configurations at once, and
different configurations (e.g., 32- and 64-bit, or debug and release) are trying
to write their own PDBs to the same file name.  I generally only build one
configuration at a time, and I use ninja (even with MSVC) rather than the Visual
Studio solution.  Off the top of my head, exactly where the Cmake-generated
build configurations try to place the PDBs.

I currently build with Ninja, having run cmake as follows:

cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Debug -DLLVM_ENABLE_SPHINX=ON
-DSPHINX_WARNINGS_AS_ERRORS:BOOL=OFF -DLLVM_BUILD_DOCS:BOOL=ON -S
c:\llvm\llvm-project\llvm

I just added -Thost=x64 to that command and got this:

CMake Error: Error: generator toolset: host=x64
Does not match the toolset used previously:
Either remove the CMakeCache.txt file and CMakeFiles directory or choose a
different binary directory.

Shall I just go ahead and follow those instructions?

Forgive my lack of knowledge; this is all new to me.


At 12/8/2020 06:36 PM, Alexandre Ganea wrote:
>Another other option is to create a secondary build folder and use ninja for
building, and VS for editing & debugging. This is the best in terms of
iteration times I would say.
>To use MSVC & Ninja you would do:
> 
>> mkdir buildninja && cd buildninja
>> cmake -G Ninja ../llvm -DCMAKE_BUILD_TYPE=Debug
-DLLVM_OPTIMIZED_TABLEGEN=ON -DLLVM_ENABLE_PROJECTS="clang;lld;llvm" 
-DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_LIBXML2=OFF
>> ninja check-all