On Thu, Oct 23, 2014 at 2:46 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote:> 2014-10-23 11:34 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >> On Thu, Oct 23, 2014 at 2:24 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>> I don't think this is the right approach. >>> >>> Currently we intentionally define malloc etc without changing the >>> names and (when stuff works ok) the linker just links all the mem >>> allocator calls with calls to our RTL. This is kind of a link-time >>> interception. >> >> How could that work in the presence of also having the MSVC CRT >> libraries linked in? When the linker finds the duplicate definition of >> any of those functions, it will produce that link error. You can use >> /FORCE:multiple to work around that, but then any usage of free() >> within compiler-rt is liable to find the asan definition. > > I don't know, it just worked :)LoL, not a vote of confidence. ;-)> >>> I'm not 100% sure off the top of my head but I think __asan_init is >>> explicitly called from the CRT init code later than calloc gets >>> called() from CRT. To handle everything correctly, calloc() must be >>> "statically" intercepted before __asan_init is called from the CRT >>> (hence before "dynamic" interceptors are set up). >>> >>> Can you please compare what happens in your build configuration compared to: >>> $ cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release >>> -DLLVM_TARGETS_TO_BUILD=X86 .. && ninja >>> instead? >> >> Do I need something special to support ninja? > > Put it onto your PATH? > e.g. you can > $ svn export http://src.chromium.org/svn/trunk/tools/depot_tools/ninja.exe > and put it into your cmake/binThat worked, and it does compile. I know basically nothing about ninja... is there a way to see what flags it passes to link.exe? ~Aaron> >> E:\llvm\compiler-rt-ninja>cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUIL >> D_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 ..\compiler-rt && ninja >> CMake Error: CMake was unable to find a build program corresponding to "Ninja". >> CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build t >> ool. >> CMake Error: Error required internal CMake variable not set, cmake may be not be >> built correctly. >> Missing variable is: >> CMAKE_C_COMPILER_ENV_VAR >> CMake Error: Error required internal CMake variable not set, cmake may be not be >> built correctly. >> Missing variable is: >> CMAKE_C_COMPILER >> CMake Error: Could not find cmake module file: E:/llvm/compiler-rt-ninja/CMakeFi >> les/2.8.12.2/CMakeCCompiler.cmake >> CMake Error: Error required internal CMake variable not set, cmake may be not be >> built correctly. >> Missing variable is: >> CMAKE_CXX_COMPILER_ENV_VAR >> CMake Error: Error required internal CMake variable not set, cmake may be not be >> built correctly. >> Missing variable is: >> CMAKE_CXX_COMPILER >> CMake Error: Could not find cmake module file: E:/llvm/compiler-rt-ninja/CMakeFi >> les/2.8.12.2/CMakeCXXCompiler.cmake >> -- Configuring incomplete, errors occurred! >> >> ~Aaron >> >>> >>> 2014-10-23 11:17 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>> I think this issue is that we were not using the INTERCEPTOR macros to >>>> define these functions. The following patch seems to work for me to >>>> get the build linking again, however, I cannot test -- when I run >>>> check-asan, I get: >>>> >>>> 2> lit.py: lit.common.cfg:59: fatal: Invalid llvm_tools_dir config >>>> attribute: 'E:/llvm/2013/$(Configuration)/bin' >>>> >>>> ~Aaron >>>> >>>> On Thu, Oct 23, 2014 at 1:20 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>>>> On Thu, Oct 23, 2014 at 1:18 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>>> http://llvm.org/bugs/show_bug.cgi?id=21241 >>>>>> ? >>>>> >>>>> Yuuuup, looks like that's it. :-) I'll see if I can debug and fix. >>>>> >>>>> Thanks! >>>>> >>>>> ~Aaron >>>>> >>>>>> >>>>>> 2014-10-23 10:18 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>>>> On Thu, Oct 23, 2014 at 1:15 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>>>>>>> On Thu, Oct 23, 2014 at 1:13 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>>>>>> Yes it is. >>>>>>>>> Are you doing a Debug or Release build? >>>>>>>>> Using ninja? >>>>>>>> >>>>>>>> Release build, cmake + MSVC (not using ninja). >>>>>>>> >>>>>>>> Perhaps I have it configured stupidly; I have it as an out-of-tree folder, did: >>>>>>>> >>>>>>>> E:\llvm\crt_build>cmake -DLLVM_CONFIG_PATH=E:\llvm\2013\Debug\bin -G >>>>>>>> "Visual Studio 12" ..\compiler-rt >>>>>>> >>>>>>> Sorry, that should read: >>>>>>> >>>>>>> E:\llvm\crt_build>cmake >>>>>>> -DLLVM_CONFIG_PATH=E:\llvm\2013\Debug\bin\llvm-config.exe -G "Visual >>>>>>> Studio 12" ..\compiler-rt >>>>>>> >>>>>>> ~Aaron >>>>>>>> >>>>>>>> Opened up the solution, hit Build All. >>>>>>>> >>>>>>>> ~Aaron >>>>>>>> >>>>>>>>> >>>>>>>>> 2014-10-23 9:51 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>>>>>>> Hello! I'm wondering whether compiler-rt is expected to be buildable >>>>>>>>>> with MSVC 2013. I am currently getting: >>>>>>>>>> >>>>>>>>>> Error 49 error LNK2005: _free already defined in asan_malloc_win.obj >>>>>>>>>> E:\llvm\crt_build\lib\asan\MSVCRT.lib(MSVCR120.dll) >>>>>>>>>> >>>>>>>>>> Which seems to be the only issue (aside from ~50 warnings, which I'll >>>>>>>>>> happily work on cleaning up). I get this when I build the ALL_BUILD >>>>>>>>>> project in the solution from cmake. >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> >>>>>>>>>> ~Aaron >>>>>>>>>> _______________________________________________ >>>>>>>>>> LLVM Developers mailing list >>>>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
On Thu, Oct 23, 2014 at 2:57 PM, Aaron Ballman <aaron at aaronballman.com> wrote:> On Thu, Oct 23, 2014 at 2:46 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >> 2014-10-23 11:34 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>> On Thu, Oct 23, 2014 at 2:24 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>> I don't think this is the right approach. >>>> >>>> Currently we intentionally define malloc etc without changing the >>>> names and (when stuff works ok) the linker just links all the mem >>>> allocator calls with calls to our RTL. This is kind of a link-time >>>> interception. >>> >>> How could that work in the presence of also having the MSVC CRT >>> libraries linked in? When the linker finds the duplicate definition of >>> any of those functions, it will produce that link error. You can use >>> /FORCE:multiple to work around that, but then any usage of free() >>> within compiler-rt is liable to find the asan definition. >> >> I don't know, it just worked :) > > LoL, not a vote of confidence. ;-) > >> >>>> I'm not 100% sure off the top of my head but I think __asan_init is >>>> explicitly called from the CRT init code later than calloc gets >>>> called() from CRT. To handle everything correctly, calloc() must be >>>> "statically" intercepted before __asan_init is called from the CRT >>>> (hence before "dynamic" interceptors are set up). >>>> >>>> Can you please compare what happens in your build configuration compared to: >>>> $ cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release >>>> -DLLVM_TARGETS_TO_BUILD=X86 .. && ninja >>>> instead? >>> >>> Do I need something special to support ninja? >> >> Put it onto your PATH? >> e.g. you can >> $ svn export http://src.chromium.org/svn/trunk/tools/depot_tools/ninja.exe >> and put it into your cmake/bin > > That worked, and it does compile. I know basically nothing about > ninja... is there a way to see what flags it passes to link.exe?I've found a way to check the linker output from the two builds, and they are different.>From the ninja build, with /VERBOSE passed to the linker:Searching D:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\LIB\LIBCMT.lib: ... At the same stage of linking, from the MSVC build, with /VERBOSE passed to the linker: 2> Searching D:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\lib\MSVCRT.lib: ... 2> Found __imp__free 2> Referenced in MSVCRT.lib(crtdll.obj) 2> Loaded MSVCRT.lib(MSVCR120.dll) 2>MSVCRT.lib(MSVCR120.dll) : error LNK2005: _free already defined in asan_malloc_win.obj ... The ninja build has: Processed /DISALLOWLIB:libcmtd.lib Processed /DISALLOWLIB:msvcrt.lib Processed /DISALLOWLIB:msvcrtd.lib The MSVC build has: 2> Processed /DEFAULTLIB:kernel32.lib 2> Processed /DISALLOWLIB:libcmt.lib 2> Processed /DISALLOWLIB:libcmtd.lib 2> Processed /DISALLOWLIB:msvcrtd.lib So the two builds are not equivalent; the ninja build is somehow disallowing msvcrt.lib while the MSVC build is not. ~Aaron
try something along the lines of "-d keeprsp -v" 2014-10-23 11:57 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>:> On Thu, Oct 23, 2014 at 2:46 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >> 2014-10-23 11:34 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>> On Thu, Oct 23, 2014 at 2:24 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>> I don't think this is the right approach. >>>> >>>> Currently we intentionally define malloc etc without changing the >>>> names and (when stuff works ok) the linker just links all the mem >>>> allocator calls with calls to our RTL. This is kind of a link-time >>>> interception. >>> >>> How could that work in the presence of also having the MSVC CRT >>> libraries linked in? When the linker finds the duplicate definition of >>> any of those functions, it will produce that link error. You can use >>> /FORCE:multiple to work around that, but then any usage of free() >>> within compiler-rt is liable to find the asan definition. >> >> I don't know, it just worked :) > > LoL, not a vote of confidence. ;-) > >> >>>> I'm not 100% sure off the top of my head but I think __asan_init is >>>> explicitly called from the CRT init code later than calloc gets >>>> called() from CRT. To handle everything correctly, calloc() must be >>>> "statically" intercepted before __asan_init is called from the CRT >>>> (hence before "dynamic" interceptors are set up). >>>> >>>> Can you please compare what happens in your build configuration compared to: >>>> $ cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release >>>> -DLLVM_TARGETS_TO_BUILD=X86 .. && ninja >>>> instead? >>> >>> Do I need something special to support ninja? >> >> Put it onto your PATH? >> e.g. you can >> $ svn export http://src.chromium.org/svn/trunk/tools/depot_tools/ninja.exe >> and put it into your cmake/bin > > That worked, and it does compile. I know basically nothing about > ninja... is there a way to see what flags it passes to link.exe? > > ~Aaron > >> >>> E:\llvm\compiler-rt-ninja>cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUIL >>> D_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 ..\compiler-rt && ninja >>> CMake Error: CMake was unable to find a build program corresponding to "Ninja". >>> CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build t >>> ool. >>> CMake Error: Error required internal CMake variable not set, cmake may be not be >>> built correctly. >>> Missing variable is: >>> CMAKE_C_COMPILER_ENV_VAR >>> CMake Error: Error required internal CMake variable not set, cmake may be not be >>> built correctly. >>> Missing variable is: >>> CMAKE_C_COMPILER >>> CMake Error: Could not find cmake module file: E:/llvm/compiler-rt-ninja/CMakeFi >>> les/2.8.12.2/CMakeCCompiler.cmake >>> CMake Error: Error required internal CMake variable not set, cmake may be not be >>> built correctly. >>> Missing variable is: >>> CMAKE_CXX_COMPILER_ENV_VAR >>> CMake Error: Error required internal CMake variable not set, cmake may be not be >>> built correctly. >>> Missing variable is: >>> CMAKE_CXX_COMPILER >>> CMake Error: Could not find cmake module file: E:/llvm/compiler-rt-ninja/CMakeFi >>> les/2.8.12.2/CMakeCXXCompiler.cmake >>> -- Configuring incomplete, errors occurred! >>> >>> ~Aaron >>> >>>> >>>> 2014-10-23 11:17 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>> I think this issue is that we were not using the INTERCEPTOR macros to >>>>> define these functions. The following patch seems to work for me to >>>>> get the build linking again, however, I cannot test -- when I run >>>>> check-asan, I get: >>>>> >>>>> 2> lit.py: lit.common.cfg:59: fatal: Invalid llvm_tools_dir config >>>>> attribute: 'E:/llvm/2013/$(Configuration)/bin' >>>>> >>>>> ~Aaron >>>>> >>>>> On Thu, Oct 23, 2014 at 1:20 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>>>>> On Thu, Oct 23, 2014 at 1:18 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>>>> http://llvm.org/bugs/show_bug.cgi?id=21241 >>>>>>> ? >>>>>> >>>>>> Yuuuup, looks like that's it. :-) I'll see if I can debug and fix. >>>>>> >>>>>> Thanks! >>>>>> >>>>>> ~Aaron >>>>>> >>>>>>> >>>>>>> 2014-10-23 10:18 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>>>>> On Thu, Oct 23, 2014 at 1:15 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>>>>>>>> On Thu, Oct 23, 2014 at 1:13 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>>>>>>> Yes it is. >>>>>>>>>> Are you doing a Debug or Release build? >>>>>>>>>> Using ninja? >>>>>>>>> >>>>>>>>> Release build, cmake + MSVC (not using ninja). >>>>>>>>> >>>>>>>>> Perhaps I have it configured stupidly; I have it as an out-of-tree folder, did: >>>>>>>>> >>>>>>>>> E:\llvm\crt_build>cmake -DLLVM_CONFIG_PATH=E:\llvm\2013\Debug\bin -G >>>>>>>>> "Visual Studio 12" ..\compiler-rt >>>>>>>> >>>>>>>> Sorry, that should read: >>>>>>>> >>>>>>>> E:\llvm\crt_build>cmake >>>>>>>> -DLLVM_CONFIG_PATH=E:\llvm\2013\Debug\bin\llvm-config.exe -G "Visual >>>>>>>> Studio 12" ..\compiler-rt >>>>>>>> >>>>>>>> ~Aaron >>>>>>>>> >>>>>>>>> Opened up the solution, hit Build All. >>>>>>>>> >>>>>>>>> ~Aaron >>>>>>>>> >>>>>>>>>> >>>>>>>>>> 2014-10-23 9:51 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>>>>>>>> Hello! I'm wondering whether compiler-rt is expected to be buildable >>>>>>>>>>> with MSVC 2013. I am currently getting: >>>>>>>>>>> >>>>>>>>>>> Error 49 error LNK2005: _free already defined in asan_malloc_win.obj >>>>>>>>>>> E:\llvm\crt_build\lib\asan\MSVCRT.lib(MSVCR120.dll) >>>>>>>>>>> >>>>>>>>>>> Which seems to be the only issue (aside from ~50 warnings, which I'll >>>>>>>>>>> happily work on cleaning up). I get this when I build the ALL_BUILD >>>>>>>>>>> project in the solution from cmake. >>>>>>>>>>> >>>>>>>>>>> Thanks! >>>>>>>>>>> >>>>>>>>>>> ~Aaron >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> LLVM Developers mailing list >>>>>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
On Thu, Oct 23, 2014 at 3:38 PM, Aaron Ballman <aaron at aaronballman.com> wrote:> On Thu, Oct 23, 2014 at 2:57 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >> On Thu, Oct 23, 2014 at 2:46 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>> 2014-10-23 11:34 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>> On Thu, Oct 23, 2014 at 2:24 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>> I don't think this is the right approach. >>>>> >>>>> Currently we intentionally define malloc etc without changing the >>>>> names and (when stuff works ok) the linker just links all the mem >>>>> allocator calls with calls to our RTL. This is kind of a link-time >>>>> interception. >>>> >>>> How could that work in the presence of also having the MSVC CRT >>>> libraries linked in? When the linker finds the duplicate definition of >>>> any of those functions, it will produce that link error. You can use >>>> /FORCE:multiple to work around that, but then any usage of free() >>>> within compiler-rt is liable to find the asan definition. >>> >>> I don't know, it just worked :) >> >> LoL, not a vote of confidence. ;-) >> >>> >>>>> I'm not 100% sure off the top of my head but I think __asan_init is >>>>> explicitly called from the CRT init code later than calloc gets >>>>> called() from CRT. To handle everything correctly, calloc() must be >>>>> "statically" intercepted before __asan_init is called from the CRT >>>>> (hence before "dynamic" interceptors are set up). >>>>> >>>>> Can you please compare what happens in your build configuration compared to: >>>>> $ cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release >>>>> -DLLVM_TARGETS_TO_BUILD=X86 .. && ninja >>>>> instead? >>>> >>>> Do I need something special to support ninja? >>> >>> Put it onto your PATH? >>> e.g. you can >>> $ svn export http://src.chromium.org/svn/trunk/tools/depot_tools/ninja.exe >>> and put it into your cmake/bin >> >> That worked, and it does compile. I know basically nothing about >> ninja... is there a way to see what flags it passes to link.exe? > > I've found a way to check the linker output from the two builds, and > they are different. > > From the ninja build, with /VERBOSE passed to the linker: > > Searching D:\Program Files (x86)\Microsoft Visual Studio > 12.0\VC\LIB\LIBCMT.lib: > ... > > At the same stage of linking, from the MSVC build, with /VERBOSE > passed to the linker: > > 2> Searching D:\Program Files (x86)\Microsoft Visual Studio > 12.0\VC\lib\MSVCRT.lib: > ... > 2> Found __imp__free > 2> Referenced in MSVCRT.lib(crtdll.obj) > 2> Loaded MSVCRT.lib(MSVCR120.dll) > 2>MSVCRT.lib(MSVCR120.dll) : error LNK2005: _free already defined in > asan_malloc_win.obj > ... > > The ninja build has: > Processed /DISALLOWLIB:libcmtd.lib > Processed /DISALLOWLIB:msvcrt.lib > Processed /DISALLOWLIB:msvcrtd.lib > > The MSVC build has: > 2> Processed /DEFAULTLIB:kernel32.lib > 2> Processed /DISALLOWLIB:libcmt.lib > 2> Processed /DISALLOWLIB:libcmtd.lib > 2> Processed /DISALLOWLIB:msvcrtd.lib > > So the two builds are not equivalent; the ninja build is somehow > disallowing msvcrt.lib while the MSVC build is not.I should note that libcmt.lib is what you get with /MT, and msvcrt.lib is what you get with /MD. I do not understand why Ninja seems to have the libraries associated with /MT when linking the DLL ([100/100] Linking CXX shared library lib\asan\clang_rt.asan_dynamic-i386.dll is where I pulled those lines from). ~Aaron