On Thu, Oct 23, 2014 at 2:24 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote:> I don't think this is the right approach. > > Currently we intentionally define malloc etc without changing the > names and (when stuff works ok) the linker just links all the mem > allocator calls with calls to our RTL. This is kind of a link-time > interception.How could that work in the presence of also having the MSVC CRT libraries linked in? When the linker finds the duplicate definition of any of those functions, it will produce that link error. You can use /FORCE:multiple to work around that, but then any usage of free() within compiler-rt is liable to find the asan definition.> I'm not 100% sure off the top of my head but I think __asan_init is > explicitly called from the CRT init code later than calloc gets > called() from CRT. To handle everything correctly, calloc() must be > "statically" intercepted before __asan_init is called from the CRT > (hence before "dynamic" interceptors are set up). > > Can you please compare what happens in your build configuration compared to: > $ cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release > -DLLVM_TARGETS_TO_BUILD=X86 .. && ninja > instead?Do I need something special to support ninja? E:\llvm\compiler-rt-ninja>cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUIL D_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 ..\compiler-rt && ninja CMake Error: CMake was unable to find a build program corresponding to "Ninja". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build t ool. CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly. Missing variable is: CMAKE_C_COMPILER_ENV_VAR CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly. Missing variable is: CMAKE_C_COMPILER CMake Error: Could not find cmake module file: E:/llvm/compiler-rt-ninja/CMakeFi les/2.8.12.2/CMakeCCompiler.cmake CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly. Missing variable is: CMAKE_CXX_COMPILER_ENV_VAR CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly. Missing variable is: CMAKE_CXX_COMPILER CMake Error: Could not find cmake module file: E:/llvm/compiler-rt-ninja/CMakeFi les/2.8.12.2/CMakeCXXCompiler.cmake -- Configuring incomplete, errors occurred! ~Aaron> > 2014-10-23 11:17 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >> I think this issue is that we were not using the INTERCEPTOR macros to >> define these functions. The following patch seems to work for me to >> get the build linking again, however, I cannot test -- when I run >> check-asan, I get: >> >> 2> lit.py: lit.common.cfg:59: fatal: Invalid llvm_tools_dir config >> attribute: 'E:/llvm/2013/$(Configuration)/bin' >> >> ~Aaron >> >> On Thu, Oct 23, 2014 at 1:20 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>> On Thu, Oct 23, 2014 at 1:18 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>> http://llvm.org/bugs/show_bug.cgi?id=21241 >>>> ? >>> >>> Yuuuup, looks like that's it. :-) I'll see if I can debug and fix. >>> >>> Thanks! >>> >>> ~Aaron >>> >>>> >>>> 2014-10-23 10:18 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>> On Thu, Oct 23, 2014 at 1:15 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>>>>> On Thu, Oct 23, 2014 at 1:13 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>>>> Yes it is. >>>>>>> Are you doing a Debug or Release build? >>>>>>> Using ninja? >>>>>> >>>>>> Release build, cmake + MSVC (not using ninja). >>>>>> >>>>>> Perhaps I have it configured stupidly; I have it as an out-of-tree folder, did: >>>>>> >>>>>> E:\llvm\crt_build>cmake -DLLVM_CONFIG_PATH=E:\llvm\2013\Debug\bin -G >>>>>> "Visual Studio 12" ..\compiler-rt >>>>> >>>>> Sorry, that should read: >>>>> >>>>> E:\llvm\crt_build>cmake >>>>> -DLLVM_CONFIG_PATH=E:\llvm\2013\Debug\bin\llvm-config.exe -G "Visual >>>>> Studio 12" ..\compiler-rt >>>>> >>>>> ~Aaron >>>>>> >>>>>> Opened up the solution, hit Build All. >>>>>> >>>>>> ~Aaron >>>>>> >>>>>>> >>>>>>> 2014-10-23 9:51 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>>>>> Hello! I'm wondering whether compiler-rt is expected to be buildable >>>>>>>> with MSVC 2013. I am currently getting: >>>>>>>> >>>>>>>> Error 49 error LNK2005: _free already defined in asan_malloc_win.obj >>>>>>>> E:\llvm\crt_build\lib\asan\MSVCRT.lib(MSVCR120.dll) >>>>>>>> >>>>>>>> Which seems to be the only issue (aside from ~50 warnings, which I'll >>>>>>>> happily work on cleaning up). I get this when I build the ALL_BUILD >>>>>>>> project in the solution from cmake. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> ~Aaron >>>>>>>> _______________________________________________ >>>>>>>> LLVM Developers mailing list >>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
2014-10-23 11:34 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>:> On Thu, Oct 23, 2014 at 2:24 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >> I don't think this is the right approach. >> >> Currently we intentionally define malloc etc without changing the >> names and (when stuff works ok) the linker just links all the mem >> allocator calls with calls to our RTL. This is kind of a link-time >> interception. > > How could that work in the presence of also having the MSVC CRT > libraries linked in? When the linker finds the duplicate definition of > any of those functions, it will produce that link error. You can use > /FORCE:multiple to work around that, but then any usage of free() > within compiler-rt is liable to find the asan definition.I don't know, it just worked :)>> I'm not 100% sure off the top of my head but I think __asan_init is >> explicitly called from the CRT init code later than calloc gets >> called() from CRT. To handle everything correctly, calloc() must be >> "statically" intercepted before __asan_init is called from the CRT >> (hence before "dynamic" interceptors are set up). >> >> Can you please compare what happens in your build configuration compared to: >> $ cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release >> -DLLVM_TARGETS_TO_BUILD=X86 .. && ninja >> instead? > > Do I need something special to support ninja?Put it onto your PATH? e.g. you can $ svn export http://src.chromium.org/svn/trunk/tools/depot_tools/ninja.exe and put it into your cmake/bin> E:\llvm\compiler-rt-ninja>cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUIL > D_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 ..\compiler-rt && ninja > CMake Error: CMake was unable to find a build program corresponding to "Ninja". > CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build t > ool. > CMake Error: Error required internal CMake variable not set, cmake may be not be > built correctly. > Missing variable is: > CMAKE_C_COMPILER_ENV_VAR > CMake Error: Error required internal CMake variable not set, cmake may be not be > built correctly. > Missing variable is: > CMAKE_C_COMPILER > CMake Error: Could not find cmake module file: E:/llvm/compiler-rt-ninja/CMakeFi > les/2.8.12.2/CMakeCCompiler.cmake > CMake Error: Error required internal CMake variable not set, cmake may be not be > built correctly. > Missing variable is: > CMAKE_CXX_COMPILER_ENV_VAR > CMake Error: Error required internal CMake variable not set, cmake may be not be > built correctly. > Missing variable is: > CMAKE_CXX_COMPILER > CMake Error: Could not find cmake module file: E:/llvm/compiler-rt-ninja/CMakeFi > les/2.8.12.2/CMakeCXXCompiler.cmake > -- Configuring incomplete, errors occurred! > > ~Aaron > >> >> 2014-10-23 11:17 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>> I think this issue is that we were not using the INTERCEPTOR macros to >>> define these functions. The following patch seems to work for me to >>> get the build linking again, however, I cannot test -- when I run >>> check-asan, I get: >>> >>> 2> lit.py: lit.common.cfg:59: fatal: Invalid llvm_tools_dir config >>> attribute: 'E:/llvm/2013/$(Configuration)/bin' >>> >>> ~Aaron >>> >>> On Thu, Oct 23, 2014 at 1:20 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>>> On Thu, Oct 23, 2014 at 1:18 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>> http://llvm.org/bugs/show_bug.cgi?id=21241 >>>>> ? >>>> >>>> Yuuuup, looks like that's it. :-) I'll see if I can debug and fix. >>>> >>>> Thanks! >>>> >>>> ~Aaron >>>> >>>>> >>>>> 2014-10-23 10:18 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>>> On Thu, Oct 23, 2014 at 1:15 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>>>>>> On Thu, Oct 23, 2014 at 1:13 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>>>>> Yes it is. >>>>>>>> Are you doing a Debug or Release build? >>>>>>>> Using ninja? >>>>>>> >>>>>>> Release build, cmake + MSVC (not using ninja). >>>>>>> >>>>>>> Perhaps I have it configured stupidly; I have it as an out-of-tree folder, did: >>>>>>> >>>>>>> E:\llvm\crt_build>cmake -DLLVM_CONFIG_PATH=E:\llvm\2013\Debug\bin -G >>>>>>> "Visual Studio 12" ..\compiler-rt >>>>>> >>>>>> Sorry, that should read: >>>>>> >>>>>> E:\llvm\crt_build>cmake >>>>>> -DLLVM_CONFIG_PATH=E:\llvm\2013\Debug\bin\llvm-config.exe -G "Visual >>>>>> Studio 12" ..\compiler-rt >>>>>> >>>>>> ~Aaron >>>>>>> >>>>>>> Opened up the solution, hit Build All. >>>>>>> >>>>>>> ~Aaron >>>>>>> >>>>>>>> >>>>>>>> 2014-10-23 9:51 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>>>>>> Hello! I'm wondering whether compiler-rt is expected to be buildable >>>>>>>>> with MSVC 2013. I am currently getting: >>>>>>>>> >>>>>>>>> Error 49 error LNK2005: _free already defined in asan_malloc_win.obj >>>>>>>>> E:\llvm\crt_build\lib\asan\MSVCRT.lib(MSVCR120.dll) >>>>>>>>> >>>>>>>>> Which seems to be the only issue (aside from ~50 warnings, which I'll >>>>>>>>> happily work on cleaning up). I get this when I build the ALL_BUILD >>>>>>>>> project in the solution from cmake. >>>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> ~Aaron >>>>>>>>> _______________________________________________ >>>>>>>>> LLVM Developers mailing list >>>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
On Thu, Oct 23, 2014 at 2:46 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote:> 2014-10-23 11:34 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >> On Thu, Oct 23, 2014 at 2:24 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>> I don't think this is the right approach. >>> >>> Currently we intentionally define malloc etc without changing the >>> names and (when stuff works ok) the linker just links all the mem >>> allocator calls with calls to our RTL. This is kind of a link-time >>> interception. >> >> How could that work in the presence of also having the MSVC CRT >> libraries linked in? When the linker finds the duplicate definition of >> any of those functions, it will produce that link error. You can use >> /FORCE:multiple to work around that, but then any usage of free() >> within compiler-rt is liable to find the asan definition. > > I don't know, it just worked :)LoL, not a vote of confidence. ;-)> >>> I'm not 100% sure off the top of my head but I think __asan_init is >>> explicitly called from the CRT init code later than calloc gets >>> called() from CRT. To handle everything correctly, calloc() must be >>> "statically" intercepted before __asan_init is called from the CRT >>> (hence before "dynamic" interceptors are set up). >>> >>> Can you please compare what happens in your build configuration compared to: >>> $ cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release >>> -DLLVM_TARGETS_TO_BUILD=X86 .. && ninja >>> instead? >> >> Do I need something special to support ninja? > > Put it onto your PATH? > e.g. you can > $ svn export http://src.chromium.org/svn/trunk/tools/depot_tools/ninja.exe > and put it into your cmake/binThat worked, and it does compile. I know basically nothing about ninja... is there a way to see what flags it passes to link.exe? ~Aaron> >> E:\llvm\compiler-rt-ninja>cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUIL >> D_TYPE=Release -DLLVM_TARGETS_TO_BUILD=X86 ..\compiler-rt && ninja >> CMake Error: CMake was unable to find a build program corresponding to "Ninja". >> CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build t >> ool. >> CMake Error: Error required internal CMake variable not set, cmake may be not be >> built correctly. >> Missing variable is: >> CMAKE_C_COMPILER_ENV_VAR >> CMake Error: Error required internal CMake variable not set, cmake may be not be >> built correctly. >> Missing variable is: >> CMAKE_C_COMPILER >> CMake Error: Could not find cmake module file: E:/llvm/compiler-rt-ninja/CMakeFi >> les/2.8.12.2/CMakeCCompiler.cmake >> CMake Error: Error required internal CMake variable not set, cmake may be not be >> built correctly. >> Missing variable is: >> CMAKE_CXX_COMPILER_ENV_VAR >> CMake Error: Error required internal CMake variable not set, cmake may be not be >> built correctly. >> Missing variable is: >> CMAKE_CXX_COMPILER >> CMake Error: Could not find cmake module file: E:/llvm/compiler-rt-ninja/CMakeFi >> les/2.8.12.2/CMakeCXXCompiler.cmake >> -- Configuring incomplete, errors occurred! >> >> ~Aaron >> >>> >>> 2014-10-23 11:17 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>> I think this issue is that we were not using the INTERCEPTOR macros to >>>> define these functions. The following patch seems to work for me to >>>> get the build linking again, however, I cannot test -- when I run >>>> check-asan, I get: >>>> >>>> 2> lit.py: lit.common.cfg:59: fatal: Invalid llvm_tools_dir config >>>> attribute: 'E:/llvm/2013/$(Configuration)/bin' >>>> >>>> ~Aaron >>>> >>>> On Thu, Oct 23, 2014 at 1:20 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>>>> On Thu, Oct 23, 2014 at 1:18 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>>> http://llvm.org/bugs/show_bug.cgi?id=21241 >>>>>> ? >>>>> >>>>> Yuuuup, looks like that's it. :-) I'll see if I can debug and fix. >>>>> >>>>> Thanks! >>>>> >>>>> ~Aaron >>>>> >>>>>> >>>>>> 2014-10-23 10:18 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>>>> On Thu, Oct 23, 2014 at 1:15 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>>>>>>> On Thu, Oct 23, 2014 at 1:13 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>>>>>> Yes it is. >>>>>>>>> Are you doing a Debug or Release build? >>>>>>>>> Using ninja? >>>>>>>> >>>>>>>> Release build, cmake + MSVC (not using ninja). >>>>>>>> >>>>>>>> Perhaps I have it configured stupidly; I have it as an out-of-tree folder, did: >>>>>>>> >>>>>>>> E:\llvm\crt_build>cmake -DLLVM_CONFIG_PATH=E:\llvm\2013\Debug\bin -G >>>>>>>> "Visual Studio 12" ..\compiler-rt >>>>>>> >>>>>>> Sorry, that should read: >>>>>>> >>>>>>> E:\llvm\crt_build>cmake >>>>>>> -DLLVM_CONFIG_PATH=E:\llvm\2013\Debug\bin\llvm-config.exe -G "Visual >>>>>>> Studio 12" ..\compiler-rt >>>>>>> >>>>>>> ~Aaron >>>>>>>> >>>>>>>> Opened up the solution, hit Build All. >>>>>>>> >>>>>>>> ~Aaron >>>>>>>> >>>>>>>>> >>>>>>>>> 2014-10-23 9:51 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>>>>>>> Hello! I'm wondering whether compiler-rt is expected to be buildable >>>>>>>>>> with MSVC 2013. I am currently getting: >>>>>>>>>> >>>>>>>>>> Error 49 error LNK2005: _free already defined in asan_malloc_win.obj >>>>>>>>>> E:\llvm\crt_build\lib\asan\MSVCRT.lib(MSVCR120.dll) >>>>>>>>>> >>>>>>>>>> Which seems to be the only issue (aside from ~50 warnings, which I'll >>>>>>>>>> happily work on cleaning up). I get this when I build the ALL_BUILD >>>>>>>>>> project in the solution from cmake. >>>>>>>>>> >>>>>>>>>> Thanks! >>>>>>>>>> >>>>>>>>>> ~Aaron >>>>>>>>>> _______________________________________________ >>>>>>>>>> LLVM Developers mailing list >>>>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev