On Thu, Oct 23, 2014 at 3:38 PM, Aaron Ballman <aaron at aaronballman.com> wrote:> On Thu, Oct 23, 2014 at 2:57 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >> On Thu, Oct 23, 2014 at 2:46 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>> 2014-10-23 11:34 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>> On Thu, Oct 23, 2014 at 2:24 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>> I don't think this is the right approach. >>>>> >>>>> Currently we intentionally define malloc etc without changing the >>>>> names and (when stuff works ok) the linker just links all the mem >>>>> allocator calls with calls to our RTL. This is kind of a link-time >>>>> interception. >>>> >>>> How could that work in the presence of also having the MSVC CRT >>>> libraries linked in? When the linker finds the duplicate definition of >>>> any of those functions, it will produce that link error. You can use >>>> /FORCE:multiple to work around that, but then any usage of free() >>>> within compiler-rt is liable to find the asan definition. >>> >>> I don't know, it just worked :) >> >> LoL, not a vote of confidence. ;-) >> >>> >>>>> I'm not 100% sure off the top of my head but I think __asan_init is >>>>> explicitly called from the CRT init code later than calloc gets >>>>> called() from CRT. To handle everything correctly, calloc() must be >>>>> "statically" intercepted before __asan_init is called from the CRT >>>>> (hence before "dynamic" interceptors are set up). >>>>> >>>>> Can you please compare what happens in your build configuration compared to: >>>>> $ cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release >>>>> -DLLVM_TARGETS_TO_BUILD=X86 .. && ninja >>>>> instead? >>>> >>>> Do I need something special to support ninja? >>> >>> Put it onto your PATH? >>> e.g. you can >>> $ svn export http://src.chromium.org/svn/trunk/tools/depot_tools/ninja.exe >>> and put it into your cmake/bin >> >> That worked, and it does compile. I know basically nothing about >> ninja... is there a way to see what flags it passes to link.exe? > > I've found a way to check the linker output from the two builds, and > they are different. > > From the ninja build, with /VERBOSE passed to the linker: > > Searching D:\Program Files (x86)\Microsoft Visual Studio > 12.0\VC\LIB\LIBCMT.lib: > ... > > At the same stage of linking, from the MSVC build, with /VERBOSE > passed to the linker: > > 2> Searching D:\Program Files (x86)\Microsoft Visual Studio > 12.0\VC\lib\MSVCRT.lib: > ... > 2> Found __imp__free > 2> Referenced in MSVCRT.lib(crtdll.obj) > 2> Loaded MSVCRT.lib(MSVCR120.dll) > 2>MSVCRT.lib(MSVCR120.dll) : error LNK2005: _free already defined in > asan_malloc_win.obj > ... > > The ninja build has: > Processed /DISALLOWLIB:libcmtd.lib > Processed /DISALLOWLIB:msvcrt.lib > Processed /DISALLOWLIB:msvcrtd.lib > > The MSVC build has: > 2> Processed /DEFAULTLIB:kernel32.lib > 2> Processed /DISALLOWLIB:libcmt.lib > 2> Processed /DISALLOWLIB:libcmtd.lib > 2> Processed /DISALLOWLIB:msvcrtd.lib > > So the two builds are not equivalent; the ninja build is somehow > disallowing msvcrt.lib while the MSVC build is not.I should note that libcmt.lib is what you get with /MT, and msvcrt.lib is what you get with /MD. I do not understand why Ninja seems to have the libraries associated with /MT when linking the DLL ([100/100] Linking CXX shared library lib\asan\clang_rt.asan_dynamic-i386.dll is where I pulled those lines from). ~Aaron
On Thu, Oct 23, 2014 at 3:42 PM, Aaron Ballman <aaron at aaronballman.com> wrote:> On Thu, Oct 23, 2014 at 3:38 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >> On Thu, Oct 23, 2014 at 2:57 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>> On Thu, Oct 23, 2014 at 2:46 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>> 2014-10-23 11:34 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>> On Thu, Oct 23, 2014 at 2:24 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>>> I don't think this is the right approach. >>>>>> >>>>>> Currently we intentionally define malloc etc without changing the >>>>>> names and (when stuff works ok) the linker just links all the mem >>>>>> allocator calls with calls to our RTL. This is kind of a link-time >>>>>> interception. >>>>> >>>>> How could that work in the presence of also having the MSVC CRT >>>>> libraries linked in? When the linker finds the duplicate definition of >>>>> any of those functions, it will produce that link error. You can use >>>>> /FORCE:multiple to work around that, but then any usage of free() >>>>> within compiler-rt is liable to find the asan definition. >>>> >>>> I don't know, it just worked :) >>> >>> LoL, not a vote of confidence. ;-) >>> >>>> >>>>>> I'm not 100% sure off the top of my head but I think __asan_init is >>>>>> explicitly called from the CRT init code later than calloc gets >>>>>> called() from CRT. To handle everything correctly, calloc() must be >>>>>> "statically" intercepted before __asan_init is called from the CRT >>>>>> (hence before "dynamic" interceptors are set up). >>>>>> >>>>>> Can you please compare what happens in your build configuration compared to: >>>>>> $ cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release >>>>>> -DLLVM_TARGETS_TO_BUILD=X86 .. && ninja >>>>>> instead? >>>>> >>>>> Do I need something special to support ninja? >>>> >>>> Put it onto your PATH? >>>> e.g. you can >>>> $ svn export http://src.chromium.org/svn/trunk/tools/depot_tools/ninja.exe >>>> and put it into your cmake/bin >>> >>> That worked, and it does compile. I know basically nothing about >>> ninja... is there a way to see what flags it passes to link.exe? >> >> I've found a way to check the linker output from the two builds, and >> they are different. >> >> From the ninja build, with /VERBOSE passed to the linker: >> >> Searching D:\Program Files (x86)\Microsoft Visual Studio >> 12.0\VC\LIB\LIBCMT.lib: >> ... >> >> At the same stage of linking, from the MSVC build, with /VERBOSE >> passed to the linker: >> >> 2> Searching D:\Program Files (x86)\Microsoft Visual Studio >> 12.0\VC\lib\MSVCRT.lib: >> ... >> 2> Found __imp__free >> 2> Referenced in MSVCRT.lib(crtdll.obj) >> 2> Loaded MSVCRT.lib(MSVCR120.dll) >> 2>MSVCRT.lib(MSVCR120.dll) : error LNK2005: _free already defined in >> asan_malloc_win.obj >> ... >> >> The ninja build has: >> Processed /DISALLOWLIB:libcmtd.lib >> Processed /DISALLOWLIB:msvcrt.lib >> Processed /DISALLOWLIB:msvcrtd.lib >> >> The MSVC build has: >> 2> Processed /DEFAULTLIB:kernel32.lib >> 2> Processed /DISALLOWLIB:libcmt.lib >> 2> Processed /DISALLOWLIB:libcmtd.lib >> 2> Processed /DISALLOWLIB:msvcrtd.lib >> >> So the two builds are not equivalent; the ninja build is somehow >> disallowing msvcrt.lib while the MSVC build is not. > > I should note that libcmt.lib is what you get with /MT, and msvcrt.lib > is what you get with /MD. I do not understand why Ninja seems to have > the libraries associated with /MT when linking the DLL ([100/100] > Linking CXX shared library lib\asan\clang_rt.asan_dynamic-i386.dll is > where I pulled those lines from).I think I figured out the issue. From the build.ninja file, I found: build lib\asan\CMakeFiles\RTAsan_dynamic.i386.dir\asan_malloc_win.cc.obj: CXX_COMPILER E$:\llvm\compiler-rt\lib\asan\asan_malloc_win.cc DEFINES = -DASAN_DYNAMIC=1 -DASAN_HAS_EXCEPTIONS=1 -DINTERCEPTION_DYNAMIC_CRT -D__func__=__FUNCTION__ DEP_FILE = lib/asan/CMakeFiles/RTAsan_dynamic.i386.dir/asan_malloc_win.cc.obj.d FLAGS = /DWIN32 /D_WINDOWS /W3 /GR /EHsc /W3 /MT /O2 /Ob2 /D NDEBUG -IE:\llvm\compiler-rt\lib\asan\.. /machine:X86 /DWIN32 /D_WINDOWS /W3 /GR /EHsc /W3 /Oy- /GS- /VERBOSE /Zi /wd4391 /wd4722 /wd4291 /wd4800 /GR- /DEBUG OBJECT_DIR = lib\asan\CMakeFiles\RTAsan_dynamic.i386.dir TARGET_PDB = "" Notice how it's passing the /MT flag instead of /MD? That's different from MSVC, which is using this for its command line options: /GS- /TP /analyze- /W3 /wd"4391" /wd"4722" /wd"4291" /wd"4800" /Zc:wchar_t /I"E:\llvm\compiler-rt\lib\asan\.." /Zi /Gm- /O2 /Ob1 /Fd"RTAsan_dynamic.i386.dir\RelWithDebInfo\vc120.pdb" /fp:precise /D "WIN32" /D "_WINDOWS" /D "NDEBUG" /D "EBUG" /D "ASAN_HAS_EXCEPTIONS=1" /D "ASAN_DYNAMIC=1" /D "INTERCEPTION_DYNAMIC_CRT" /D "__func__=__FUNCTION__" /D "CMAKE_INTDIR=\"RelWithDebInfo\"" /D "_MBCS" /errorReport:prompt /WX- /Zc:forScope /GR- /Gd /Oy- /MD /Fa"RelWithDebInfo/" /EHsc /nologo /Fo"RTAsan_dynamic.i386.dir\RelWithDebInfo\" /Fp"RTAsan_dynamic.i386.dir\RelWithDebInfo\RTAsan_dynamic.i386.pch" (Note how it uses /MD) So, basically, ninja is linking as though everything was being compiled for a static library, when we're actually building a DLL (from what I can tell, anyway). ~Aaron
compiler-rt libs must be built with /MT, so the MSVS build is doing the wrong thing here. 2014-10-23 12:52 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>:> On Thu, Oct 23, 2014 at 3:42 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >> On Thu, Oct 23, 2014 at 3:38 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>> On Thu, Oct 23, 2014 at 2:57 PM, Aaron Ballman <aaron at aaronballman.com> wrote: >>>> On Thu, Oct 23, 2014 at 2:46 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>> 2014-10-23 11:34 GMT-07:00 Aaron Ballman <aaron at aaronballman.com>: >>>>>> On Thu, Oct 23, 2014 at 2:24 PM, Timur Iskhodzhanov <timurrrr at google.com> wrote: >>>>>>> I don't think this is the right approach. >>>>>>> >>>>>>> Currently we intentionally define malloc etc without changing the >>>>>>> names and (when stuff works ok) the linker just links all the mem >>>>>>> allocator calls with calls to our RTL. This is kind of a link-time >>>>>>> interception. >>>>>> >>>>>> How could that work in the presence of also having the MSVC CRT >>>>>> libraries linked in? When the linker finds the duplicate definition of >>>>>> any of those functions, it will produce that link error. You can use >>>>>> /FORCE:multiple to work around that, but then any usage of free() >>>>>> within compiler-rt is liable to find the asan definition. >>>>> >>>>> I don't know, it just worked :) >>>> >>>> LoL, not a vote of confidence. ;-) >>>> >>>>> >>>>>>> I'm not 100% sure off the top of my head but I think __asan_init is >>>>>>> explicitly called from the CRT init code later than calloc gets >>>>>>> called() from CRT. To handle everything correctly, calloc() must be >>>>>>> "statically" intercepted before __asan_init is called from the CRT >>>>>>> (hence before "dynamic" interceptors are set up). >>>>>>> >>>>>>> Can you please compare what happens in your build configuration compared to: >>>>>>> $ cmake -GNinja -DLLVM_ENABLE_ASSERTIONS=ON -DCMAKE_BUILD_TYPE=Release >>>>>>> -DLLVM_TARGETS_TO_BUILD=X86 .. && ninja >>>>>>> instead? >>>>>> >>>>>> Do I need something special to support ninja? >>>>> >>>>> Put it onto your PATH? >>>>> e.g. you can >>>>> $ svn export http://src.chromium.org/svn/trunk/tools/depot_tools/ninja.exe >>>>> and put it into your cmake/bin >>>> >>>> That worked, and it does compile. I know basically nothing about >>>> ninja... is there a way to see what flags it passes to link.exe? >>> >>> I've found a way to check the linker output from the two builds, and >>> they are different. >>> >>> From the ninja build, with /VERBOSE passed to the linker: >>> >>> Searching D:\Program Files (x86)\Microsoft Visual Studio >>> 12.0\VC\LIB\LIBCMT.lib: >>> ... >>> >>> At the same stage of linking, from the MSVC build, with /VERBOSE >>> passed to the linker: >>> >>> 2> Searching D:\Program Files (x86)\Microsoft Visual Studio >>> 12.0\VC\lib\MSVCRT.lib: >>> ... >>> 2> Found __imp__free >>> 2> Referenced in MSVCRT.lib(crtdll.obj) >>> 2> Loaded MSVCRT.lib(MSVCR120.dll) >>> 2>MSVCRT.lib(MSVCR120.dll) : error LNK2005: _free already defined in >>> asan_malloc_win.obj >>> ... >>> >>> The ninja build has: >>> Processed /DISALLOWLIB:libcmtd.lib >>> Processed /DISALLOWLIB:msvcrt.lib >>> Processed /DISALLOWLIB:msvcrtd.lib >>> >>> The MSVC build has: >>> 2> Processed /DEFAULTLIB:kernel32.lib >>> 2> Processed /DISALLOWLIB:libcmt.lib >>> 2> Processed /DISALLOWLIB:libcmtd.lib >>> 2> Processed /DISALLOWLIB:msvcrtd.lib >>> >>> So the two builds are not equivalent; the ninja build is somehow >>> disallowing msvcrt.lib while the MSVC build is not. >> >> I should note that libcmt.lib is what you get with /MT, and msvcrt.lib >> is what you get with /MD. I do not understand why Ninja seems to have >> the libraries associated with /MT when linking the DLL ([100/100] >> Linking CXX shared library lib\asan\clang_rt.asan_dynamic-i386.dll is >> where I pulled those lines from). > > I think I figured out the issue. From the build.ninja file, I found: > > build lib\asan\CMakeFiles\RTAsan_dynamic.i386.dir\asan_malloc_win.cc.obj: > CXX_COMPILER E$:\llvm\compiler-rt\lib\asan\asan_malloc_win.cc > DEFINES = -DASAN_DYNAMIC=1 -DASAN_HAS_EXCEPTIONS=1 > -DINTERCEPTION_DYNAMIC_CRT -D__func__=__FUNCTION__ > DEP_FILE = lib/asan/CMakeFiles/RTAsan_dynamic.i386.dir/asan_malloc_win.cc.obj.d > FLAGS = /DWIN32 /D_WINDOWS /W3 /GR /EHsc /W3 /MT /O2 /Ob2 /D NDEBUG > -IE:\llvm\compiler-rt\lib\asan\.. /machine:X86 /DWIN32 > /D_WINDOWS /W3 /GR /EHsc /W3 /Oy- /GS- /VERBOSE /Zi /wd4391 /wd4722 > /wd4291 /wd4800 /GR- /DEBUG > OBJECT_DIR = lib\asan\CMakeFiles\RTAsan_dynamic.i386.dir > TARGET_PDB = "" > > Notice how it's passing the /MT flag instead of /MD? That's different > from MSVC, which is using this for its command line options: > > /GS- /TP /analyze- /W3 /wd"4391" /wd"4722" /wd"4291" /wd"4800" > /Zc:wchar_t /I"E:\llvm\compiler-rt\lib\asan\.." /Zi /Gm- /O2 /Ob1 > /Fd"RTAsan_dynamic.i386.dir\RelWithDebInfo\vc120.pdb" /fp:precise /D > "WIN32" /D "_WINDOWS" /D "NDEBUG" /D "EBUG" /D "ASAN_HAS_EXCEPTIONS=1" > /D "ASAN_DYNAMIC=1" /D "INTERCEPTION_DYNAMIC_CRT" /D > "__func__=__FUNCTION__" /D "CMAKE_INTDIR=\"RelWithDebInfo\"" /D > "_MBCS" /errorReport:prompt /WX- /Zc:forScope /GR- /Gd /Oy- /MD > /Fa"RelWithDebInfo/" /EHsc /nologo > /Fo"RTAsan_dynamic.i386.dir\RelWithDebInfo\" > /Fp"RTAsan_dynamic.i386.dir\RelWithDebInfo\RTAsan_dynamic.i386.pch" > > (Note how it uses /MD) > > So, basically, ninja is linking as though everything was being > compiled for a static library, when we're actually building a DLL > (from what I can tell, anyway). > > ~Aaron