Hello, LLVM developers I would like to ask about the changes in the "nonpublic" AddressSanitizer flags in llvm 3.5. Prior to 3.5 (namely, on 3.4.2) there was a compile-time flag to control the shadow offset. For example, one could do the following to force the large offset: -mllvm -asan-mapping-offset-log=44 Now, it has been removed and on x86_64 linux the smaller offset is always used. Our use case for the large offset was using NVIDIA OpenGL/CUDA drivers which apparently mmap certain memory ranges and fail when the requested address does not match the return value of mmap. Using a larger offset allows to work around the problem and use both the binary driver and ASan which is very useful. Could someone explain the reason behind forcing a small shadow offset on 64-bit linux and add a public documented option to control it? It is also interesting that OS X and FreeBSD use larger offset. Meanwhile, those who are interested in using ASan and NVIDIA, can edit "lib/Transforms/Instrumentation/AddressSanitizer.cpp" and replace "Mapping.Offset = kSmallX86_64ShadowOffset" with "Mapping.Offset kDefaultShadowOffset64". -- Regards, Alexander
On Thu, Sep 25, 2014 at 2:01 AM, Alexander Tarasikov < alexander.tarasikov at gmail.com> wrote:> Hello, LLVM developers > > I would like to ask about the changes in the "nonpublic" > AddressSanitizer flags in llvm 3.5. > > Prior to 3.5 (namely, on 3.4.2) there was a compile-time flag to > control the shadow offset. For example, one could do the following to > force the large offset: > -mllvm -asan-mapping-offset-log=44 > > Now, it has been removed and on x86_64 linux the smaller offset is always > used. > > Our use case for the large offset was using NVIDIA OpenGL/CUDA drivers > which apparently mmap certain memory ranges and fail when the > requested address does not match the return value of mmap. Using a > larger offset allows to work around the problem and use both the > binary driver and ASan which is very useful. > > Could someone explain the reason behind forcing a small shadow offset > on 64-bit linuxThe reason is performance and code size -- both get better by 3%-5% with the small offset.> and add a public documented option to control it?Oh my... This is not as trivial as it sounds. We removed this option because (as we thought) no one was using it and because it was actually hard to maintain. This is a compile-time option, which uses a different ABI, and objects built with different flags are incompatible. GCC variant of asan does not support it either and GCC folks explicitly said that they are not going to. I don't have a good solution. Others? In the meantime I would strongly suggest to try fixing the issue on the NVIDIA side. It> is also interesting that OS X and FreeBSD use larger offset. >Right. The small offset didn't work there for some reasons...> > Meanwhile, those who are interested in using ASan and NVIDIA, can edit > "lib/Transforms/Instrumentation/AddressSanitizer.cpp" and replace > "Mapping.Offset = kSmallX86_64ShadowOffset" with "Mapping.Offset > kDefaultShadowOffset64". >you will also need to change the run-time library. --kcc> > -- > Regards, Alexander > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140925/1ae00e28/attachment.html>
Hi Alexander, Kostya, I'm actually one of that flag's clients, but not for long. It's actually handy when bringing up ASan in a new environment, but the shadow offset is one of the first things that will be fixed and the flexibility of having a flag might give people the idea that it's something that's easy to change by simply changing the flag. It might also mess up performance. We're using a large-ish offset when testing (though not as large as 2^44) and we get an extra instruction + reg used because of that (mov shadow_offset, reg + or shadow_offset_reg, shadow_index_reg). It's not ideal, but we're still actively porting this. Eventually, changing the shadow offset is not that hard. All it should take is change the appropriate version in the ASan pass and asan_mapping.h. Since the flag is of very limited use (comes in handy when starting to port ASan, at most), I don't think it's a very big value to have it, either. Alexander: Can you change your compiler so the ASan pass knows what offset you want to use for your use-case (by adding a triple to getShadowMapping() in AddressSanitizer.cpp, for example)? Regards, Filipe F On Thu, Sep 25, 2014 at 6:33 PM, Kostya Serebryany <kcc at google.com> wrote:> > > On Thu, Sep 25, 2014 at 2:01 AM, Alexander Tarasikov < > alexander.tarasikov at gmail.com> wrote: > >> Hello, LLVM developers >> >> I would like to ask about the changes in the "nonpublic" >> AddressSanitizer flags in llvm 3.5. >> >> Prior to 3.5 (namely, on 3.4.2) there was a compile-time flag to >> control the shadow offset. For example, one could do the following to >> force the large offset: >> -mllvm -asan-mapping-offset-log=44 >> >> Now, it has been removed and on x86_64 linux the smaller offset is always >> used. >> >> Our use case for the large offset was using NVIDIA OpenGL/CUDA drivers >> which apparently mmap certain memory ranges and fail when the >> requested address does not match the return value of mmap. Using a >> larger offset allows to work around the problem and use both the >> binary driver and ASan which is very useful. >> >> Could someone explain the reason behind forcing a small shadow offset >> on 64-bit linux > > > The reason is performance and code size -- both get better by 3%-5% with > the small offset. > > >> and add a public documented option to control it? > > > Oh my... This is not as trivial as it sounds. > We removed this option because (as we thought) no one was using it and > because it was actually hard to maintain. > This is a compile-time option, which uses a different ABI, and objects > built with different flags are incompatible. > GCC variant of asan does not support it either and GCC folks explicitly > said that they are not going to. > > I don't have a good solution. Others? > In the meantime I would strongly suggest to try fixing the issue on the > NVIDIA side. > > > It >> is also interesting that OS X and FreeBSD use larger offset. >> > > > Right. The small offset didn't work there for some reasons... > > >> >> Meanwhile, those who are interested in using ASan and NVIDIA, can edit >> "lib/Transforms/Instrumentation/AddressSanitizer.cpp" and replace >> "Mapping.Offset = kSmallX86_64ShadowOffset" with "Mapping.Offset >> kDefaultShadowOffset64". >> > you will also need to change the run-time library. > > --kcc > >> >> -- >> Regards, Alexander >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev >> > > > _______________________________________________ > LLVM Developers mailing list > LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140925/abc98bf2/attachment.html>
On 09/26/2014 05:33 AM, Kostya Serebryany wrote:> GCC variant of asan does not support it either and GCC folks explicitly > said that they are not going to.FYI we'll probably have it for KAsan to simplify experiments with kernel memory layouts.
On Sep 26, 2014 5:33 AM, "Kostya Serebryany" <kcc at google.com> wrote:>Hi, Kostya. Thanks for replying> > > Oh my... This is not as trivial as it sounds. > We removed this option because (as we thought) no one was using it andbecause it was actually hard to maintain.> This is a compile-time option, which uses a different ABI, and objectsbuilt with different flags are incompatible.> GCC variant of asan does not support it either and GCC folks explicitlysaid that they are not going to. Thank for this. Now it makes sense. I did not actually think of it myself.> > I don't have a good solution. Others? > In the meantime I would strongly suggest to try fixing the issue on theNVIDIA side. Unfortunately I am not working at NVIDIA so I cannot change that. But at work we're using Nix package manager from NixOS so patching llvm, clang and rebuilding all the packages is trivial though a bit time consuming.> > >> It >> is also interesting that OS X and FreeBSD use larger offset. > > > > Right. The small offset didn't work there for some reasons... > >> >> >> Meanwhile, those who are interested in using ASan and NVIDIA, can edit >> "lib/Transforms/Instrumentation/AddressSanitizer.cpp" and replace >> "Mapping.Offset = kSmallX86_64ShadowOffset" with "Mapping.Offset >> kDefaultShadowOffset64". > > you will also need to change the run-time library.Sorry, do you mean libc or what runtime? I will take a deeper look into llvm and clang source to make sure I didn't break anything while patching our local tree. Basically we're now building our project with ASan and the rest of system is not instrumented, so I think we don't have to worry about using different offset sizes until we start instrumenting dependencies, right?> > --kcc >> >> >> -- >> Regards, Alexander >> _______________________________________________ >> LLVM Developers mailing list >> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140926/441e600c/attachment.html>