Lang Hames via llvm-dev
2020-Aug-10 06:26 UTC
[llvm-dev] ORC JIT Weekly #19 -- Relocatable object level mocking with llvm-jitlink.
Hi All, There was no update last week -- I'm still trying to get back into a regular schedule. Open-source changes since the last update were: (1) Some bug fixes for JITLink MachO / arm64 support (PAGE21/PAGEOFF12 now handle addends correctly). (2) llvm-jitlink now supports loading archives as well as relocatable objects. (3) llvm-jitlink now supports basic object-file level mocking and testing. That last one is unusual so I want to expand on it. The llvm-jitlink tool acquired a couple of new options to support this feature: "--phony-externals" which resolves all otherwise unresolved symbols to null instead of erroring, and "-harness" which allows a set of objects to be designated as a test harness, with the regular object files implicitly treated as objects to be tested. If the -harness option is passed llvm-jitlink installs a JITLink plugin to make some modifications to the usual linker rules: -- All symbols referenced by the harness object(s) are promoted to default visibility so that the harness can directly access static symbols inside the test object(s) -- All symbols defined by the harness object(s) override (rather than clashing with) symbols defined in the test object(s). With these modifications in place we can selectively test functions in an object file by mocking their callees. For example, suppose we have an object file, test_code.o, compiled from the following C source (which we do not have access to): void irrelevant_function() { irrelevant_external(); } int function_to_mock(int X) { return /* some function of X */; } static void function_to_test() { ... int Y = function_to_mock(); printf("Y is %i\n", Y); } We would like to know how function_to_test behaves when we change the behavior of function_to_mock. We can do this by writing a test harness: void function_to_test(); int function_to_mock(int X) { printf("used mock utility function\n"); return 42; } int main(int argc, char *argv[]) { function_to_test(): return 0; } This harness code could not be linked against test_code.o under normal circumstances: function_to_test is static and will fail to resolve from the harness code, function_to_mock is a duplicate definition, and irrelevant_external is undefined. Our new llvm-jitlink options take care of these issues however, and running the test looks like: % clang -c -o test_code_harness.o test_code_harness.c % llvm-jitlink -phony-externals test_code.o -harness test_code_harness.o used mock utility function Y is 42 This utility may be of interest to people who want to perform some very late testing on build products to verify that compiled code behaves as expected. On basic C test cases it works reasonably well. C++ test cases are another matter: Any code involving classes tends to have a lot of non-trivial surface area (e.g. vtables) that would require great care to mock. Perhaps we need a new tool to tackle that problem. In the meantime, if nothing else this provides a neat example of how JITLink plugins can be used to modify the default linker behavior. Regards, Lang. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200809/7f5ea64c/attachment.html>
Stefan Gränitz via llvm-dev
2020-Aug-16 20:32 UTC
[llvm-dev] ORC JIT Weekly #19 -- Relocatable object level mocking with llvm-jitlink.
Hi Lang Thanks for explaining the new mocking and testing feature in the llvm-jitlink tool in such great detail! Indeed, it sounds like an interesting by-product of your jitlink efforts. "a lot of non-trivial surface area" is a very polite description of the C++ symbols disaster :) Best Stefan On 10/08/2020 08:26, Lang Hames wrote:> Hi All, > > There was no update last week -- I'm still trying to get back into a > regular schedule. > > Open-source changes since the last update were: > > (1) Some bug fixes for JITLink MachO / arm64 support > (PAGE21/PAGEOFF12 now handle addends correctly). > (2) llvm-jitlink now supports loading archives as well as relocatable > objects. > (3) llvm-jitlink now supports basic object-file level mocking and testing. > > That last one is unusual so I want to expand on it. The llvm-jitlink > tool acquired a couple of new options to support this feature: > "--phony-externals" which resolves all otherwise unresolved symbols to > null instead of erroring, and "-harness" which allows a set of objects > to be designated as a test harness, with the regular object files > implicitly treated as objects to be tested. If the -harness option is > passed llvm-jitlink installs a JITLink plugin to make some > modifications to the usual linker rules: > > -- All symbols referenced by the harness object(s) are promoted to > default visibility so that the harness can directly access static > symbols inside the test object(s) > -- All symbols defined by the harness object(s) override (rather than > clashing with) symbols defined in the test object(s). > > With these modifications in place we can selectively test functions in > an object file by mocking their callees. For example, suppose we have > an object file, test_code.o, compiled from the following C source > (which we do not have access to): > > void irrelevant_function() { irrelevant_external(); } > > int function_to_mock(int X) { > return /* some function of X */; > } > > static void function_to_test() { > ... > int Y = function_to_mock(); > printf("Y is %i\n", Y); > } > > We would like to know how function_to_test behaves when we change the > behavior of function_to_mock. We can do this by writing a test harness: > > void function_to_test(); > > int function_to_mock(int X) { > printf("used mock utility function\n"); > return 42; > } > > int main(int argc, char *argv[]) { > function_to_test(): > return 0; > } > > This harness code could not be linked against test_code.o under normal > circumstances: function_to_test is static and will fail to resolve > from the harness code, function_to_mock is a duplicate definition, and > irrelevant_external is undefined. Our new llvm-jitlink options take > care of these issues however, and running the test looks like: > > % clang -c -o test_code_harness.o test_code_harness.c > % llvm-jitlink -phony-externals test_code.o -harness test_code_harness.o > used mock utility function > Y is 42 > > This utility may be of interest to people who want to perform some > very late testing on build products to verify that compiled code > behaves as expected. On basic C test cases it works reasonably well. > C++ test cases are another matter: Any code involving classes tends to > have a lot of non-trivial surface area (e.g. vtables) that would > require great care to mock. Perhaps we need a new tool to tackle that > problem. > > In the meantime, if nothing else this provides a neat example of how > JITLink plugins can be used to modify the default linker behavior. > > Regards, > Lang.-- https://flowcrypt.com/pub/stefan.graenitz at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200816/2617e70b/attachment.html>
Lang Hames via llvm-dev
2020-Aug-17 04:27 UTC
[llvm-dev] ORC JIT Weekly #19 -- Relocatable object level mocking with llvm-jitlink.
Hi Stefan, "a lot of non-trivial surface area" is a very polite description of the C++> symbols disaster :)Indeed. I think tooling may make this tractable: E.g. if debug info is available then perhaps we can make a tool to synthesize ASTs for classes that the user wants to mock. I haven't had a chance to think through the ideas in detail yet though. -- Lang. On Sun, Aug 16, 2020 at 1:32 PM Stefan Gränitz <stefan.graenitz at gmail.com> wrote:> Hi Lang > > Thanks for explaining the new mocking and testing feature in the > llvm-jitlink tool in such great detail! Indeed, it sounds like an > interesting by-product of your jitlink efforts. > > "a lot of non-trivial surface area" is a very polite description of the > C++ symbols disaster :) > > Best > Stefan > > On 10/08/2020 08:26, Lang Hames wrote: > > Hi All, > > There was no update last week -- I'm still trying to get back into a > regular schedule. > > Open-source changes since the last update were: > > (1) Some bug fixes for JITLink MachO / arm64 support (PAGE21/PAGEOFF12 now > handle addends correctly). > (2) llvm-jitlink now supports loading archives as well as relocatable > objects. > (3) llvm-jitlink now supports basic object-file level mocking and testing. > > That last one is unusual so I want to expand on it. The llvm-jitlink tool > acquired a couple of new options to support this feature: > "--phony-externals" which resolves all otherwise unresolved symbols to null > instead of erroring, and "-harness" which allows a set of objects to be > designated as a test harness, with the regular object files implicitly > treated as objects to be tested. If the -harness option is passed > llvm-jitlink installs a JITLink plugin to make some modifications to the > usual linker rules: > > -- All symbols referenced by the harness object(s) are promoted to default > visibility so that the harness can directly access static symbols inside > the test object(s) > -- All symbols defined by the harness object(s) override (rather than > clashing with) symbols defined in the test object(s). > > With these modifications in place we can selectively test functions in an > object file by mocking their callees. For example, suppose we have an > object file, test_code.o, compiled from the following C source (which we do > not have access to): > > void irrelevant_function() { irrelevant_external(); } > > int function_to_mock(int X) { > return /* some function of X */; > } > > static void function_to_test() { > ... > int Y = function_to_mock(); > printf("Y is %i\n", Y); > } > > We would like to know how function_to_test behaves when we change the > behavior of function_to_mock. We can do this by writing a test harness: > > void function_to_test(); > > int function_to_mock(int X) { > printf("used mock utility function\n"); > return 42; > } > > int main(int argc, char *argv[]) { > function_to_test(): > return 0; > } > > This harness code could not be linked against test_code.o under normal > circumstances: function_to_test is static and will fail to resolve from the > harness code, function_to_mock is a duplicate definition, and > irrelevant_external is undefined. Our new llvm-jitlink options take care of > these issues however, and running the test looks like: > > % clang -c -o test_code_harness.o test_code_harness.c > % llvm-jitlink -phony-externals test_code.o -harness test_code_harness.o > used mock utility function > Y is 42 > > This utility may be of interest to people who want to perform some very > late testing on build products to verify that compiled code behaves as > expected. On basic C test cases it works reasonably well. C++ test cases > are another matter: Any code involving classes tends to have a lot of > non-trivial surface area (e.g. vtables) that would require great care to > mock. Perhaps we need a new tool to tackle that problem. > > In the meantime, if nothing else this provides a neat example of how > JITLink plugins can be used to modify the default linker behavior. > > Regards, > Lang. > > -- https://flowcrypt.com/pub/stefan.graenitz at gmail.com > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200816/0ef9919d/attachment-0001.html>