Jessica Paquette via llvm-dev
2019-Jun-13 23:30 UTC
[llvm-dev] [RFC] Remarks-based code size analysis tool
Hi everyone, I frequently find that I want to quickly find out which functions have changed the most in size when I compile a program with one version of the compiler and another version of the compiler. Optimization remarks provide deeper insight into what the compiler does to individual functions during the compilation process. A tool based off of remarks would give us the means to say *exactly* what a pass did to a function during the compilation process. We also have remarks from the asm-printer which tell us the number of instructions in a function at any given time. It’s now possible to access remarks through object files. So, I thought it would be an interesting experiment to write a C++ tool using the new remarks API to write a simple size tool. A size tool using remarks would be able to tell us about the following things: 1) Which functions grew the most in size between one compilation and another compilation 2) Which functions were/were not inlined between compilations (and similar insights into other passes) 3) Which compiler passes caused the largest increases in each function I have a prototype for a size tool which accomplishes (1) using asm-printer remarks. (2) and (3) can be accomplished by searching for remarks corresponding to interesting passes and by searching for size-info remarks respectively. For now, I just decided to implement the minimum set of features for a useful tool. The patch is available here: https://reviews.llvm.org/D63306 <https://reviews.llvm.org/D63306> The tool in the patch works like this: $ ./sizediff old_version_of_program new_version_of_program And it produces output like this: function_name old_instruction_count new_instruction_count delta This output can tell me the following things: - Which functions were not present in old_version_of_program (they have a size of 0) - Which functions were not present in new_version_of_program (same idea) - Which functions grew the most - Which functions shrunk the most For a concrete example of the output, here’s some output I got by compiling the stepanov_container benchmark for AArch64 with -Os and -O3 respectively: https://hastebin.com/hucexocuxi.md <https://hastebin.com/hucexocuxi.md> If anyone is interested, feel free to try it out/review/etc. I think that a tool like this would be quite useful. :) - Jessica -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190613/cc3f2702/attachment.html>
Sjoerd Meijer via llvm-dev
2019-Jun-14 06:00 UTC
[llvm-dev] [RFC] Remarks-based code size analysis tool
Hi Jessica, We often have exactly the same problem:> I frequently find that I want to quickly find out which functions have changed the most in size when I compile a program with one version of the compiler and another version of the compiler.Your tool looks very interesting and useful. I think we are very interested. In fact, I am looking at code-size problems at the moment, and will give it a try. I think we can help with the review too. FWIW, when I investigate code-size problems, I usually start looking at the final linked image (for context, that's mostly baremetal images). Unused sections and functions will be removed by the linker, which e.g. avoids unnecessary analysing them which may happen when you only look at object files. I use a downstream tool for binaries that list the code-size per function. When I generate such a report for different images, I can easily diff them and see where codesize difference are the biggest. I then isolate a function, compile it with debug and print-after-all, so that I can do a diff again where changes are introduced. And when image sizes are small, and library inclusion is thus relatively important, the linker in verbose mode telling me why and which objects and functions are included is extremely useful too. Cheers, Sjoerd. ________________________________ From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Jessica Paquette via llvm-dev <llvm-dev at lists.llvm.org> Sent: 14 June 2019 00:30 To: Jessica Paquette via llvm-dev Subject: [llvm-dev] [RFC] Remarks-based code size analysis tool Hi everyone, I frequently find that I want to quickly find out which functions have changed the most in size when I compile a program with one version of the compiler and another version of the compiler. Optimization remarks provide deeper insight into what the compiler does to individual functions during the compilation process. A tool based off of remarks would give us the means to say *exactly* what a pass did to a function during the compilation process. We also have remarks from the asm-printer which tell us the number of instructions in a function at any given time. It’s now possible to access remarks through object files. So, I thought it would be an interesting experiment to write a C++ tool using the new remarks API to write a simple size tool. A size tool using remarks would be able to tell us about the following things: 1) Which functions grew the most in size between one compilation and another compilation 2) Which functions were/were not inlined between compilations (and similar insights into other passes) 3) Which compiler passes caused the largest increases in each function I have a prototype for a size tool which accomplishes (1) using asm-printer remarks. (2) and (3) can be accomplished by searching for remarks corresponding to interesting passes and by searching for size-info remarks respectively. For now, I just decided to implement the minimum set of features for a useful tool. The patch is available here: https://reviews.llvm.org/D63306 The tool in the patch works like this: $ ./sizediff old_version_of_program new_version_of_program And it produces output like this: function_name old_instruction_count new_instruction_count delta This output can tell me the following things: - Which functions were not present in old_version_of_program (they have a size of 0) - Which functions were not present in new_version_of_program (same idea) - Which functions grew the most - Which functions shrunk the most For a concrete example of the output, here’s some output I got by compiling the stepanov_container benchmark for AArch64 with -Os and -O3 respectively: https://hastebin.com/hucexocuxi.md If anyone is interested, feel free to try it out/review/etc. I think that a tool like this would be quite useful. :) - Jessica -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190614/39038dd7/attachment.html>
Jessica Paquette via llvm-dev
2019-Jun-14 17:20 UTC
[llvm-dev] [RFC] Remarks-based code size analysis tool
Hi Sjoerd,> Your tool looks very interesting and useful. I think we are very interested. In fact, I am looking at code-size problems at the moment, and will give it a try. I think we can help with the review too.That would be great!> FWIW, when I investigate code-size problems, I usually start looking at the final linked image (for context, that's mostly baremetal images). Unused sections and functions will be removed by the linker, which e.g. avoids unnecessary analysing them which may happen when you only look at object files. I use a downstream tool for binaries that list the code-size per function. When I generate such a report for different images, I can easily diff them and see where codesize difference are the biggest. I then isolate a function, compile it with debug and print-after-all, so that I can do a diff again where changes are introduced. And when image sizes are small, and library inclusion is thus relatively important, the linker in verbose mode telling me why and which objects and functions are included is extremely useful too.I have a similar tool! :) I think that you’re right wrt looking at functions that actually appear in the final linked image. In my experience, this is the common case. (My personal script just looks at the output of llvm-nm) I think that combining the two approaches could be interesting. By using remarks in conjunction with the symbols present in the final linked image, it should be possible to signify to the user if a function was eliminated by the compiler (and which pass did it) or the linker. I think the default output should be something like this: - Only show functions that appear in the binary - If a function appears in one image, but not the other, check if there is a remark stating that its instruction count went to 0. If there is, state that it was eliminated by the compiler. - … If there isn’t, state that it was eliminated by the linker I think that for a first pass, just to get the functionality going, I’d like to just focus on the remarks-based diffing. After that, I think it would be a natural step to add information about the actual binary to the tool as well. - Jessica> On Jun 13, 2019, at 11:00 pm, Sjoerd Meijer <Sjoerd.Meijer at arm.com> wrote: > > Hi Jessica, > > We often have exactly the same problem: > > > I frequently find that I want to quickly find out which functions have changed the most in size when I compile a program with one version of the compiler and another version of the compiler. > > Your tool looks very interesting and useful. I think we are very interested. In fact, I am looking at code-size problems at the moment, and will give it a try. I think we can help with the review too. > > FWIW, when I investigate code-size problems, I usually start looking at the final linked image (for context, that's mostly baremetal images). Unused sections and functions will be removed by the linker, which e.g. avoids unnecessary analysing them which may happen when you only look at object files. I use a downstream tool for binaries that list the code-size per function. When I generate such a report for different images, I can easily diff them and see where codesize difference are the biggest. I then isolate a function, compile it with debug and print-after-all, so that I can do a diff again where changes are introduced. And when image sizes are small, and library inclusion is thus relatively important, the linker in verbose mode telling me why and which objects and functions are included is extremely useful too. > > Cheers, > Sjoerd. > > > > From: llvm-dev <llvm-dev-bounces at lists.llvm.org> on behalf of Jessica Paquette via llvm-dev <llvm-dev at lists.llvm.org> > Sent: 14 June 2019 00:30 > To: Jessica Paquette via llvm-dev > Subject: [llvm-dev] [RFC] Remarks-based code size analysis tool > > Hi everyone, > > I frequently find that I want to quickly find out which functions have changed the most in size when I compile a program with one version of the compiler and another version of the compiler. > > Optimization remarks provide deeper insight into what the compiler does to individual functions during the compilation process. A tool based off of remarks would give us the means to say *exactly* what a pass did to a function during the compilation process. We also have remarks from the asm-printer which tell us the number of instructions in a function at any given time. > > It’s now possible to access remarks through object files. So, I thought it would be an interesting experiment to write a C++ tool using the new remarks API to write a simple size tool. A size tool using remarks would be able to tell us about the following things: > > 1) Which functions grew the most in size between one compilation and another compilation > 2) Which functions were/were not inlined between compilations (and similar insights into other passes) > 3) Which compiler passes caused the largest increases in each function > > I have a prototype for a size tool which accomplishes (1) using asm-printer remarks. (2) and (3) can be accomplished by searching for remarks corresponding to interesting passes and by searching for size-info remarks respectively. For now, I just decided to implement the minimum set of features for a useful tool. The patch is available here: > > https://reviews.llvm.org/D63306 <https://reviews.llvm.org/D63306> > > The tool in the patch works like this: > > $ ./sizediff old_version_of_program new_version_of_program > > And it produces output like this: > > function_name old_instruction_count new_instruction_count delta > > This output can tell me the following things: > > - Which functions were not present in old_version_of_program (they have a size of 0) > - Which functions were not present in new_version_of_program (same idea) > - Which functions grew the most > - Which functions shrunk the most > > For a concrete example of the output, here’s some output I got by compiling the stepanov_container benchmark for AArch64 with -Os and -O3 respectively: > > https://hastebin.com/hucexocuxi.md <https://hastebin.com/hucexocuxi.md> > > If anyone is interested, feel free to try it out/review/etc. I think that a tool like this would be quite useful. :) > > - Jessica > IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190614/4fe65957/attachment.html>