Hello, I've been working on implementing some basic functionality in order to use the llvm profiling functionality inside of a kernel (the Xen hypervisor). The only functionality I'm interested in is being able to reset the counters, get the size of the data, and dump the data into a memory buffer. I have to admit I haven't been able to find a lot of documentation about how this data is stored inside of the different sections, but with the code in compiler_rt/libs/profile I've been able to craft something that seems to do something sensible. I have however a couple of questions: - The "Values" field in __llvm_profile_data always seems to be NULL. Is this expected? What/why could cause this? - The fields in the NumValueSites array inside of __llvm_profile_data also seem to always be 0. - Since what I'm coding is a decoupled replacement for the profiling runtime inside of compiler_rt, is there anyway that at compile or run time I can fetch the version of the profiling data? I'm mostly worried that at some point llvm will bump the version and change the layout, and then I will have to update my runtime accordingly, but without llvm reporting the version used by the compiler it's going to be very hard to keep backwards compatibility or to detect version bumps. Thanks, Roger.
Adding who I think are the maintainers of the profile related llvm bits, sorry should have done that earlier. On Wed, Oct 25, 2017 at 08:26:18AM +0100, Roger Pau Monné via llvm-dev wrote:> Hello, > > I've been working on implementing some basic functionality in order to > use the llvm profiling functionality inside of a kernel (the Xen > hypervisor). The only functionality I'm interested in is being able to > reset the counters, get the size of the data, and dump the data into a > memory buffer. > > I have to admit I haven't been able to find a lot of documentation > about how this data is stored inside of the different sections, but > with the code in compiler_rt/libs/profile I've been able to craft > something that seems to do something sensible. > > I have however a couple of questions: > > - The "Values" field in __llvm_profile_data always seems to be NULL. > Is this expected? What/why could cause this? > > - The fields in the NumValueSites array inside of __llvm_profile_data > also seem to always be 0. > > - Since what I'm coding is a decoupled replacement for the profiling > runtime inside of compiler_rt, is there anyway that at compile or > run time I can fetch the version of the profiling data? > I'm mostly worried that at some point llvm will bump the version > and change the layout, and then I will have to update my runtime > accordingly, but without llvm reporting the version used by the > compiler it's going to be very hard to keep backwards > compatibility or to detect version bumps. > > Thanks, Roger. > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
+David Li and Vedant Kumar who are active in this area On Wed, Oct 25, 2017 at 2:06 AM, Roger Pau Monné via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Adding who I think are the maintainers of the profile related llvm > bits, sorry should have done that earlier. > > On Wed, Oct 25, 2017 at 08:26:18AM +0100, Roger Pau Monné via llvm-dev > wrote: > > Hello, > > > > I've been working on implementing some basic functionality in order to > > use the llvm profiling functionality inside of a kernel (the Xen > > hypervisor). The only functionality I'm interested in is being able to > > reset the counters, get the size of the data, and dump the data into a > > memory buffer. > > > > I have to admit I haven't been able to find a lot of documentation > > about how this data is stored inside of the different sections, but > > with the code in compiler_rt/libs/profile I've been able to craft > > something that seems to do something sensible. > > > > I have however a couple of questions: > > > > - The "Values" field in __llvm_profile_data always seems to be NULL. > > Is this expected? What/why could cause this? > > > > - The fields in the NumValueSites array inside of __llvm_profile_data > > also seem to always be 0. > > > > - Since what I'm coding is a decoupled replacement for the profiling > > runtime inside of compiler_rt, is there anyway that at compile or > > run time I can fetch the version of the profiling data? > > I'm mostly worried that at some point llvm will bump the version > > and change the layout, and then I will have to update my runtime > > accordingly, but without llvm reporting the version used by the > > compiler it's going to be very hard to keep backwards > > compatibility or to detect version bumps. > > > > Thanks, Roger. > > _______________________________________________ > > LLVM Developers mailing list > > llvm-dev at lists.llvm.org > > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-- Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171025/92b50c98/attachment.html>
On Wed, Oct 25, 2017 at 12:26 AM, Roger Pau Monné via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hello, > > I've been working on implementing some basic functionality in order to > use the llvm profiling functionality inside of a kernel (the Xen > hypervisor). The only functionality I'm interested in is being able to > reset the counters, get the size of the data, and dump the data into a > memory buffer. > > I have to admit I haven't been able to find a lot of documentation > about how this data is stored inside of the different sections, but > with the code in compiler_rt/libs/profile I've been able to craft > something that seems to do something sensible. > > I have however a couple of questions: > > - The "Values" field in __llvm_profile_data always seems to be NULL. > Is this expected? What/why could cause this? >This is for value profiling. For now there are only two kinds: indirect call targets and memcpy/memset size. If the function body does not have any value sites, the field will be null.> > - The fields in the NumValueSites array inside of __llvm_profile_data > also seem to always be 0. >See above.> > - Since what I'm coding is a decoupled replacement for the profiling > runtime inside of compiler_rt, is there anyway that at compile or > run time I can fetch the version of the profiling data? >At compile time, it is the macro: INST_PROF_RAW_VERSION. At runtime, it is the second field of the raw profile header. I'm mostly worried that at some point llvm will bump the version> and change the layout, and then I will have to update my runtime > accordingly, but without llvm reporting the version used by the > compiler it's going to be very hard to keep backwards > compatibility or to detect version bumps. > >yes, the raw profile format can change anytime. We only try to keep backward compatibility for indexed format. At runtime with in-process profile merging, if the source raw profile data's format version is different from the current runtime version, an error will be emitted. David> Thanks, Roger. > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171025/50d10875/attachment-0001.html>
On Wed, Oct 25, 2017 at 09:13:54AM -0700, Xinliang David Li wrote:> On Wed, Oct 25, 2017 at 12:26 AM, Roger Pau Monné via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > > > Hello, > > > > I've been working on implementing some basic functionality in order to > > use the llvm profiling functionality inside of a kernel (the Xen > > hypervisor). The only functionality I'm interested in is being able to > > reset the counters, get the size of the data, and dump the data into a > > memory buffer. > > > > I have to admit I haven't been able to find a lot of documentation > > about how this data is stored inside of the different sections, but > > with the code in compiler_rt/libs/profile I've been able to craft > > something that seems to do something sensible. > > > > I have however a couple of questions: > > > > - The "Values" field in __llvm_profile_data always seems to be NULL. > > Is this expected? What/why could cause this? > > > > This is for value profiling. For now there are only two kinds: indirect > call targets and memcpy/memset size. If the function body does not have any > value sites, the field will be null.You will have to bear with me because my knowledge of compiler internals is very limited. What is exactly a "value site"? Can you give me some code examples that trigger this in C?> > > > - Since what I'm coding is a decoupled replacement for the profiling > > runtime inside of compiler_rt, is there anyway that at compile or > > run time I can fetch the version of the profiling data? > > > > At compile time, it is the macro: INST_PROF_RAW_VERSION. At runtime, it is > the second field of the raw profile header.I'm not able to use INST_PROF_RAW_VERSION at compile time. Are you sure this is exported? If I do: cc -fprofile-instr-generate -fcoverage-mapping -dM -E - < /dev/null I don't see INST_PROF_RAW_VERSION neither any similar defines.> I'm mostly worried that at some point llvm will bump the version > > and change the layout, and then I will have to update my runtime > > accordingly, but without llvm reporting the version used by the > > compiler it's going to be very hard to keep backwards > > compatibility or to detect version bumps. > > > > > yes, the raw profile format can change anytime. We only try to keep > backward compatibility for indexed format. > > At runtime with in-process profile merging, if the source raw profile > data's format version is different from the current runtime version, an > error will be emitted.Keep in mind this is a kernel, so the source is compiled with "-fprofile-instr-generate -fcoverage-mapping", but the profiling runtime in compiler_rt is not linked against the kernel. I would like to have a reliable way that I could use to detect version bumps in the internal coverage data, so that I can implement the required changes in my in-kernel coverage code. I have a series ready for Xen in order to implement this, I will send the patch with the in-kernel profiling implementation to this list for review. Thanks, Roger.