llvm dev - Apr 2016 - [cfe-dev] [PATCH/DRAFT] Embed metadata into object file

> On Mar 30, 2016, at 6:47 AM, Christian Dietrich via cfe-dev <cfe-dev at
lists.llvm.org> wrote:
> 
> Hi,
> 
> so this is my first contribution to LLVM/clang, so I hope I come close
> to the required coding standards and guidelines.
> 
> First, I will describe the scenario I want to solve: For a few days, the
> clang plugin interface allows to execute the a plugin just before the
> actual main action (e.g., compiling an translation unit). In my case,
> the plugin we're developing will analyze the AST and generate some
> information. This information should be available in the generated
> object file. So, I had to solve the question: how do I smuggle some
> metadata on the module level, from the frontend to the generated object
> file (in an hopefully sane and reusable manner).
> 
> So, the solution I thought about was the following. The gathered
> information is but into a NamedMetadataStringAttr and attached to the
> TranslationUnitDecl. This would be, as far as I can see, the first
> annotation on the TranslationUnit level. Attributes seem to me the best
> solution at that point, since there is already a good infrastructure in
> clang.
> 
>   TranslationUnitDecl
>   | NamedMetadataAttr implicit llvm.extra_section clang.analysis
"Information"
> 
> The clang CodeGen then generates a named metadata node on the
> llvm::Module level:
> 
>   !llvm.extra_sections = {!0}
>   !0 = !{!"clang.analysis", !"Information"}
> 
> The attached patch, then takes all key-value pairs from the named
> llvm.extra_sections named metadata, and appends them as zero-terminated
> strings to the desired section.
Depending on your needs, just using a global with the “section” attribute might
also work for you:
http://llvm.org/docs/LangRef.html#global-variables


-- adrian
> 
> I'm not sure, wheter this is the best solution to the problem, and
> wheter it is general enough. I was also thinking about the possibility
> to attach LLVM passes from the plugin interface to the clang/CodeGen
> backend, but I could not figure out how to do this without breaking all
> the used abstraction.
> 
> I post this to llvm-dev, as well as, to cfe-dev, since both changes,
> although idependend of each other, relate to one another.
> 
> chris
> -- 
> Christian Dietrich, M.Sc. (Wissenschaftlicher Mitarbeiter)
> Lehrstuhl für Informatik 4 (Verteilte Systeme und Betriebssysteme)
> Friedrich-Alexander-Universität Erlangen-Nürnberg
> Martensstr. 1
> 91058 Erlangen
> 
> Tel:    (09131) 85-27280
> Fax:    (09131) 85-28732
> eMail:  christian.dietrich at fau.de
> WWW:    http://www4.cs.fau.de/~dietrich
>
<0001-Attach-LLVM-metadata-with-NamedMetadataStringAttr.patch><0001-Use-llvm.extra_section-to-smuggle-data-into-object-s.patch>_______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

> On Apr 1, 2016, at 5:20 AM, Christian Dietrich via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Adrian Prantl <aprantl at apple.com> writes:
> 
>> Depending on your needs, just using a global with the “section”
>> attribute might also work for you:
>> http://llvm.org/docs/LangRef.html#global-variables
> 
> I was aware of that possibility. But, there are several drawbacks to
> using global variables and the current infrastructure from a clang
> plugin's point of view:
> 
> 1. It feels like a hack. It is not an idiomatic way of transporting
>   information alongside with an translation unit.
> 
> 2. A clang plugin is mostly defined by it's ASTConsumer; there is no
>   direct access to the produced LLVM intermediate representation. I
>   would have to insert AST elements that result in the attributed
>   global variable. [1]
> 
> 3. A more idealistic problem I have with this solution is that we change
>   the actual (semantic) content of the module. But I only want to carry
>   _metadata_ about the module alongside.
> 
> 4. There already is an good metadata infrastructure in LLVM and a very
>   good attribute infrastructure in clang. Why not utilize them?
> 
> So my question is:
> 
>   I want to generate and attach (structured) metadata at every point in
>   life cycle (AST, LLVM module, object file) of a module and retrieve it
>   from the compilation results. How can I do that?
Worded like that, I can see a close analogy with  "Debug Info".

> At the moment, my patches are only a proposal to solve this for a very
> specifc use case (mine): attaching single strings. I think, furthermore,
> that it would also be handy to materialize and retrieve more complex
> metadata types to/from object files. For example, we could serialize
> LLVM metadata trees into separate sections:
> 
>   !llvm.extra_sections = {!0}
>   !0 = !{!"clang.myanalysis", !1, !2}
>   !1 = !{ i32 15, i32 28, i32 142}
>   !2 = !{!"key", i32 10000, !"key2", i32 9999}
> 
> I'm not sure whether this would be useful and reusable for others. So:
> Do you think this would be useful for other developers as well?
> 
> chris
> 
> 
> [1] This is a _seperate_ drawback of clang plugins at the moment: They
>    cannot define an LLVM IR transformation that should be applied when
>    the plugin is active.
You can write clang plugin that are LLVM pass and insert them in the pipeline.

> I think it would be useful for many analyzes
>    to have access both to the AST and to the IR.
This is more fuzzy to me, I don't know enough about clang but I'm not
sure the design allow to keep a link from the IR to the clang AST? (If it is the
case, I'd be curious to see how it works).

Best,

-- 
Mehdi