llvm dev - Mar 2017 - [cfe-dev] proposal - pragma section directive in clang

+llvm-dev properly this time.

On Fri, 10 Mar 2017 at 09:42 James Molloy <james at jamesmolloy.co.uk>
wrote:
> Hi Reid, all,
>
> +llvm-dev as this RFC involves changes in Clang and LLVM.
>
> This RFC has stagnated and I think that's partially because the
proposal
> isn't particularly elegant and is light on details. We've been
having a
> rethink and have a slightly different implementation to propose that we (I)
> hope will be nicer.
>
> ** Rationale (for llvm-dev) **
>
> The goal of this proposed feature is to provide a migration path toward
> Clang for developers in the automotive domain. As Javed has mentioned,
> AUTOSAR, an automotive standard, mandates the use of a #pragma in header
> files to determine in which sections initialized and uninitialized data get
> put.
>
> This feature is implemented in our legacy ARM Compiler 5 toolchain and
> we're also aware of GCC forks used across the automotive space that
have
> this feature implemented compatible with the ARM Compiler 5 implementation.
>
> The documentation is here:
>
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472m/chr1359124985290.html
>
> ** Proposed syntax and (vague) semantics **
>
> As this is a new pragma for Clang and isn't ARM-specific, we've
invented a
> less ARM-specific syntax. Bikeshedding is expected and welcome.
>
>   #pragma clang section bss(".mybss")
rodata(".myrodata") data(".mydata")
> text(".mytext")
>
> The pragma applies to all global variable and function declarations from
> the pragma to the end of the translation unit. The pragma should ideally be
> pushable and poppable, but that is outside the scope of this RFC. The
> pragma also applies to static local declarations within functions.
>
> All global variables and functions affected by this pragma have their
> default ELF section destinations changed. Globals with
> __attribute__((section())) are not affected (the attribute trumps the
> pragma).
>
> This pragma is only defined to work sensibly for ELF targets.
>
> ** Proposed implementation **
>
> There are, I believe, three possible implementation stategies:
>   1) Clang internally sets the "section" on all globals it
creates. No
> changes in LLVM.
>   2) Clang sets some module-level attribute describing the default section
> names. The LLVM backend takes this into account when deciding the section
> name for a global when it emits it (AsmPrinter).
>   3) Clang sets the default section names as attributes on all globals.
> The LLVM backend takes this into account when deciding the section name for
> a global when it emits it (AsmPrinter).
>
> (1) and (3) work with LTO. (2) interacts badly with LTO so is discounted.
>
> (1) requires Clang to perform the decision into exactly what section a
> global will go, which is the wrong place for several previously mentioned
> reasons (midend optimizations could promote .data -> .bss, clang
currently
> doesn't have the mechanics to test if an initializer is zero or not,
LLVM
> does).
>
> (2) and (3) have the advantage that the section type does not need to be
> inferred by LLVM from its name. This means we don't need the horrible
> string matching (m/^.bss./ -> BSS, m/^.data./ -> Data, etc) in the
> AsmPrinter - users can specify whatever names they like for bss sections
> without confusing the compiler.
>
> Our previous proposal was (1). We are now proposing (3).
>
> For (3), I think there are three distinct steps, none of which are
> *particularly* invasive:
>
>   a) Allow arbitrary attributes on GlobalVariables. Currently these are
> restricted to Functions only; I believe mainly because noone had a usecase
> for attributes on variables.
>   b) Teach the clang frontend about the new pragma and CodeGen to add the
> attributes to globals
>   c) Teach AsmPrinter to inspect these attributes if they exist and take
> them into account when choosing sections.
>
> All comments more than welcome,
>
> James
>
> On Fri, 3 Mar 2017 at 09:09 James Molloy <james at jamesmolloy.co.uk>
wrote:
>
> Hi Reid,
>
> Thanks for your comments. In truth, we don't have a particular
requirement
> for behaviour under LTO or pushability/poppability, so we can define this
> in terms of what's best for Clang.
>
> As context is always useful, the goal of this proposed feature is to
> provide a migration path toward Clang for developers in the automotive
> domain. As Javed has mentioned, AUTOSAR which is an automotive standard,
> mandates the use of a #pragma in header files to determine in which
> sections initialized and uninitialized data get put.
>
> This feature is implemented in our legacy ARM Compiler 5 toolchain and
> we're also aware of GCC forks used across the automotive space that
have
> this feature implemented compatible with the ARM Compiler 5 implementation.
>
> The documentation is here:
>
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472m/chr1359124985290.html
>
> We do not aim for or anticipate syntax-level compatibility; merely the
> ability to do something similar in Clang. In particular Clang can't
support
> the "section_type" specifier without significant rewrite, as LLVM
decides
> the section type (NOBITS/PROGBITS) by textual matching on the section name
> (gross, but the fix is well beyond our scope).
>
> Given that documentation, you can also see why the Microsoft compatibility
> option *almost* met our requirements - all except the behaviour in this
> case:
>
> #pragma bss_seg('.bss.mybss')
> int i; // Microsoft extension will put i in .bss.mybss
> int j = 0; // Microsoft extension will put j in .data, whereas we really
> need it in .bss.mybss
>
> So to specifically answer your questions:
>
> In ARM Compiler 5, all pragmas are pushable and poppable. Clang doesn't
> have this feature generally yet, but when/if it does, I don't see why
this
> pragma shouldn't be affected. So yes, we should consider it pushable
and
> poppable.
>
> I think the only reasonable behaviour under LTO must be that the two TUs
> (may) have different bss and data sections. Anything else would be very
> strange behaviour from the user's perspective.
>
> Your example of a static global migrating from .data into .bss after
> optimization is interesting. With our proposal to explicitly name sections
> in IR, this optimization would be inhibited. I personally think that's
fine
> :)
>
> Cheers,
>
> James
>
> On Thu, 2 Mar 2017 at 21:42 Reid Kleckner via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
> Would these pragmas be translation-unit global or could they be pushed and
> popped like the MSVC pragmas?
>
> If two TUs are combined through LTO, can the two TUs have different bss
> and data sections?
>
> If the answer to both is "no", then I think we should use a
module flag
> instead of manually setting the section from the frontend.
>
> Here is a C++ example where, after optimization, a global may end up in
> .bss instead of .data:
> // c++
> static int f() { return 0; }
> static int x = f();
> int *g() { return &x; }
>
> After global opt we get this:
> @_ZL1x = internal global i32 0, align 4
>
> Normally LLVM will put this in .bss or use .lcomm for it. Your proposal
> will put it in the user's data section instead of their bss section. If
> that's important, we should use a module flag for this.
>
> On Thu, Mar 2, 2017 at 11:35 AM, Javed Absar via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
> Hi all:
>
>
>
> We would like to propose a clang pragma directive to allow specialized
> section names.
> The semantics of it could be as follows. The pragma section name is
> declared in global
> scope. All global variables and functions get assigned to the
> corresponding specialized
> section name if one is present. With this feature, the following code:
>
> // foo.c
> #pragma bss_section(".bss.alpha")
> #pragma data_section(".data.beta")
> #pragma code_section(".code.gamma")
> #pragma const_section(".const.delta")
> int a;
> int b=2;
> const int d = 5;
> int c(){
>   return d;
> }
>
> ..will emit llvm-ir as:
>
> target triple = "armv7-arm-none-eabi"
> @a = global i32 0, section ".bss.alpha", align 4
> @b = global i32 2, section ".data.beta", align 4
> @d = constant i32 5, section ".const.delta", align 4
>
> ; Function Attrs: noinline nounwind
> define i32 @c() #0 section ".code.gamma" {
> entry:
>   ret i32 5
> }
>
> This pragma will be very useful for embedded code which
> need to control where the different sections are placed in memory.
>
> Microsoft -fms-extension provides similar feature, but our proposal is for
> a
> general use.  Attributes are an alternative that is also currently
> available,
> but attributes are applicable only to specific declarations and not entire
> file. Many real embedded users prefer the pragma option.
> This will be a welcome enabler for them. Also, AUTOSAR, which is an
>
> automotive standard mandates use of a #pragma solution over an attribute
> one
>
> Looking forward to comments and suggestions.
> Best Regards
>
> Javed
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170310/904f0e35/attachment-0001.html>

Hi All,

We supply both clang and gcc builds in our Crossworks for ARM product 
that have command line options to name the default section names used by 
the compiler for example

-mtext=.text.libdebugio -mdata=.data.libdebugio -mbss=.bss.libdebugio 
-mrodata=.rodata.libdebugio

Without this placing code/data into sections (in the presence of 
-ffunction-sections -fdata-sections) is pretty much impossible with 
binutils objcopy/ld.

Is it worth considering this as an alternative/addition to #pragmas? 
Patch files are available here

http://www.rowley.co.uk/crossworks/gpl_sources.htm

if anyone is interested.

Regards
Michael
> +llvm-dev properly this time.
>
> On Fri, 10 Mar 2017 at 09:42 James Molloy <james at jamesmolloy.co.uk 
> <mailto:james at jamesmolloy.co.uk>> wrote:
>
>     Hi Reid, all,
>
>     +llvm-dev as this RFC involves changes in Clang and LLVM.
>
>     This RFC has stagnated and I think that's partially because the
>     proposal isn't particularly elegant and is light on details.
We've
>     been having a rethink and have a slightly different implementation
>     to propose that we (I) hope will be nicer.
>
>     ** Rationale (for llvm-dev) **
>
>     The goal of this proposed feature is to provide a migration path
>     toward Clang for developers in the automotive domain. As Javed has
>     mentioned, AUTOSAR, an automotive standard, mandates the use of a
>     #pragma in header files to determine in which sections initialized
>     and uninitialized data get put.
>
>     This feature is implemented in our legacy ARM Compiler 5 toolchain
>     and we're also aware of GCC forks used across the automotive space
>     that have this feature implemented compatible with the ARM
>     Compiler 5 implementation.
>
>     The documentation is here:
>    
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472m/chr1359124985290.html
>
>     ** Proposed syntax and (vague) semantics **
>
>     As this is a new pragma for Clang and isn't ARM-specific, we've
>     invented a less ARM-specific syntax. Bikeshedding is expected and
>     welcome.
>
>       #pragma clang section bss(".mybss")
rodata(".myrodata")
>     data(".mydata") text(".mytext")
>
>     The pragma applies to all global variable and function
>     declarations from the pragma to the end of the translation unit.
>     The pragma should ideally be pushable and poppable, but that is
>     outside the scope of this RFC. The pragma also applies to static
>     local declarations within functions.
>
>     All global variables and functions affected by this pragma have
>     their default ELF section destinations changed. Globals with
>     __attribute__((section())) are not affected (the attribute trumps
>     the pragma).
>
>     This pragma is only defined to work sensibly for ELF targets.
>
>     ** Proposed implementation **
>
>     There are, I believe, three possible implementation stategies:
>       1) Clang internally sets the "section" on all globals it
>     creates. No changes in LLVM.
>       2) Clang sets some module-level attribute describing the default
>     section names. The LLVM backend takes this into account when
>     deciding the section name for a global when it emits it (AsmPrinter).
>       3) Clang sets the default section names as attributes on all
>     globals. The LLVM backend takes this into account when deciding
>     the section name for a global when it emits it (AsmPrinter).
>
>     (1) and (3) work with LTO. (2) interacts badly with LTO so is
>     discounted.
>
>     (1) requires Clang to perform the decision into exactly what
>     section a global will go, which is the wrong place for several
>     previously mentioned reasons (midend optimizations could promote
>     .data -> .bss, clang currently doesn't have the mechanics to
test
>     if an initializer is zero or not, LLVM does).
>
>     (2) and (3) have the advantage that the section type does not need
>     to be inferred by LLVM from its name. This means we don't need the
>     horrible string matching (m/^.bss./ -> BSS, m/^.data./ -> Data,
>     etc) in the AsmPrinter - users can specify whatever names they
>     like for bss sections without confusing the compiler.
>
>     Our previous proposal was (1). We are now proposing (3).
>
>     For (3), I think there are three distinct steps, none of which are
>     *particularly* invasive:
>
>       a) Allow arbitrary attributes on GlobalVariables. Currently
>     these are restricted to Functions only; I believe mainly because
>     noone had a usecase for attributes on variables.
>       b) Teach the clang frontend about the new pragma and CodeGen to
>     add the attributes to globals
>       c) Teach AsmPrinter to inspect these attributes if they exist
>     and take them into account when choosing sections.
>
>     All comments more than welcome,
>
>     James
>
>     On Fri, 3 Mar 2017 at 09:09 James Molloy <james at jamesmolloy.co.uk
>     <mailto:james at jamesmolloy.co.uk>> wrote:
>
>         Hi Reid,
>
>         Thanks for your comments. In truth, we don't have a particular
>         requirement for behaviour under LTO or
>         pushability/poppability, so we can define this in terms of
>         what's best for Clang.
>
>         As context is always useful, the goal of this proposed feature
>         is to provide a migration path toward Clang for developers in
>         the automotive domain. As Javed has mentioned, AUTOSAR which
>         is an automotive standard, mandates the use of a #pragma in
>         header files to determine in which sections initialized and
>         uninitialized data get put.
>
>         This feature is implemented in our legacy ARM Compiler 5
>         toolchain and we're also aware of GCC forks used across the
>         automotive space that have this feature implemented compatible
>         with the ARM Compiler 5 implementation.
>
>         The documentation is here:
>        
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472m/chr1359124985290.html
>
>         We do not aim for or anticipate syntax-level compatibility;
>         merely the ability to do something similar in Clang. In
>         particular Clang can't support the "section_type"
specifier
>         without significant rewrite, as LLVM decides the section type
>         (NOBITS/PROGBITS) by textual matching on the section name
>         (gross, but the fix is well beyond our scope).
>
>         Given that documentation, you can also see why the Microsoft
>         compatibility option *almost* met our requirements - all
>         except the behaviour in this case:
>
>         #pragma bss_seg('.bss.mybss')
>         int i; // Microsoft extension will put i in .bss.mybss
>         int j = 0; // Microsoft extension will put j in .data, whereas
>         we really need it in .bss.mybss
>
>         So to specifically answer your questions:
>
>         In ARM Compiler 5, all pragmas are pushable and poppable.
>         Clang doesn't have this feature generally yet, but when/if it
>         does, I don't see why this pragma shouldn't be affected. So
>         yes, we should consider it pushable and poppable.
>
>         I think the only reasonable behaviour under LTO must be that
>         the two TUs (may) have different bss and data sections.
>         Anything else would be very strange behaviour from the user's
>         perspective.
>
>         Your example of a static global migrating from .data into .bss
>         after optimization is interesting. With our proposal to
>         explicitly name sections in IR, this optimization would be
>         inhibited. I personally think that's fine :)
>
>         Cheers,
>
>         James
>
>         On Thu, 2 Mar 2017 at 21:42 Reid Kleckner via cfe-dev
>         <cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>> wrote:
>
>             Would these pragmas be translation-unit global or could
>             they be pushed and popped like the MSVC pragmas?
>
>             If two TUs are combined through LTO, can the two TUs have
>             different bss and data sections?
>
>             If the answer to both is "no", then I think we should
use
>             a module flag instead of manually setting the section from
>             the frontend.
>
>             Here is a C++ example where, after optimization, a global
>             may end up in .bss instead of .data:
>             // c++
>             static int f() { return 0; }
>             static int x = f();
>             int *g() { return &x; }
>
>             After global opt we get this:
>             @_ZL1x = internal global i32 0, align 4
>
>             Normally LLVM will put this in .bss or use .lcomm for it.
>             Your proposal will put it in the user's data section
>             instead of their bss section. If that's important, we
>             should use a module flag for this.
>
>             On Thu, Mar 2, 2017 at 11:35 AM, Javed Absar via cfe-dev
>             <cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>>
>             wrote:
>
>                 Hi all:
>
>                 We would like to propose a clang pragma directive to
>                 allow specialized section names.
>                 The semantics of it could be as follows. The pragma
>                 section name is declared in global
>                 scope. All global variables and functions get assigned
>                 to the corresponding specialized
>                 section name if one is present. With this feature, the
>                 following code:
>
>                 // foo.c
>                 #pragma bss_section(".bss.alpha")
>                 #pragma data_section(".data.beta")
>                 #pragma code_section(".code.gamma")
>                 #pragma const_section(".const.delta")
>                 int a;
>                 int b=2;
>                 const int d = 5;
>                 int c(){
>                   return d;
>                 }
>
>                 ..will emit llvm-ir as:
>
>                 target triple = "armv7-arm-none-eabi"
>                 @a = global i32 0, section ".bss.alpha", align 4
>                 @b = global i32 2, section ".data.beta", align 4
>                 @d = constant i32 5, section ".const.delta",
align 4
>
>                 ; Function Attrs: noinline nounwind
>                 define i32 @c() #0 section ".code.gamma" {
>                 entry:
>                   ret i32 5
>                 }
>
>                 This pragma will be very useful for embedded code which
>                 need to control where the different sections are
>                 placed in memory.
>
>                 Microsoft -fms-extension provides similar feature, but
>                 our proposal is for a
>                 general use.  Attributes are an alternative that is
>                 also currently available,
>                 but attributes are applicable only to specific
>                 declarations and not entire
>                 file. Many real embedded users prefer the pragma option.
>                 This will be a welcome enabler for them. Also,
>                 AUTOSAR, which is an
>
>                 automotive standard mandates use of a #pragma solution
>                 over an attribute one
>
>                 Looking forward to comments and suggestions.
>                 Best Regards
>
>                 Javed
>
>
>                 _______________________________________________
>                 cfe-dev mailing list
>                 cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>
>                 http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
>             _______________________________________________
>             cfe-dev mailing list
>             cfe-dev at lists.llvm.org <mailto:cfe-dev at
lists.llvm.org>
>             http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170310/85543ec0/attachment.html>

On 3/10/17 2:47 AM, James Molloy via llvm-dev wrote:
> +llvm-dev properly this time.
>
> On Fri, 10 Mar 2017 at 09:42 James Molloy <james at jamesmolloy.co.uk
> <mailto:james at jamesmolloy.co.uk>> wrote:
>
>
>     The documentation is here:
>    
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472m/chr1359124985290.html
>
>     ** Proposed syntax and (vague) semantics **
>
>     As this is a new pragma for Clang and isn't ARM-specific, we've
>     invented a less ARM-specific syntax. Bikeshedding is expected and
>     welcome.
>
>       #pragma clang section bss(".mybss")
rodata(".myrodata")
>     data(".mydata") text(".mytext")
There's some prior art in the GreenHills compiler too, where the example 
above would be spelled:

     #pragma ghs section bss=".mybss", rodata=".myrodata", 
data=".mydata", text=".mytext"

Maybe it's worth using that syntax, i.e:

     #pragma clang section bss=".mybss", rodata=".myrodata",
data=".mydata", text=".mytext"

Alternatively, supporting vendor-specific spellings might be useful, if 
there's a consistency argument to be made for clang doing it its own way.

Docs are pages 108/109 here:
http://www.ece.ualberta.ca/~cmpe490/documents/ghs/build_arm.pdf


Jon
>
>     The pragma applies to all global variable and function declarations
>     from the pragma to the end of the translation unit. The pragma
>     should ideally be pushable and poppable, but that is outside the
>     scope of this RFC. The pragma also applies to static local
>     declarations within functions.
>
>     All global variables and functions affected by this pragma have
>     their default ELF section destinations changed. Globals with
>     __attribute__((section())) are not affected (the attribute trumps
>     the pragma).
>
>     This pragma is only defined to work sensibly for ELF targets.
>

-- 
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded

Thanks Reid/Jonathon for your replies.


Reid,

An important case against module level flags is that it wont allow changing or 
resetting section names e.g.


int a;

#pragma clang section bss = "xyz"

int b;


In case above, users would like to see only 'b' placed in 'xyz'
and not 'a' as well.

Link pointed to by Jonathon seems to require same behavior.


Jonathon,

We are happy to revise the spelling to -

#pragma clang section bss = "xyz"

as you propose.

Best Regards, Javed.


From: Jonathan Roelofs <jonathan at codesourcery.com>

Sent: 13 March 2017 18:12:57
To: James Molloy; Reid Kleckner; Javed Absar
Cc: LLVM Dev; nd; cfe-dev at lists.llvm.org
Subject: Re: [llvm-dev] [cfe-dev] proposal - pragma section directive in clang



On 3/10/17 2:47 AM, James Molloy via llvm-dev wrote:
> +llvm-dev properly this time.
>
> On Fri, 10 Mar 2017 at 09:42 James Molloy <james at jamesmolloy.co.uk
> <mailto:james at jamesmolloy.co.uk>> wrote:
>
>
>     The documentation is here:
>    
http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0472m/chr1359124985290.html
>
>     ** Proposed syntax and (vague) semantics **
>
>     As this is a new pragma for Clang and isn't ARM-specific, we've
>     invented a less ARM-specific syntax. Bikeshedding is expected and
>     welcome.
>
>       #pragma clang section bss(".mybss")
rodata(".myrodata")
>     data(".mydata") text(".mytext")
There's some prior art in the GreenHills compiler too, where the example
above would be spelled:

     #pragma ghs section bss=".mybss", rodata=".myrodata",
data=".mydata", text=".mytext"

Maybe it's worth using that syntax, i.e:

     #pragma clang section bss=".mybss", rodata=".myrodata",
data=".mydata", text=".mytext"

Alternatively, supporting vendor-specific spellings might be useful, if
there's a consistency argument to be made for clang doing it its own way.

Docs are pages 108/109 here:
http://www.ece.ualberta.ca/~cmpe490/documents/ghs/build_arm.pdf


Jon
>
>     The pragma applies to all global variable and function declarations
>     from the pragma to the end of the translation unit. The pragma
>     should ideally be pushable and poppable, but that is outside the
>     scope of this RFC. The pragma also applies to static local
>     declarations within functions.
>
>     All global variables and functions affected by this pragma have
>     their default ELF section destinations changed. Globals with
>     __attribute__((section())) are not affected (the attribute trumps
>     the pragma).
>
>     This pragma is only defined to work sensibly for ELF targets.
>

--
Jon Roelofs
jonathan at codesourcery.com
CodeSourcery / Mentor Embedded
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170314/4f802839/attachment.html>