llvm dev - Jul 2016 - grouping global variables by alignment: safe to do at LLVM level, or only at Clang level?

>> My question this time is this: is re-ordering globals at the LLVM level
both possible [...]
On 07/25/2016 10:53 PM, Mehdi Amini wrote:
> Isn’t something a linker could/should do?
AFAIK & TTBOMK, a compiler should do it [if allowed by the language] and a
linker should not
mess it up.  I don`t know that a linker is allowed to make assumptions about
code not
traversing labels in its data accesses.  After all, a compiler may have assigned
labels to e.g.
the start of a "struct" instance and to a specific part of that same
struct.  That doesn`t mean
the linker is free to move the middle of the struct somewhere else far from the
start of the
struct.

IOW, a linker is required to handle the code no matter what the source language
was and what
its specification says about implementations being allowed to re-order things
and whether or
not the language allows the programmer to _validly_ assume that when two arrays
of identical
element type are created back-to-back, the second array immediately follows the
first, thus
effectively creating a single array with its element count being the sum of the
two element
counts of the as-written arrays.



On 07/25/2016 10:53 PM, Mehdi Amini wrote:
> Such programming language should not use LLVM global variables if they need
to provide such guarantee,
>  but probably use instead something more like a global array when lowering
to LLVM IR.
Agreed, but I can`t [en]force that.  That would need to be decided by the LLVM
steering
committee or something like that, then documented publicly.

Thanks for your reply.

Regards,

Abe

On 26 July 2016 at 08:46, Abe Skolnik via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> AFAIK & TTBOMK, a compiler should do it [if allowed by the language]
and a
> linker should not mess it up.  I don`t know that a linker is allowed to
make
> assumptions about code not traversing labels in its data accesses.
In ELF land, it's not, which is why you pass -fdata-sections. That
option makes Clang put every global into its own section, and the
linker *is* allowed to assume accesses can't cross section boundaries.

In MachO land, the linker is allowed to make that assumption if the
object file has been marked with a .subsections_via_symbols directive
(which it is by default on all Darwin targets).

It really should be the linker's job in the general case, both for
optimal code and to avoid duplicating effort.

Tim.

> On Jul 26, 2016, at 8:46 AM, Abe Skolnik <a.skolnik at samsung.com>
wrote:
> 
>>> My question this time is this: is re-ordering globals at the LLVM
level both possible [...]
> 
> On 07/25/2016 10:53 PM, Mehdi Amini wrote:
> 
>> Isn’t something a linker could/should do?
> 
> AFAIK & TTBOMK, a compiler should do it [if allowed by the language]
Compiler only see one translation unit at a time, the linker sees multiple, it
is not clear to me why you are thinking the compiler should handle it?
> and a linker should not mess it up.  I don`t know that a linker is allowed
to make assumptions about code not traversing labels in its data accesses. 
After all, a compiler may have assigned labels to e.g. the start of a
"struct" instance and to a specific part of that same struct.  That
doesn`t mean the linker is free to move the middle of the struct somewhere else
far from the start of the struct.
> 
> IOW, a linker is required to handle the code no matter what the source
language was and what its specification says about implementations being allowed
to re-order things and whether or not the language allows the programmer to
_validly_ assume that when two arrays of identical element type are created
back-to-back, the second array immediately follows the first, thus effectively
creating a single array with its element count being the sum of the two element
counts of the as-written arrays.
> 
> 
> 
> On 07/25/2016 10:53 PM, Mehdi Amini wrote:
> 
>> Such programming language should not use LLVM global variables if they
need to provide such guarantee,
>> but probably use instead something more like a global array when
lowering to LLVM IR.
> 
> Agreed, but I can`t [en]force that.  That would need to be decided by the
LLVM steering committee or something like that, then documented publicly.
I’m saying that it is *already* the current state of affairs, AFAIK we don’t
provide any guarantee about what you describes.

— 
Mehdi

> I’m saying that it is *already* the current state of affairs, AFAIK we
don’t provide any guarantee about what you describes.
We already do some quite invasive things to real-world globals that
would violate such assumptions. We increase their alignment beyond ABI
requirements if we think it's helpful; the GlobalMerge pass sorts
globals before shoving them all under one symbol (in increasing order
size for some reason that escapes me right now).

Tim.

Arriving late to this thread, but to add what others have said: reordering
or sorting data items based on alignment is straightforward to do in a
linker plugin. An example can be found in this test (part of gold linker
test suite):

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=gold/testsuite/plugin_layout_with_alignment.sh;h=c5f07aec287e87de955dd924efefabd56e8d6213;hb=HEAD

The source for the plugin is at

https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=blob;f=gold/testsuite/plugin_section_alignment.cc;h=6f64bdc6e8687bc5cfb61109503844d9608ee757;hb=HEAD

Than


On Tue, Jul 26, 2016 at 12:54 PM, Mehdi Amini via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
>
> > On Jul 26, 2016, at 8:46 AM, Abe Skolnik <a.skolnik at
samsung.com> wrote:
> >
> >>> My question this time is this: is re-ordering globals at the
LLVM
> level both possible [...]
> >
> > On 07/25/2016 10:53 PM, Mehdi Amini wrote:
> >
> >> Isn’t something a linker could/should do?
> >
> > AFAIK & TTBOMK, a compiler should do it [if allowed by the
language]
>
> Compiler only see one translation unit at a time, the linker sees
> multiple, it is not clear to me why you are thinking the compiler should
> handle it?
>
> > and a linker should not mess it up.  I don`t know that a linker is
> allowed to make assumptions about code not traversing labels in its data
> accesses.  After all, a compiler may have assigned labels to e.g. the start
> of a "struct" instance and to a specific part of that same
struct.  That
> doesn`t mean the linker is free to move the middle of the struct somewhere
> else far from the start of the struct.
> >
> > IOW, a linker is required to handle the code no matter what the source
> language was and what its specification says about implementations being
> allowed to re-order things and whether or not the language allows the
> programmer to _validly_ assume that when two arrays of identical element
> type are created back-to-back, the second array immediately follows the
> first, thus effectively creating a single array with its element count
> being the sum of the two element counts of the as-written arrays.
> >
> >
> >
> > On 07/25/2016 10:53 PM, Mehdi Amini wrote:
> >
> >> Such programming language should not use LLVM global variables if
they
> need to provide such guarantee,
> >> but probably use instead something more like a global array when
> lowering to LLVM IR.
> >
> > Agreed, but I can`t [en]force that.  That would need to be decided by
> the LLVM steering committee or something like that, then documented
> publicly.
>
> I’m saying that it is *already* the current state of affairs, AFAIK we
> don’t provide any guarantee about what you describes.
>
> —
> Mehdi
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160809/08a95f1e/attachment.html>