llvm dev - Mar 2012 - [LLVMdev] Stack alignment on X86 AVX seems incorrect

Hi Elena,

You're correct. LLVM does not align the stack to 32-bytes for AVX and
unaligned moves should be used for YMM spills.

I wrote some code to align the stack to 32-bytes when AVX spills are
present; it does break the x86-64 ABI though. If upstream would be
interested in this code, I can arrange with my employer to send a patch to
the mailing list.

-Cameron

On Mar 1, 2012, at 4:09 PM, <llvmdev-request at cs.uiuc.edu> wrote:

Message: 2
Date: Thu, 1 Mar 2012 18:16:46 +0000
From: "Demikhovsky, Elena" <elena.demikhovsky at intel.com>
Subject: [LLVMdev] Stack alignment on X86 AVX seems incorrect
To: "llvmdev at cs.uiuc.edu" <llvmdev at cs.uiuc.edu>
Message-ID:
<A0DC88CEB3010344830D52D66533DA8E0C2E7A at HASMSX103.ger.corp.intel.com>
Content-Type: text/plain; charset="windows-1252"

./llc -mattr=+avx -stack-alignment=16 < basic.ll | grep movaps | grep ymm |
grep rbp
       vmovaps -176(%rbp), %ymm14
       vmovaps -144(%rbp), %ymm11
       vmovaps -240(%rbp), %ymm13
       vmovaps -208(%rbp), %ymm9
       vmovaps -272(%rbp), %ymm7
       vmovaps -304(%rbp), %ymm0
       vmovaps -112(%rbp), %ymm0
       vmovaps -80(%rbp), %ymm1
       vmovaps -112(%rbp), %ymm0
       vmovaps -80(%rbp), %ymm0
       vmovaps -176(%rbp), %ymm15
       vmovaps -144(%rbp), %ymm0
       vmovaps -240(%rbp), %ymm0
       vmovaps -208(%rbp), %ymm0
       vmovaps -272(%rbp), %ymm0
       vmovaps -304(%rbp), %ymm0

vmovaps should not access stack if it is not aligned to 32

- Elena
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120301/91d47a80/attachment.html>

Cameron,

Aligning the stack to 32 bytes when there are auto AVX vector variables 
present shouldn't necessarily break the x86-64 ABI, as long as smaller 
auto variables remain properly aligned.  A similar approach was taken 
for i386 in GCC in order to support SSE vectors.

Perhaps you could elaborate where the ABI was violated when your patch 
is applied.

HTH

-- 
Evandro Menezes          Austin, TX          emenezes at codeaurora.org
Qualcomm Innovation Center, Inc is a member of the Code Aurora Forum


On 03/01/12 15:18, Cameron McInally wrote:
> Hi Elena,
>
> You're correct. LLVM does not align the stack to 32-bytes for AVX and
> unaligned moves should be used for YMM spills.
>
> I wrote some code to align the stack to 32-bytes when AVX spills are
> present; it does break the x86-64 ABI though. If upstream would be
> interested in this code, I can arrange with my employer to send a patch
> to the mailing list.
>
> -Cameron
>
> On Mar 1, 2012, at 4:09 PM, <llvmdev-request at cs.uiuc.edu
> <mailto:llvmdev-request at cs.uiuc.edu>> wrote:
>
>> Message: 2
>> Date: Thu, 1 Mar 2012 18:16:46 +0000
>> From: "Demikhovsky, Elena" <elena.demikhovsky at intel.com
>> <mailto:elena.demikhovsky at intel.com>>
>> Subject: [LLVMdev] Stack alignment on X86 AVX seems incorrect
>> To: "llvmdev at cs.uiuc.edu <mailto:llvmdev at
cs.uiuc.edu>"
>> <llvmdev at cs.uiuc.edu <mailto:llvmdev at cs.uiuc.edu>>
>> Message-ID:
>> <A0DC88CEB3010344830D52D66533DA8E0C2E7A at
HASMSX103.ger.corp.intel.com
>> <mailto:A0DC88CEB3010344830D52D66533DA8E0C2E7A at
HASMSX103.ger.corp.intel.com>>
>> Content-Type: text/plain; charset="windows-1252"
>>
>> ./llc -mattr=+avx -stack-alignment=16 < basic.ll | grep movaps |
grep
>> ymm | grep rbp
>>        vmovaps -176(%rbp), %ymm14
>>        vmovaps -144(%rbp), %ymm11
>>        vmovaps -240(%rbp), %ymm13
>>        vmovaps -208(%rbp), %ymm9
>>        vmovaps -272(%rbp), %ymm7
>>        vmovaps -304(%rbp), %ymm0
>>        vmovaps -112(%rbp), %ymm0
>>        vmovaps -80(%rbp), %ymm1
>>        vmovaps -112(%rbp), %ymm0
>>        vmovaps -80(%rbp), %ymm0
>>        vmovaps -176(%rbp), %ymm15
>>        vmovaps -144(%rbp), %ymm0
>>        vmovaps -240(%rbp), %ymm0
>>        vmovaps -208(%rbp), %ymm0
>>        vmovaps -272(%rbp), %ymm0
>>        vmovaps -304(%rbp), %ymm0
>>
>> vmovaps should not access stack if it is not aligned to 32
>>
>> - Elena
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

On Thu, Mar 1, 2012 at 1:18 PM, Cameron McInally
<cameron.mcinally at nyu.edu> wrote:
> Hi Elena,
>
> You're correct. LLVM does not align the stack to 32-bytes for AVX and
> unaligned moves should be used for YMM spills.
Really?  There's supposed to be code to realign the stack when
necessary... it's possible that code is broken, though.

-Eli

On Thu, Mar 1, 2012 at 4:29 PM, Evandro Menezes <emenezes at
codeaurora.org>wrote:
...
> Aligning the stack to 32 bytes when there are auto AVX vector variables
> present shouldn't necessarily break the x86-64 ABI, as long as smaller
auto
> variables remain properly aligned.  A similar approach was taken for i386
> in GCC in order to support SSE vectors.
>
> Perhaps you could elaborate where the ABI was violated when your patch is
> applied.
>
Sorry, I confused myself. This was worked out about a year ago, so it's not
in my cache.

You're right. In my last email I wrote "align the stack", when I
should
have written "align the frame when variable sized objects are in
play".
Take main(...) for example, with a few alloca's. If one would like to spill
AVX regs with aligned moves, one must align the frame to 32 bytes to ensure
that the spill slots are aligned correctly, since spill slots are based off
of the frame pointer.

The x86-64 ABI lays out the stack frame as...

...
16(%rbp)  mem arg[0]
8(%rbp)    return address
0(%rbp)    previous %rbp
-8(%rbp)   stack

My patch breaks the ABI since the ABI requires the return address to be
found at 8(%rbp) and mem arg[0] at 16(%rbp). Unfortunately, to align the
frame at runtime, it's sometimes required to insert padding in between the
two.

I'll ask for my company's permission to share my implementation. Until
then, I'll have to bite my tongue.

-Cameron
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20120301/e9a0be25/attachment.html>