llvm dev - Nov 2020 - LLVM X86 MachineBasicBlock inserting push and pop instructions causes segmentation fault

Hello,

I am working on a project where I need to insert some logic before each
machine basic block.
In particular, it involves setting some global variables and calling a
function. I'm able to add the instructions and verify they get added, but
when the compiled program runs, it stops with a segfault.

For brevity, I'm not sharing the whole code here but basically I have a X86
MachineFunctionPass added to addPreEmitPass2 stage which simply inserts a
push rcx immediately followed by pop rcx before each basic block (only the
relevant logic portions are included):

/* Inserts push rcx followed by pop rcx before each MachineBasicBlock */
void VirtualTimeManager::__insertVtlLogic(MachineFunction &MF,
    MachineBasicBlock* origMBB) {
    const llvm::TargetInstrInfo &TII = *MF.getSubtarget().getInstrInfo();
    auto MMI = &MF.getMMI();
    llvm::Module &M = const_cast<Module &>(*MMI->getModule());
    if (origMBB->empty() || !origMBB->isLegalToHoistInto())
         return;
    llvm::BuildMI(*origMBB, origMBB->begin(), DebugLoc(),
        TII.get(X86::POP64r)).addReg(X86::RCX);
    /* SOME ADDITIONAL LOGIC COMMENTED OUT */
    llvm::BuildMI(*origMBB, origMBB->begin(), DebugLoc(),
        TII.get(X86::PUSH64r)).addReg(X86::RCX);
}
bool VirtualTimeManager::runOnMachineFunction(MachineFunction &MF) {
         for (auto &MBB : MF) {
               MachineBasicBlock* origMBB = &MBB;
                __insertVtlLogic(MF, origMBB);
        }
       return true;
}

When I compile and run a program (e.g from SPEC-2006 benchmark (sjeng)),
using this pass, the program segfaults. I don't understand how simply
adding a push, followed by pop is affecting the program execution. Could
anyone please offer some insights. It would be greatly appreciated. I'm new
to LLVM and struggling to understand what is causing this issue.

Thanks,
Vignesh
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201111/8c3716c3/attachment.html>

Best guess is that you're clobbering something in the red zone.
https://en.wikipedia.org/wiki/Red_zone_(computing).  It's not guarantee
that the area of the stack above the stack pointer is unused. I think you
can check if the redzone is used by checking getUsesRedZone() in
X86MachineFunctionInfo. If the red zone is used, its not safe to insert a
push/pop. I think you can turn off the red zone by passing -mno-red-zone to
clang.



~Craig


On Thu, Nov 12, 2020 at 10:18 AM vignesh babu via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hello,
>
> I am working on a project where I need to insert some logic before each
> machine basic block.
> In particular, it involves setting some global variables and calling a
> function. I'm able to add the instructions and verify they get added,
but
> when the compiled program runs, it stops with a segfault.
>
> For brevity, I'm not sharing the whole code here but basically I have a
> X86 MachineFunctionPass added to addPreEmitPass2 stage which simply inserts
> a push rcx immediately followed by pop rcx before each basic block (only
> the relevant logic portions are included):
>
> /* Inserts push rcx followed by pop rcx before each MachineBasicBlock */
> void VirtualTimeManager::__insertVtlLogic(MachineFunction &MF,
>     MachineBasicBlock* origMBB) {
>     const llvm::TargetInstrInfo &TII =
*MF.getSubtarget().getInstrInfo();
>     auto MMI = &MF.getMMI();
>     llvm::Module &M = const_cast<Module
&>(*MMI->getModule());
>     if (origMBB->empty() || !origMBB->isLegalToHoistInto())
>          return;
>     llvm::BuildMI(*origMBB, origMBB->begin(), DebugLoc(),
>         TII.get(X86::POP64r)).addReg(X86::RCX);
>     /* SOME ADDITIONAL LOGIC COMMENTED OUT */
>     llvm::BuildMI(*origMBB, origMBB->begin(), DebugLoc(),
>         TII.get(X86::PUSH64r)).addReg(X86::RCX);
> }
> bool VirtualTimeManager::runOnMachineFunction(MachineFunction &MF) {
>          for (auto &MBB : MF) {
>                MachineBasicBlock* origMBB = &MBB;
>                 __insertVtlLogic(MF, origMBB);
>         }
>        return true;
> }
>
> When I compile and run a program (e.g from SPEC-2006 benchmark (sjeng)),
> using this pass, the program segfaults. I don't understand how simply
> adding a push, followed by pop is affecting the program execution. Could
> anyone please offer some insights. It would be greatly appreciated. I'm
new
> to LLVM and struggling to understand what is causing this issue.
>
> Thanks,
> Vignesh
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201112/0747d2c0/attachment.html>

Hi Craig,

Sorry for the late reply, I was travelling. Thank you very much for your
suggestion. This turned out to be the exact problem.

I was able to get around it by disabling red-zone or by decrementing the
stack pointer by 128 bytes (typical size of red-zone) before pushing
anything and reverting the stack pointer after any pop operations.

Thanks,
Vignesh

On Thu, Nov 12, 2020 at 12:48 PM Craig Topper <craig.topper at gmail.com>
wrote:
> Best guess is that you're clobbering something in the red zone.
> https://en.wikipedia.org/wiki/Red_zone_(computing).  It's not guarantee
> that the area of the stack above the stack pointer is unused. I think you
> can check if the redzone is used by checking getUsesRedZone() in
> X86MachineFunctionInfo. If the red zone is used, its not safe to insert a
> push/pop. I think you can turn off the red zone by passing -mno-red-zone to
> clang.
>
>
>
> ~Craig
>
>
> On Thu, Nov 12, 2020 at 10:18 AM vignesh babu via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Hello,
>>
>> I am working on a project where I need to insert some logic before each
>> machine basic block.
>> In particular, it involves setting some global variables and calling a
>> function. I'm able to add the instructions and verify they get
added, but
>> when the compiled program runs, it stops with a segfault.
>>
>> For brevity, I'm not sharing the whole code here but basically I
have a
>> X86 MachineFunctionPass added to addPreEmitPass2 stage which simply
inserts
>> a push rcx immediately followed by pop rcx before each basic block
(only
>> the relevant logic portions are included):
>>
>> /* Inserts push rcx followed by pop rcx before each MachineBasicBlock
*/
>> void VirtualTimeManager::__insertVtlLogic(MachineFunction &MF,
>>     MachineBasicBlock* origMBB) {
>>     const llvm::TargetInstrInfo &TII =
*MF.getSubtarget().getInstrInfo();
>>     auto MMI = &MF.getMMI();
>>     llvm::Module &M = const_cast<Module
&>(*MMI->getModule());
>>     if (origMBB->empty() || !origMBB->isLegalToHoistInto())
>>          return;
>>     llvm::BuildMI(*origMBB, origMBB->begin(), DebugLoc(),
>>         TII.get(X86::POP64r)).addReg(X86::RCX);
>>     /* SOME ADDITIONAL LOGIC COMMENTED OUT */
>>     llvm::BuildMI(*origMBB, origMBB->begin(), DebugLoc(),
>>         TII.get(X86::PUSH64r)).addReg(X86::RCX);
>> }
>> bool VirtualTimeManager::runOnMachineFunction(MachineFunction &MF)
{
>>          for (auto &MBB : MF) {
>>                MachineBasicBlock* origMBB = &MBB;
>>                 __insertVtlLogic(MF, origMBB);
>>         }
>>        return true;
>> }
>>
>> When I compile and run a program (e.g from SPEC-2006 benchmark
(sjeng)),
>> using this pass, the program segfaults. I don't understand how
simply
>> adding a push, followed by pop is affecting the program execution.
Could
>> anyone please offer some insights. It would be greatly appreciated.
I'm new
>> to LLVM and struggling to understand what is causing this issue.
>>
>> Thanks,
>> Vignesh
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20201116/c99edd09/attachment.html>