llvm dev - Jul 2011 - [LLVMdev] Question about SimplifyXorInst

Hi all,

I am master student in Edinburgh, UK. I am doing my MSc project with  
LLVM compiler and I have to modify LLVM to implement the StackGuard  
with a XOR random Canary. However, I am not familiar with LLVM.

My problem is that I want to XOR the random canary word with the  
return address which are both 32 bits. I found a method called  
SimplifyXorInst(Value *, Value *, const TargetData *,const  
DominatorTree *, unsigned) but I do not know whether it is right  
method and how to use it. Could you give me some advice or sample code  
about it?

Thank you
Best wishes,
Ying

-- 
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336.

On 7/20/11 8:24 AM, Xueying ZHANG wrote:
> Hi all,
>
> I am master student in Edinburgh, UK. I am doing my MSc project with
> LLVM compiler and I have to modify LLVM to implement the StackGuard
> with a XOR random Canary. However, I am not familiar with LLVM.
This sounds similar to the StackProtector feature that LLVM already
has.
>
> My problem is that I want to XOR the random canary word with the
> return address which are both 32 bits. I found a method called
> SimplifyXorInst(Value *, Value *, const TargetData *,const
> DominatorTree *, unsigned) but I do not know whether it is right
> method and how to use it. Could you give me some advice or sample code
> about it?
I have no idea what SimplifyXorInst() does, but since the address of the 
return address is not visible at the LLVM IR level (to the best of my 
knowledge), it is probably not what you want.

The most straightforward thing to do, I think, is to write a 
MachineFunctionPass that adds prologue/epilogue code during code 
generation to do the xor'ing of the return address.

I haven't written a MachineFunctionPass, so I can't really help with any
details.  My best suggestion is to find the StackProtector code in LLVM 
and see how it works.  The doxygen docs are also valuable 
(http://llvm.org/doxygen/hierarchy.html).

-- John T.
>
> Thank you
> Best wishes,
> Ying
>

On Jul 20, 2011, at 6:24 AM, Xueying ZHANG wrote:
> Hi all,
> 
> I am master student in Edinburgh, UK. I am doing my MSc project with  
> LLVM compiler and I have to modify LLVM to implement the StackGuard  
> with a XOR random Canary. However, I am not familiar with LLVM.
> 
> My problem is that I want to XOR the random canary word with the  
> return address which are both 32 bits. I found a method called  
> SimplifyXorInst(Value *, Value *, const TargetData *,const  
> DominatorTree *, unsigned) but I do not know whether it is right  
> method and how to use it. Could you give me some advice or sample code  
> about it?
> 
Hi Ying,

Like John said, it sounds like you want to modify the stack protector code. Look
at this file:

	lib/CodeGen/StackProtector.cpp

and see if it could help you out. The SimplifyXorInst function is part of the
"Instruction Combine" pass, which is an optimization pass. I doubt
that it's the correct place to do what you're suggesting doing.

-bw