llvm dev - Sep 2015 - adding prefixes to certain instructions x86 -- where to start?

Hello,

I’ve been using LLVM IR passes for my research for about a year now, but for my
next step I think I might have to dig into a backend.  I'm hoping someone
could give me a pointer on how to get started.

The thing I would like to do is add and override address-size override prefix
[1] to a given x86-64 instruction.  I’m hoping I can do something like:

1) Mark some IR instructions with metadata in my pass
2) Hack the backend to look for my metadata, and if found add the prefix when
the machine instruction is emitted

Does this seem feasible?  Does the LLVM x86 backend currently have the
capability of adding instruction prefixes and could someone please point out
where I should look in the code for it?

Thanks,
Scott A. Carr
PhD Student
Purdue University CS

[1]
http://wiki.osdev.org/X86-64_Instruction_Encoding#Operand-size_and_address-size_override_prefix

What is it you are ACTUALLY trying to do?

In other words, why would you want a different address size...
Understanding that would probably help provide a better answer (I have
absolutely no idea how to solve the actual question, but I suspect
understanding what the overall goal is will help a whole lot)

--
Mats

On 11 September 2015 at 15:38, Scott A. Carr via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> Hello,
>
> I’ve been using LLVM IR passes for my research for about a year now, but
> for my next step I think I might have to dig into a backend.  I'm
hoping
> someone could give me a pointer on how to get started.
>
> The thing I would like to do is add and override address-size override
> prefix [1] to a given x86-64 instruction.  I’m hoping I can do something
> like:
>
> 1) Mark some IR instructions with metadata in my pass
> 2) Hack the backend to look for my metadata, and if found add the prefix
> when the machine instruction is emitted
>
> Does this seem feasible?  Does the LLVM x86 backend currently have the
> capability of adding instruction prefixes and could someone please point
> out where I should look in the code for it?
>
> Thanks,
> Scott A. Carr
> PhD Student
> Purdue University CS
>
> [1]
>
http://wiki.osdev.org/X86-64_Instruction_Encoding#Operand-size_and_address-size_override_prefix
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150911/5c4253d1/attachment.html>

I'm trying to make a security sandbox.  For example, lets say my program 
has a LoadInst in the LLVM IR and I know I want to confine the address 
range this LoadInst is accessing.  Maybe that LoadInst gets emitted as a 
MOV machine code instruction by the backend. During execution an 
attacker could potentially control the operands of the MOV instruction 
through some exploit, but usually he cannot modify the 
instructions/prefixes because the code is not writable. So the prefix 
can potentially let me confine the attacker to an address range even if 
he controls the instruction operands.

I hope that makes some sense.  If someone knows of a different approach 
-- a very light weight sandbox implemented in LLVM I'd be interested 
looking into it.

Thanks,
Scott

On 09/11/2015 10:55 AM, mats petersson wrote:
> What is it you are ACTUALLY trying to do?
>
> In other words, why would you want a different address size... 
> Understanding that would probably help provide a better answer (I have 
> absolutely no idea how to solve the actual question, but I suspect 
> understanding what the overall goal is will help a whole lot)
>
> --
> Mats
>
> On 11 September 2015 at 15:38, Scott A. Carr via llvm-dev 
> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>
>     Hello,
>
>     I’ve been using LLVM IR passes for my research for about a year
>     now, but for my next step I think I might have to dig into a
>     backend.  I'm hoping someone could give me a pointer on how to get
>     started.
>
>     The thing I would like to do is add and override address-size
>     override prefix [1] to a given x86-64 instruction.  I’m hoping I
>     can do something like:
>
>     1) Mark some IR instructions with metadata in my pass
>     2) Hack the backend to look for my metadata, and if found add the
>     prefix when the machine instruction is emitted
>
>     Does this seem feasible?  Does the LLVM x86 backend currently have
>     the capability of adding instruction prefixes and could someone
>     please point out where I should look in the code for it?
>
>     Thanks,
>     Scott A. Carr
>     PhD Student
>     Purdue University CS
>
>     [1]
>    
http://wiki.osdev.org/X86-64_Instruction_Encoding#Operand-size_and_address-size_override_prefix
>     _______________________________________________
>     LLVM Developers mailing list
>     llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150911/e0366b71/attachment.html>