llvm dev - Jul 2019 - Generating completely position agnostic code

I'm on a mission to generate code that can be loaded from disk without
any modifications. This means no relocations can occur.

Trying to see if this can be done for C++ code that uses STL but has
no global variables, and a single function, but of course Clang will
generate more functions for STL code.

I want to provide an array of function pointers so that for all
interactions STL needs to do with LIBC that I'm able to just provide
it via indirect calls.

Has anyone had success with such a thing in LLVM?

Qs for you:

The code that is being loaded from disk... is it wholly self-contained, or
is your executable potentially made up of several pieces that each need to
be loaded from disk?

What does it mean to use the STL but not have global variables?  std::cout
is a global variable, so you can't even do Hello World without globals.

= = 
Architectures such as 68K and PowerPC and RISC-V have a dedicated register
for accessing global variables, rather than the PC-relative globals used in
other architectures.  This makes them inherently more amenable to what you
describe, since you can put the "array of function pointers" into
global
space, as part of setting up global space in general, and then load the
code from disk, and go.  There is no relocation needed since all access to
globals is done via the global register, not relative to wherever the
program was loaded.  Of course, access to something like libc might
normally need post-loading relocation, but if you do what you're talking
about and use an "array of function pointers" to get to libc, no
relocation
would be needed.

For what it's worth, the original 68K-based Macintosh used a scheme quite
similar to this.  The big difference for the Mac was that to get to the OS
(the equivalent of libc), it didn't use an array of function pointers, per
se; it used a certain range of illegal instructions, which generated
exceptions when used, and the (highly optimized) exception handlers would
recover from the exception by dispatching to an OS routine determined by
the specific bits in the illegal instruction.

On Sun, Jun 30, 2019 at 9:07 PM Hayden Livingston via llvm-dev <
llvm-dev at lists.llvm.org> wrote:
> I'm on a mission to generate code that can be loaded from disk without
> any modifications. This means no relocations can occur.
>
> Trying to see if this can be done for C++ code that uses STL but has
> no global variables, and a single function, but of course Clang will
> generate more functions for STL code.
>
> I want to provide an array of function pointers so that for all
> interactions STL needs to do with LIBC that I'm able to just provide
> it via indirect calls.
>
> Has anyone had success with such a thing in LLVM?
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190701/17e97979/attachment.html>

It is wholly self-contained. It's code that has no references to
anything beyond a set of pointers passed in as arguments to the
function. This piece of code doesn't do any OS work at all. It is
purely calling function pointers, doing math and allocating memory.

On Mon, Jul 1, 2019 at 12:57 AM Jorg Brown <jorg.brown at gmail.com>
wrote:
>
> Qs for you:
>
> The code that is being loaded from disk... is it wholly self-contained, or
is your executable potentially made up of several pieces that each need to be
loaded from disk?
>
> What does it mean to use the STL but not have global variables?  std::cout
is a global variable, so you can't even do Hello World without globals.
>
> = = >
> Architectures such as 68K and PowerPC and RISC-V have a dedicated register
for accessing global variables, rather than the PC-relative globals used in
other architectures.  This makes them inherently more amenable to what you
describe, since you can put the "array of function pointers" into
global space, as part of setting up global space in general, and then load the
code from disk, and go.  There is no relocation needed since all access to
globals is done via the global register, not relative to wherever the program
was loaded.  Of course, access to something like libc might normally need
post-loading relocation, but if you do what you're talking about and use an
"array of function pointers" to get to libc, no relocation would be
needed.
>
> For what it's worth, the original 68K-based Macintosh used a scheme
quite similar to this.  The big difference for the Mac was that to get to the OS
(the equivalent of libc), it didn't use an array of function pointers, per
se; it used a certain range of illegal instructions, which generated exceptions
when used, and the (highly optimized) exception handlers would recover from the
exception by dispatching to an OS routine determined by the specific bits in the
illegal instruction.
>
> On Sun, Jun 30, 2019 at 9:07 PM Hayden Livingston via llvm-dev <llvm-dev
at lists.llvm.org> wrote:
>>
>> I'm on a mission to generate code that can be loaded from disk
without
>> any modifications. This means no relocations can occur.
>>
>> Trying to see if this can be done for C++ code that uses STL but has
>> no global variables, and a single function, but of course Clang will
>> generate more functions for STL code.
>>
>> I want to provide an array of function pointers so that for all
>> interactions STL needs to do with LIBC that I'm able to just
provide
>> it via indirect calls.
>>
>> Has anyone had success with such a thing in LLVM?
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

I'd definitely like to throw my vote behind this. I've managed to
write what are effectively bare-metal BIOS bootloaders for dedicated
tasks on x86 using LLVM, but I did have to modify the x86 backend to
achieve it, and it isn't complete. Most of the work came down to
modifying the functions that handle reference classification.
Combining this with building via lld and code extraction via objcopy
allows me to do most of what I need. That said, this is from C code,
not C++; I'd be really interested to see it working with C++, but
complex static initialization seems to be a difficult problem in that
regard.

On Mon, Jul 1, 2019 at 12:08 AM Hayden Livingston via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> I'm on a mission to generate code that can be loaded from disk without
> any modifications. This means no relocations can occur.
>
> Trying to see if this can be done for C++ code that uses STL but has
> no global variables, and a single function, but of course Clang will
> generate more functions for STL code.
>
> I want to provide an array of function pointers so that for all
> interactions STL needs to do with LIBC that I'm able to just provide
> it via indirect calls.
>
> Has anyone had success with such a thing in LLVM?
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

On Sun, Jun 30, 2019 at 09:07:21PM -0700, Hayden Livingston via llvm-dev
wrote:
> I'm on a mission to generate code that can be loaded from disk without
> any modifications. This means no relocations can occur.
Why? Implementing self relocation is not that difficult really. It's
also quite difficult to avoid unless you can make sure that no pointer
variables are statically initialized. That means no arrays of strings,
no vtables etc.

Joerg

Because I don't think it is required. All I want to do is generate
machine code for a single function that has all its references coming
in as pointers on the stack and it's just calling functions and doing
math. I'm literally asking LLVM convert text in the form of C to x86
ASM.

Maybe I'll paste LLVM IR to make it clear what I'm asking for.



On Mon, Jul 1, 2019 at 4:25 PM Joerg Sonnenberger via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> On Sun, Jun 30, 2019 at 09:07:21PM -0700, Hayden Livingston via llvm-dev
wrote:
> > I'm on a mission to generate code that can be loaded from disk
without
> > any modifications. This means no relocations can occur.
>
> Why? Implementing self relocation is not that difficult really. It's
> also quite difficult to avoid unless you can make sure that no pointer
> variables are statically initialized. That means no arrays of strings,
> no vtables etc.
>
> Joerg
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev