llvm dev - May 2012 - [LLVMdev] lld file format as native OS executable format

Hello,

Would it be feasible to use the internal lld file format as the native 
executable format for an OS? Are there performance or space 
considerations that would make this a poor choice?

Cheers,
Shea Levy

P.S. please CC me on replies, I'm not subscribed.

On 5/2/12 10:29 AM, Shea Levy wrote:
> Hello,
>
> Would it be feasible to use the internal lld file format as the native
> executable format for an OS? Are there performance or space
> considerations that would make this a poor choice?
By lld, do you mean the LLVM IR bitcode format, or are you referring to 
something else?

As far as using LLVM IR as a format for executables, there was a 
research paper on that topic with an older version of LLVM 
(http://llvm.org/pubs/2003-10-01-LLVA.html).  You'd probably want to 
redo the experiments on file size and such since LLVM's on-disk format 
has changed considerably over the years, but it might give you an idea 
of what issues to consider and what trouble spots there might be.

Also note that there's follow-on work to support an entire OS on the 
LLVM IR without the inline assembly language feature.  Just look for the 
publications written by me if you need that.
:)

The more significant challenge, I think, would be translating LLVM IR to 
native code on-the-fly.  That'll add overhead; you will probably want a 
way to cache native code for executables, and that adds some new issues 
into OS design (such as how you securely fetch and update the native 
code translations).

To summarize, is it feasible?  Yes.  Is it a good idea?  I'm not 
certain; it definitely depends on what environment your OS is going to 
run in and what your goals are.
>
> Cheers,
> Shea Levy
>
> P.S. please CC me on replies, I'm not subscribed.
Please note that I had to approve your message to the mailing list.  It 
is better if you subscribe and then disable delivery of messages.  I 
usually only approve messages from non-subscribers once in the morning 
when I receive an email about the pending requests from yesterday.

-- John T.
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Shea,

Feasible? Sure, anything is feasible. A good idea? Not really. The internal lld
file format is an _intermediate_ data structure designed to make creating the
final executable straightforward and fast. It isn't designed to be an
executable format itself. In fact, the reason why linkers can often be slow is
because the intermediate and final object file formats are conflated.

davez


On May 2, 2012, at 8:29 AM, Shea Levy <shea at shealevy.com> wrote:
> Hello,
> 
> Would it be feasible to use the internal lld file format as the native 
> executable format for an OS? Are there performance or space 
> considerations that would make this a poor choice?
> 
> Cheers,
> Shea Levy
> 
> P.S. please CC me on replies, I'm not subscribed.
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

On 05/02/2012 12:59 PM, John Criswell wrote:
> On 5/2/12 10:29 AM, Shea Levy wrote:
>> Hello,
>>
>> Would it be feasible to use the internal lld file format as the native
>> executable format for an OS? Are there performance or space
>> considerations that would make this a poor choice?
>
> By lld, do you mean the LLVM IR bitcode format, or are you referring 
> to something else?
I meant the format used by the nascent lld project (lld.llvm.org). The 
scenario I had in mind would involve native code inside the relevant atoms.
>
> As far as using LLVM IR as a format for executables, there was a 
> research paper on that topic with an older version of LLVM 
> (http://llvm.org/pubs/2003-10-01-LLVA.html).  You'd probably want to 
> redo the experiments on file size and such since LLVM's on-disk format 
> has changed considerably over the years, but it might give you an idea 
> of what issues to consider and what trouble spots there might be.
>
> Also note that there's follow-on work to support an entire OS on the 
> LLVM IR without the inline assembly language feature.  Just look for 
> the publications written by me if you need that.
> :)
>
> The more significant challenge, I think, would be translating LLVM IR 
> to native code on-the-fly.  That'll add overhead; you will probably 
> want a way to cache native code for executables, and that adds some 
> new issues into OS design (such as how you securely fetch and update 
> the native code translations).
>
> To summarize, is it feasible?  Yes.  Is it a good idea?  I'm not 
> certain; it definitely depends on what environment your OS is going to 
> run in and what your goals are.
>
>>
>> Cheers,
>> Shea Levy
>>
>> P.S. please CC me on replies, I'm not subscribed.
>
> Please note that I had to approve your message to the mailing list.  
> It is better if you subscribe and then disable delivery of messages.  
> I usually only approve messages from non-subscribers once in the 
> morning when I receive an email about the pending requests from 
> yesterday.
>
> -- John T.
>
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

On 05/02/2012 01:00 PM, Dave Zarzycki wrote:
> Shea,
>
> Feasible? Sure, anything is feasible. A good idea? Not really. The internal
lld file format is an _intermediate_ data structure designed to make creating
the final executable straightforward and fast. It isn't designed to be an
executable format itself. In fact, the reason why linkers can often be slow is
because the intermediate and final object file formats are conflated.
Hm, OK. Does the target executable format play a role in the 
intermediate steps the linker takes? That is, are there certain features 
or optimizations that only work for specific target formats?
> davez
>
>
> On May 2, 2012, at 8:29 AM, Shea Levy<shea at shealevy.com>  wrote:
>
>> Hello,
>>
>> Would it be feasible to use the internal lld file format as the native
>> executable format for an OS? Are there performance or space
>> considerations that would make this a poor choice?
>>
>> Cheers,
>> Shea Levy
>>
>> P.S. please CC me on replies, I'm not subscribed.
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev