llvm dev - Aug 2013 - [LLVMdev] Building a stable bitcode format for PNaCl - based on LLVM IR

Hello,

[The first paragraph is safe to skip if you already know what PNaCl is.]
The Portable Native Client (PNaCl) project is a toolchain for producing
portable bitcode from C and C++ code and running in securely and
efficiently on the web via Native Client. For more details see this
presentation from the last Google I/O:
https://developers.google.com/events/io/sessions/325679543and
http://www.chromium.org/nativeclient/pnacl/building-and-testing-portable-native-client

PNaCl uses a subset of LLVM IR as its bitcode. Our goal is a single bitcode
file that can be "translated" on a target machine to a sandboxed
native
executable for the target architecture and executed. This presents a number
of challenges with architecture independence and backwards compatibility.

This is a document we've been using internally to coordinate the effort to
simplify LLVM IR to the level where it's suitable to serve as a portable,
backwards compatible bitcode. After a general introduction, it presents
concrete steps the PNaCl toolchain performs to simplify LLVM IR, with some
discussion of their pros/cons. This is based on a few years of observing
changes in LLVM IR and their meaning for PNaCl.

We've considered the points made by Dan in his "LLVM IR is a compiler
IR"
post (http://lists.cs.uiuc.edu/pipermail/llvmdev/2011-October/043719.html)
and have also discussed this with him a couple of times since then. We
believe that the changes described by the attached document, together with
other PNaCl-specific characteristics, make the chosen subset suitable for
the task.

Any comments and questions are very welcome. Our goal in publishing this is
to facilitate an open discussion, as well as serve as a reference point for
other projects interested in using LLVM IR for portable and/or stable use
cases. Note that the document is a work-in-progress and some details may
change. We intend to publish a more structured reference for PNaCl bitcode
at some point in the future.

The document is available as a PDF here:
https://docs.google.com/a/chromium.org/viewer?a=v&pid=sites&srcid=Y2hyb21pdW0ub3JnfGRldnxneDo0OWYwZjVkYWFjOWNjODE1

And in text here:
https://sites.google.com/a/chromium.org/dev/nativeclient/pnacl/stability-of-the-pnacl-bitcode-abi

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130618/1d003f7a/attachment.html>

Instead of a blacklist, why not a whitelist? Given the size of LangRef,
you're bound to leave something out of your blacklist that needs to be
there (also, future additions to LLVM IR will need to be added to the
blacklist; are you sure you can catch *all* of them?). A whitelist seems
much less prone to breakage or unexpected behavior.

-- Sean Silva
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130618/20f45018/attachment.html>

On Tue, Jun 18, 2013 at 12:03 PM, Sean Silva <silvas at purdue.edu> wrote:
> Instead of a blacklist, why not a whitelist? Given the size of LangRef,
> you're bound to leave something out of your blacklist that needs to be
> there (also, future additions to LLVM IR will need to be added to the
> blacklist; are you sure you can catch *all* of them?). A whitelist seems
> much less prone to breakage or unexpected behavior.
>
>
Hi Sean,

Which blacklist are you referring to? In all places where we specifically
allow or disallow certain constructs (such as specific instructions,
intrinsics, linkage modes and so on) we use a whitelisting strategy.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130618/56c04fdd/attachment.html>

Is it possible to use PNaCl infrastructure (i.e. translation and 
execution in a sandbox) without a Chrome ?

I mean a something like a standalone VM like Java or Mono/C#.

Dmitri

Am 18.06.2013 17:22, schrieb Eli Bendersky:
> Hello,
>
> [The first paragraph is safe to skip if you already know what PNaCl is.]
> The Portable Native Client (PNaCl) project is a toolchain for producing
> portable bitcode from C and C++ code and running in securely and
> efficiently on the web via Native Client. For more details see this
> presentation from the last Google I/O:
> https://developers.google.com/events/io/sessions/325679543and
>
http://www.chromium.org/nativeclient/pnacl/building-and-testing-portable-native-client
>
> PNaCl uses a subset of LLVM IR as its bitcode. Our goal is a single
> bitcode file that can be "translated" on a target machine to a
sandboxed
> native executable for the target architecture and executed. This
> presents a number of challenges with architecture independence and
> backwards compatibility.
>
> This is a document we've been using internally to coordinate the effort
> to simplify LLVM IR to the level where it's suitable to serve as a
> portable, backwards compatible bitcode. After a general introduction, it
> presents concrete steps the PNaCl toolchain performs to simplify LLVM
> IR, with some discussion of their pros/cons. This is based on a few
> years of observing changes in LLVM IR and their meaning for PNaCl.
>
> We've considered the points made by Dan in his "LLVM IR is a
compiler
> IR" post
> (http://lists.cs.uiuc.edu/pipermail/llvmdev/2011-October/043719.html)
> and have also discussed this with him a couple of times since then. We
> believe that the changes described by the attached document, together
> with other PNaCl-specific characteristics, make the chosen subset
> suitable for the task.
>
> Any comments and questions are very welcome. Our goal in publishing this
> is to facilitate an open discussion, as well as serve as a reference
> point for other projects interested in using LLVM IR for portable and/or
> stable use cases. Note that the document is a work-in-progress and some
> details may change. We intend to publish a more structured reference for
> PNaCl bitcode at some point in the future.
>
> The document is available as a PDF here:
>
https://docs.google.com/a/chromium.org/viewer?a=v&pid=sites&srcid=Y2hyb21pdW0ub3JnfGRldnxneDo0OWYwZjVkYWFjOWNjODE1
>
> And in text here:
>
https://sites.google.com/a/chromium.org/dev/nativeclient/pnacl/stability-of-the-pnacl-bitcode-abi
>
> Eli
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>

On 18 June 2013 15:27, Dmitri Rubinstein
<dmitri.rubinstein at googlemail.com>wrote:
> Is it possible to use PNaCl infrastructure (i.e. translation and execution
> in a sandbox) without a Chrome ?
>
> I mean a something like a standalone VM like Java or Mono/C#.
>
Yes.  The NaCl tool 'sel_ldr' will run a program inside a sandbox
outside
of the web browser.  We do a lot of the testing of PNaCl this way.

Cheers,
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130618/83bee129/attachment.html>

> The document is available as a PDF here:
>
https://docs.google.com/a/chromium.org/viewer?a=v&pid=sites&srcid=Y2hyb21pdW0ub3JnfGRldnxneDo0OWYwZjVkYWFjOWNjODE1
>
> And in text here:
>
https://sites.google.com/a/chromium.org/dev/nativeclient/pnacl/stability-of-the-pnacl-bitcode-abi
First, sorry for jumping in an old thread. I had this marked but had
not been able to read it before.

The document talks about ABI stability, but it is a bit unclear how
far this goes. From older discussions, there are/were 3 interesting
cases

1 Loading a stand alone application (pexe) in a new VM/browser.
2 Using a native library compiled with SDK N in an IR application
compiled with  SDK M.
3 Using a IR library compiled with SDK N in an IR application compiled
with SDK M.

The document discusses 1. Items 2 and 3 are probably harder, but not
mentioned. Is there any support for user provided dynamic libraries?

Cheers,
Rafael

On Fri, Aug 9, 2013 at 9:12 AM, Rafael Espíndola <rafael.espindola at
gmail.com
> wrote:
> > The document is available as a PDF here:
> >
>
https://docs.google.com/a/chromium.org/viewer?a=v&pid=sites&srcid=Y2hyb21pdW0ub3JnfGRldnxneDo0OWYwZjVkYWFjOWNjODE1
> >
> > And in text here:
> >
>
https://sites.google.com/a/chromium.org/dev/nativeclient/pnacl/stability-of-the-pnacl-bitcode-abi
>
> First, sorry for jumping in an old thread. I had this marked but had
> not been able to read it before.
>
> The document talks about ABI stability, but it is a bit unclear how
> far this goes. From older discussions, there are/were 3 interesting
> cases
>
> 1 Loading a stand alone application (pexe) in a new VM/browser.
> 2 Using a native library compiled with SDK N in an IR application
> compiled with  SDK M.
> 3 Using a IR library compiled with SDK N in an IR application compiled
> with SDK M.
>
> The document discusses 1. Items 2 and 3 are probably harder, but not
> mentioned. Is there any support for user provided dynamic libraries?
>
Hi Rafael,

In the first release, there is no support for user-provided dynamic
libraries. The user is expected to compile her whole application statically
into a single .pexe and distribute that. We do plan to add "dynamic"
libraries in future releases, and we tried to construct the ABI spec in a
way that would not interfere with these plans.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20130809/13a823da/attachment.html>