llvm dev - Sep 2015 - LLVM as a back end for HHVM

On 9/4/15 1:12 AM, Sanjoy Das via llvm-dev wrote:
> Specifically on "Location records" --
>
> Is it legal for the optimizer to drop the `!locrec` metadata that you
> attach to instructions?  The general convention in LLVM is that
> dropping metadata should not affect correctness, and if the location
> record information is not best-effort or optional then metadata is
> perhaps not the best way to represent it.
Unfortunately not - all of our uses of locrecs are required for correctness.
>
> We're currently developing a scheme called "operand bundles"
(more
> details at [1], patches at [2]) that can be used to tag calls and
> invokes with arbitrary values in a way that that they won't be dropped
> by the optimizer.  The exact semantics of operand bundles are still
> under discussion, and I'd like to make mechanism broadly useful,
> including for things like location records.  So it'd be great if you
> take a look to see if location records can be implemented using
> operand bundles.  I was thinking of something like
>
>    musttail call void @foo(i64 %val) [ "locrec"(i32 42) ]
>
> where the "locrec"(i32 42) gets lowered to some suitable form in
> SelectionDAG.
That sounds like it should work. One of the ideas behind locrecs was 
that they'd work with any instruction, not just call. We currently only 
use locrecs on call/invoke, and I can't think of anything we haven't yet
implemented that would benefit from locrecs on other instructions (that 
may change in the future, of course).
> I'm also curious about HHVM's notion of side exits -- how do you
track
> the abstract state to which you have to exit to?  Our primary use-case
> for operand bundles is to track the abstract state of a thread (the
> "interpreter state") that we need for side exits and asynchronous
code
> invalidation.
All VM state syncing for side exits is explicit in the IR we lower to 
LLVM (as a series of stores on an unlikely path), so we don't need 
anything special from LLVM here. We use locrecs to update our jump 
smashing stubs, so they know the address of the jump that entered the 
stub and should be smashed.

-Brett

On 09/04/2015 11:36 AM, Brett Simmers via llvm-dev
wrote:
> On 9/4/15 1:12 AM, Sanjoy Das via llvm-dev wrote:
>> Specifically on "Location records" --
>>
>> Is it legal for the optimizer to drop the `!locrec` metadata that you
>> attach to instructions?  The general convention in LLVM is that
>> dropping metadata should not affect correctness, and if the location
>> record information is not best-effort or optional then metadata is
>> perhaps not the best way to represent it.
>
> Unfortunately not - all of our uses of locrecs are required for 
> correctness.
This will need to be a function attribute or operand bundle when 
upstreamed then, but that's a pretty simple change to
make.
>
>>
>> We're currently developing a scheme called "operand
bundles" (more
>> details at [1], patches at [2]) that can be used to tag calls and
>> invokes with arbitrary values in a way that that they won't be
dropped
>> by the optimizer.  The exact semantics of operand bundles are still
>> under discussion, and I'd like to make mechanism broadly useful,
>> including for things like location records.  So it'd be great if
you
>> take a look to see if location records can be implemented using
>> operand bundles.  I was thinking of something like
>>
>>    musttail call void @foo(i64 %val) [ "locrec"(i32 42) ]
>>
>> where the "locrec"(i32 42) gets lowered to some suitable form
in
>> SelectionDAG.
>
> That sounds like it should work. One of the ideas behind locrecs was 
> that they'd work with any instruction, not just call. We currently 
> only use locrecs on call/invoke, and I can't think of anything we 
> haven't yet implemented that would benefit from locrecs on other 
> instructions (that may change in the future, of course).
Interesting.  What type of use cases are you imagining for locrecs on 
non-call instructions?  Are you thinking of things like implicit null 
and div-by-zero checks?  (The former is already supported in LLVM 
today.)  Or something else entirely?
>
>> I'm also curious about HHVM's notion of side exits -- how do
you track
>> the abstract state to which you have to exit to?  Our primary use-case
>> for operand bundles is to track the abstract state of a thread (the
>> "interpreter state") that we need for side exits and
asynchronous code
>> invalidation.
>
> All VM state syncing for side exits is explicit in the IR we lower to 
> LLVM (as a series of stores on an unlikely path), so we don't need 
> anything special from LLVM here. We use locrecs to update our jump 
> smashing stubs, so they know the address of the jump that entered the 
> stub and should be smashed.
On the VM state synching side of things, I'm curious about the tradeoffs 
involved in the approach you've used.  I'm guessing from what you said 
that you're essentially pre-reserving a set of allocas for the spill 
locations, somehow registering those with your runtime once, then 
emitting stores down the unlikely path into those allocas.  Is that 
roughly right?

How are you handling things like constants and duplicate values 
appearing in the VM state?  Our experience has been that constants are 
fairly common and so are duplicate values (particularly when combined 
with GC state).  It would seem like your frame sizes would be inflated 
if you had to pre-reserve space for each constant and each copy of a 
value.  Have you found this to be true?  If so, has it been problematic 
for you?

Philip

On 9/8/15, 9:35 AM, "Philip Reames" <listmail at
philipreames.com> wrote:
>On 09/04/2015 11:36 AM, Brett Simmers via llvm-dev wrote:
>> On 9/4/15 1:12 AM, Sanjoy Das via llvm-dev wrote:
>>> Specifically on "Location records" --
>>>
>>> Is it legal for the optimizer to drop the `!locrec` metadata that
you
>>> attach to instructions?  The general convention in LLVM is that
>>> dropping metadata should not affect correctness, and if the
location
>>> record information is not best-effort or optional then metadata is
>>> perhaps not the best way to represent it.
>>
>> Unfortunately not - all of our uses of locrecs are required for
>> correctness.
>This will need to be a function attribute or operand bundle when
>upstreamed then, but that's a pretty simple change to make.
I think switching from metadata to operand bundles wouldn't be a problem,
assuming ³locrec² operand will have no effect on optimizations and
codegen.
>>
>>>
>>> We're currently developing a scheme called "operand
bundles" (more
>>> details at [1], patches at [2]) that can be used to tag calls and
>>> invokes with arbitrary values in a way that that they won't be
dropped
>>> by the optimizer.  The exact semantics of operand bundles are still
>>> under discussion, and I'd like to make mechanism broadly
useful,
>>> including for things like location records.  So it'd be great
if you
>>> take a look to see if location records can be implemented using
>>> operand bundles.  I was thinking of something like
>>>
>>>    musttail call void @foo(i64 %val) [ "locrec"(i32 42) ]
>>>
>>> where the "locrec"(i32 42) gets lowered to some suitable
form in
>>> SelectionDAG.
>>
>> That sounds like it should work. One of the ideas behind locrecs was
>> that they'd work with any instruction, not just call. We currently
>> only use locrecs on call/invoke, and I can't think of anything we
>> haven't yet implemented that would benefit from locrecs on other
>> instructions (that may change in the future, of course).
>Interesting.  What type of use cases are you imagining for locrecs on
>non-call instructions?  Are you thinking of things like implicit null
>and div-by-zero checks?  (The former is already supported in LLVM
>today.)  Or something else entirely?
One possible scenario is locating a constant address generation, e.g. for
an
indirect call destination. It¹s rather hypothetical example,
as we don¹t use it in this way at the moment. Substituting such address
isn¹t quite straightforward as the value could be scattered across several
instructions on some architectures, or be placed in a data section.

In general, we found locrecs useful for annotating IR and exploring
resulting
assembly. You could mark instruction in the IR, and the assembly dump will
include annotations showing all machine instructions generated from it.
However, this particular feature is orthogonal to our JIT requirements
and could go in separately if there¹s enough interest.

Maksim

On 9/8/15 9:35 AM, Philip Reames wrote:
> On the VM state synching side of things, I'm curious about the
tradeoffs
> involved in the approach you've used.  I'm guessing from what you
said
> that you're essentially pre-reserving a set of allocas for the spill
> locations, somehow registering those with your runtime once, then
> emitting stores down the unlikely path into those allocas.  Is that
> roughly right?
It's even simpler than that: the stores to spill VM state go directly to 
the VM locations where those values belong. We only ever side-exit at 
bytecode boundaries, so every live value has a place to live on the eval 
stack or in a local variable (every PHP function has a fixed number of 
local variables, and space for those is allocated on the eval stack). 
This means that the side-exit path doesn't have to know if the next 
bytecode is going to executed by the interpreter or by more jitted code, 
since they'll both read values from the same locations.

There are some downsides to this, of course, and we've been thinking 
about ways to have a faster ABI between snippets of jitted code, like 
passing the top n elements of the eval stack in registers. But we have 
no concrete plans to do that in the near future.
> How are you handling things like constants and duplicate values
> appearing in the VM state?  Our experience has been that constants are
> fairly common and so are duplicate values (particularly when combined
> with GC state).  It would seem like your frame sizes would be inflated
> if you had to pre-reserve space for each constant and each copy of a
> value.  Have you found this to be true?  If so, has it been problematic
> for you?
If I'm understanding this question correctly it doesn't apply to our 
situation given my answer to the previous question, but let me know if 
that's not the case and I can try to expand :). We don't currently have 
a GC - everything is done using reference counting, though we do have a 
few people working on changing that.

-Brett