Linux Virtualization - Sep 2014 - Standardizing an MSR or other hypercall to get an RNG seed?

Defining a standard way of transferring random numbers between the host and the
guest is an excellent idea.

As the person who writes the RNG code in Windows, I have a few comments:

DETECTION:
It should be possible to detect this feature through CPUID or similar mechanism.
That allows the code that uses this feature to be written without needing the
ability to catch CPU exceptions. I could be wrong, but as far as I know there is
no support for exception handling in the Windows OS loader where we gather our
initial random state.

EFFICIENCY:
Is there a way we can transfer more bytes per interaction? With a single 64-bit
MSR we always need multiple reads to get a seed, and each of them results in a
context switch to the host, which is expensive. This is even worse for 32-bit
guests. Windows would typically need to fetch 64 bytes of random data at boot
and at regular intervals. It is not a show-stopper, but better efficiency would
be nice.

GUEST-TO-HOST:
Can we also define a way to have random values flow from the guest to the host?
Guests are also gathering entropy from their own sources, and if we allow the
guests to send random data to the host, then the host can treat it as an entropy
source and all the VMs on a single host can share their entropy. (This is not a
security problem; any reasonable host RNG cannot be hurt even by maliciously
chosen entropy inputs.)


I don't know much about how hypervisors work on the inside, but maybe we can
define a mechanism for standardized hypervisor calls that work on all
hypervisors that support this feature. Then we could define a function to do an
entropy exchange: the guest provides N bytes of random data to the host, and the
host replies with N bytes of random data. The data exchange can now be done
through memory.

A standardized hypervisor-call mechanism also seems generally useful for future
features, whereas the MSR solution is very limited in what it can do. We might
end up with standardized hypervisor-calls in the future for some other reason,
and then the MSR solution looks very odd.

Niels


-----Original Message-----
From: H. Peter Anvin [mailto:hpa at zytor.com] 
Sent: Thursday, September 18, 2014 11:39 AM
To: Andy Lutomirski; Nakajima, Jun
Cc: KY Srinivasan; Paolo Bonzini; Mathew John; Theodore Ts'o; John Starks;
kvm list; Gleb Natapov; Niels Ferguson; David Hepkin; Jake Oshins; Linux
Virtualization
Subject: Re: Standardizing an MSR or other hypercall to get an RNG seed?

Quite frankly it might make more sense to define a cross-VM *cpuid* range.  The
cpuid leaf can just point to the MSR.  The big question is who will be willing
to be the registrar.

On September 18, 2014 11:35:39 AM PDT, Andy Lutomirski <luto at
amacapital.net> wrote:
>On Thu, Sep 18, 2014 at 10:42 AM, Nakajima, Jun 
><jun.nakajima at intel.com> wrote:
>> On Thu, Sep 18, 2014 at 10:20 AM, KY Srinivasan <kys at
microsoft.com>
>wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Paolo Bonzini [mailto:paolo.bonzini at gmail.com] On
Behalf Of
>Paolo
>>>> Bonzini
>>>> Sent: Thursday, September 18, 2014 10:18 AM
>>>> To: Nakajima, Jun; KY Srinivasan
>>>> Cc: Mathew John; Theodore Ts'o; John Starks; kvm list; Gleb
>Natapov; Niels
>>>> Ferguson; Andy Lutomirski; David Hepkin; H. Peter Anvin; Jake
>Oshins; Linux
>>>> Virtualization
>>>> Subject: Re: Standardizing an MSR or other hypercall to get an
RNG
>seed?
>>>>
>>>> Il 18/09/2014 19:13, Nakajima, Jun ha scritto:
>>>> > In terms of the address for the MSR, I suggest that you
choose
>one
>>>> > from the range between 40000000H - 400000FFH. The SDM
(35.1
>>>> > ARCHITECTURAL MSRS) says "All existing and future
processors will
>not
>>>> > implement any features using any MSR in this range."
Hyper-V
>already
>>>> > defines many synthetic MSRs in this range, and I think it
would
>be
>>>> > reasonable for you to pick one for this to avoid a
conflict?
>>>>
>>>> KVM is not using any MSR in that range.
>>>>
>>>> However, I think it would be better to have the MSR (and
perhaps
>CPUID)
>>>> outside the hypervisor-reserved ranges, so that it becomes
>architecturally
>>>> defined.  In some sense it is similar to the HYPERVISOR CPUID
>feature.
>>>
>>> Yes, given that we want this to be hypervisor agnostic.
>>>
>>
>> Actually, that MSR address range has been reserved for that purpose,
>along with:
>> - CPUID.EAX=1 -> ECX bit 31 (always returns 0 on bare metal)
>> - CPUID.EAX=4000_00xxH leaves (i.e. HYPERVISOR CPUID)
>
>I don't know whether this is documented anywhere, but Linux tries to 
>detect a hypervisor by searching CPUID leaves 0x400xyz00 for 
>"KVMKVMKVM\0\0\0", so at least Linux can handle the KVM leaves
being in
>a somewhat variable location.
>
>Do we consider this mechanism to work across all hypervisors and 
>guests?  That is, could we put something like "CrossHVPara\0"
>somewhere in that range, where each hypervisor would be free to decide 
>exactly where it ends up?
>
>--Andy
--
Sent from my mobile phone.  Please pardon brevity and lack of formatting.

On Thu, Sep 18, 2014 at 11:54 AM, Niels Ferguson <niels at microsoft.com>
wrote:
> Defining a standard way of transferring random numbers between the host and
the guest is an excellent idea.
>
> As the person who writes the RNG code in Windows, I have a few comments:
>
> DETECTION:
> It should be possible to detect this feature through CPUID or similar
mechanism. That allows the code that uses this feature to be written without
needing the ability to catch CPU exceptions. I could be wrong, but as far as I
know there is no support for exception handling in the Windows OS loader where
we gather our initial random state.
>
Linux is like this, too, except that I have experimental code to
create an IDT in that code, so we can handle it.  I agree, though,
that using CPUID in early boot is easier.
> EFFICIENCY:
> Is there a way we can transfer more bytes per interaction? With a single
64-bit MSR we always need multiple reads to get a seed, and each of them results
in a context switch to the host, which is expensive. This is even worse for
32-bit guests. Windows would typically need to fetch 64 bytes of random data at
boot and at regular intervals. It is not a show-stopper, but better efficiency
would be nice.
I thought about this for a while and didn't come up with anything that
wouldn't messy.  We could fudge the MSR rax/rdx high bits to get 128
bits, but that's nonportable and awful to implement.  We could return
a random number directly from CPUID, but that's weird.

In very informal benchmarking, rdmsr wasn't that bad.  On the other
hand, I wasn't immediately planning on using the msr on an ongoing
basis on Linux guests except after suspend/resume.
>
> GUEST-TO-HOST:
> Can we also define a way to have random values flow from the guest to the
host? Guests are also gathering entropy from their own sources, and if we allow
the guests to send random data to the host, then the host can treat it as an
entropy source and all the VMs on a single host can share their entropy. (This
is not a security problem; any reasonable host RNG cannot be hurt even by
maliciously chosen entropy inputs.)
>
wrmsr on the same MSR?
>
> I don't know much about how hypervisors work on the inside, but maybe
we can define a mechanism for standardized hypervisor calls that work on all
hypervisors that support this feature. Then we could define a function to do an
entropy exchange: the guest provides N bytes of random data to the host, and the
host replies with N bytes of random data. The data exchange can now be done
through memory.
>
> A standardized hypervisor-call mechanism also seems generally useful for
future features, whereas the MSR solution is very limited in what it can do. We
might end up with standardized hypervisor-calls in the future for some other
reason, and then the MSR solution looks very odd.
I think there'll be resistance to a standardized hypercall mechanism,
just because the implementations tend to be complex.  Hyper-V uses a
special page in guest physical memory that contains a trampoline.

We could use wrmsr to a register where the payload is a pointer to a
buffer to receive random bytes, but that loses some of the simplicity
of just calling rdmsr a few times.

--Andy

The chief advantage I see to using a hypercall based mechanism is that it would
work across more architectures.  MSR's and CPUID's are specific to X86. 
If we ever wanted this same mechanism to be available on an architecture that
doesn't support MSR's,  a hypercall based approach would allow for a
more consistent mechanism across the architectures.

I agree, though, that converging on a common hypercall interface that would be
implemented by all of the hypervisors would likely be much harder to achieve.

Thanks...

David

-----Original Message-----
From: Andy Lutomirski [mailto:luto at amacapital.net] 
Sent: Thursday, September 18, 2014 12:04 PM
To: Niels Ferguson
Cc: H. Peter Anvin; Nakajima, Jun; KY Srinivasan; Paolo Bonzini; Mathew John;
Theodore Ts'o; John Starks; kvm list; Gleb Natapov; David Hepkin; Jake
Oshins; Linux Virtualization
Subject: Re: Standardizing an MSR or other hypercall to get an RNG seed?

On Thu, Sep 18, 2014 at 11:54 AM, Niels Ferguson <niels at microsoft.com>
wrote:
> Defining a standard way of transferring random numbers between the host and
the guest is an excellent idea.
>
> As the person who writes the RNG code in Windows, I have a few comments:
>
> DETECTION:
> It should be possible to detect this feature through CPUID or similar
mechanism. That allows the code that uses this feature to be written without
needing the ability to catch CPU exceptions. I could be wrong, but as far as I
know there is no support for exception handling in the Windows OS loader where
we gather our initial random state.
>
Linux is like this, too, except that I have experimental code to create an IDT
in that code, so we can handle it.  I agree, though, that using CPUID in early
boot is easier.
> EFFICIENCY:
> Is there a way we can transfer more bytes per interaction? With a single
64-bit MSR we always need multiple reads to get a seed, and each of them results
in a context switch to the host, which is expensive. This is even worse for
32-bit guests. Windows would typically need to fetch 64 bytes of random data at
boot and at regular intervals. It is not a show-stopper, but better efficiency
would be nice.
I thought about this for a while and didn't come up with anything that
wouldn't messy.  We could fudge the MSR rax/rdx high bits to get 128 bits,
but that's nonportable and awful to implement.  We could return a random
number directly from CPUID, but that's weird.

In very informal benchmarking, rdmsr wasn't that bad.  On the other hand, I
wasn't immediately planning on using the msr on an ongoing basis on Linux
guests except after suspend/resume.
>
> GUEST-TO-HOST:
> Can we also define a way to have random values flow from the guest to 
> the host? Guests are also gathering entropy from their own sources, 
> and if we allow the guests to send random data to the host, then the 
> host can treat it as an entropy source and all the VMs on a single 
> host can share their entropy. (This is not a security problem; any 
> reasonable host RNG cannot be hurt even by maliciously chosen entropy 
> inputs.)
>
wrmsr on the same MSR?
>
> I don't know much about how hypervisors work on the inside, but maybe
we can define a mechanism for standardized hypervisor calls that work on all
hypervisors that support this feature. Then we could define a function to do an
entropy exchange: the guest provides N bytes of random data to the host, and the
host replies with N bytes of random data. The data exchange can now be done
through memory.
>
> A standardized hypervisor-call mechanism also seems generally useful for
future features, whereas the MSR solution is very limited in what it can do. We
might end up with standardized hypervisor-calls in the future for some other
reason, and then the MSR solution looks very odd.
I think there'll be resistance to a standardized hypercall mechanism, just
because the implementations tend to be complex.  Hyper-V uses a special page in
guest physical memory that contains a trampoline.

We could use wrmsr to a register where the payload is a pointer to a buffer to
receive random bytes, but that loses some of the simplicity of just calling
rdmsr a few times.

--Andy