Virtualization - Sep 2014 - Standardizing an MSR or other hypercall to get an RNG seed?

I'm not sure what you mean by "this mechanism?"  Are you
suggesting that each hypervisor put "CrossHVPara\0" somewhere in the
0x40000000 - 0x400fffff CPUID range, and an OS has to do a full scan of this
CPUID range on boot to find it?  That seems pretty inefficient.  An OS will take
1000's of hypervisor intercepts on every boot just to search this CPUID
range.

I suggest we come to consensus on a specific CPUID leaf where an OS needs to
look to determine if a hypervisor supports this capability.  We could define a
new CPUID leaf range at a well-defined location, or we could just use one of the
existing CPUID leaf ranges implemented by an existing hypervisor.  I'm not
familiar with the KVM CPUID leaf range, but in the case of Hyper-V, the Hyper-V
CPUID leaf range was architected to allow for other hypervisors to implement it
and just show through specific capabilities supported by the hypervisor.  So, we
could define a bit in the Hyper-V CPUID leaf range (since Xen and KVM also
implement this range), but that would require Linux to look in that range on
boot to discover this capability.

Thanks...

David

-----Original Message-----
From: Andy Lutomirski [mailto:luto at amacapital.net] 
Sent: Thursday, September 18, 2014 12:07 PM
To: Paolo Bonzini
Cc: Jun Nakajima; KY Srinivasan; Mathew John; Theodore Ts'o; John Starks;
kvm list; Gleb Natapov; Niels Ferguson; David Hepkin; H. Peter Anvin; Jake
Oshins; Linux Virtualization
Subject: Re: Standardizing an MSR or other hypercall to get an RNG seed?

On Thu, Sep 18, 2014 at 11:58 AM, Paolo Bonzini <pbonzini at redhat.com>
wrote:
>
>> > Actually, that MSR address range has been reserved for that 
>> > purpose, along
>> > with:
>> > - CPUID.EAX=1 -> ECX bit 31 (always returns 0 on bare metal)
>> > - CPUID.EAX=4000_00xxH leaves (i.e. HYPERVISOR CPUID)
>>
>> I don't know whether this is documented anywhere, but Linux tries
to
>> detect a hypervisor by searching CPUID leaves 0x400xyz00 for 
>> "KVMKVMKVM\0\0\0", so at least Linux can handle the KVM
leaves being
>> in a somewhat variable location.
>>
>> Do we consider this mechanism to work across all hypervisors and 
>> guests?  That is, could we put something like "CrossHVPara\0"
>> somewhere in that range, where each hypervisor would be free to 
>> decide exactly where it ends up?
>
> That's also possible, but extending the hypervisor CPUID range beywond 
> 400000FFH is not officially sanctioned by Intel.
>
> Xen started doing that in order to expose both Hyper-V and Xen CPUID 
> leaves, and KVM followed the practice.
>
Whoops.

Might Intel be willing to extend that range to 0x40000000 - 0x400fffff?  And
would Microsoft be okay with using this mechanism for discovery?

Do we have anyone from VMware in this thread?  I don't have any VMware
contacts.

--Andy

On 09/18/2014 03:00 PM, Andy Lutomirski wrote:
> On Thu, Sep 18, 2014 at 2:46 PM, David Hepkin <davidhep at
microsoft.com> wrote:
>> I'm not sure what you mean by "this mechanism?"  Are you
suggesting that each hypervisor put "CrossHVPara\0" somewhere in the
0x40000000 - 0x400fffff CPUID range, and an OS has to do a full scan of this
CPUID range on boot to find it?  That seems pretty inefficient.  An OS will take
1000's of hypervisor intercepts on every boot just to search this CPUID
range.
> 
> Linux already does this, which is arguably unfortunate.  But it's not
> quite that bad; the KVM and Xen code is only scanning at increments of
> 0x100.
> 
> I think that Linux as a guest would have no problem with checking the
> Hyper-V range or some new range.  I don't think that Linux would want
> to have to set a guest OS identity, and it's not entirely clear to me
> whether this would be necessary to use the Hyper-V mechanism.
> 
We really don't want to have to do this in early code, though.
>>
>> I suggest we come to consensus on a specific CPUID leaf where an OS
needs to look to determine if a hypervisor supports this capability.  We could
define a new CPUID leaf range at a well-defined location, or we could just use
one of the existing CPUID leaf ranges implemented by an existing hypervisor. 
I'm not familiar with the KVM CPUID leaf range, but in the case of Hyper-V,
the Hyper-V CPUID leaf range was architected to allow for other hypervisors to
implement it and just show through specific capabilities supported by the
hypervisor.  So, we could define a bit in the Hyper-V CPUID leaf range (since
Xen and KVM also implement this range), but that would require Linux to look in
that range on boot to discover this capability.
> 
> I also don't know whether QEMU and KVM would be okay with implementing
> the host side of the Hyper-V mechanism by default.  They would have to
> implement at least leaves 0x40000001 and 0x4000002, plus correctly
> reporting zeros through whatever leaf is used for this new feature.
> Gleb?  Paolo?
> 
The problem is what happens with a noncooperating hypervisor.  I guess
we could put a magic number in one of the leaf registers, but still...

	-hpa

On Thu, Sep 18, 2014 at 2:57 PM, H. Peter Anvin <hpa at zytor.com>
wrote:
> On 09/18/2014 02:46 PM, David Hepkin wrote:
>> I'm not sure what you mean by "this mechanism?"  Are you
suggesting that each hypervisor put "CrossHVPara\0" somewhere in the
0x40000000 - 0x400fffff CPUID range, and an OS has to do a full scan of this
CPUID range on boot to find it?  That seems pretty inefficient.  An OS will take
1000's of hypervisor intercepts on every boot just to search this CPUID
range.
>>
>> I suggest we come to consensus on a specific CPUID leaf where an OS
needs to look to determine if a hypervisor supports this capability.  We could
define a new CPUID leaf range at a well-defined location, or we could just use
one of the existing CPUID leaf ranges implemented by an existing hypervisor. 
I'm not familiar with the KVM CPUID leaf range, but in the case of Hyper-V,
the Hyper-V CPUID leaf range was architected to allow for other hypervisors to
implement it and just show through specific capabilities supported by the
hypervisor.  So, we could define a bit in the Hyper-V CPUID leaf range (since
Xen and KVM also implement this range), but that would require Linux to look in
that range on boot to discover this capability.
>>
>
> Yes, I would agree that if anything we should define a new range unique
> to this cross-VM interface, e.g. 0x48000000.
So, as a concrete straw-man:

CPUID leaf 0x48000000 would return a maximum leaf number in EAX (e.g.
0x48000001) along with a signature value (e.g. "CrossHVPara\0") in
EBX, ECX, and EDX.

CPUID 0x48000001.EAX would contain an MSR number to read to get a
random number if supported and zero if not supported.

Questions:

1. Can we use a fixed MSR number?  This would be a little bit simpler,
but it would depend on getting a wider MSR range from Intel.

2. Who would host and maintain such a spec?  I could do it on github,
but this seems a bit silly.  Other options would include Intel,
Microsoft, or perhaps the Linux Foundation.  I don't know whether
Intel or LF would want to do this, and MS isn't exactly
vendor-neutral.  (Even L-F isn't entirely neutral, since they sort of
represent two hypervisors.)  Or we could do something temporary and
then try to work with a group like OASIS, but that might end up being
a lot of work.

--Andy

On Thu, Sep 18, 2014 at 03:00:05PM -0700, Andy Lutomirski
wrote:
> On Thu, Sep 18, 2014 at 2:46 PM, David Hepkin <davidhep at
microsoft.com> wrote:
> > I suggest we come to consensus on a specific CPUID leaf where an OS
needs to look to determine if a hypervisor supports this capability.  We could
define a new CPUID leaf range at a well-defined location, or we could just use
one of the existing CPUID leaf ranges implemented by an existing hypervisor. 
I'm not familiar with the KVM CPUID leaf range, but in the case of Hyper-V,
the Hyper-V CPUID leaf range was architected to allow for other hypervisors to
implement it and just show through specific capabilities supported by the
hypervisor.  So, we could define a bit in the Hyper-V CPUID leaf range (since
Xen and KVM also implement this range), but that would require Linux to look in
that range on boot to discover this capability.
> 
> I also don't know whether QEMU and KVM would be okay with implementing
> the host side of the Hyper-V mechanism by default.  They would have to
> implement at least leaves 0x40000001 and 0x4000002, plus correctly
> reporting zeros through whatever leaf is used for this new feature.
> Gleb?  Paolo?
> 
KVM and any other hypervisor out there already implement capability
detection mechanism in 0x40000000 range, and of course all of them do
it differently. Linux detects what hypervior it runs on very early and
switch to correspondent code to handle each hypervisor. Quite frankly
I do not see what problem you are trying to fix with standardizing MSR
to get RND and detection mechanism for this MSR. RND MSR is in no way
unique here. There are other mechanisms that are virtually identical
between hypervisors but have different gust/hypervisor interfaces and
are detected differently on different hypervisors. Examples are pvclock,
pveoi may be others.

--
			Gleb.