Linux Virtualization - Sep 2014 - Standardizing an MSR or other hypercall to get an RNG seed?

On Thu, Sep 18, 2014 at 11:58 AM, Paolo Bonzini <pbonzini at redhat.com>
wrote:
>
>> > Actually, that MSR address range has been reserved for that
purpose, along
>> > with:
>> > - CPUID.EAX=1 -> ECX bit 31 (always returns 0 on bare metal)
>> > - CPUID.EAX=4000_00xxH leaves (i.e. HYPERVISOR CPUID)
>>
>> I don't know whether this is documented anywhere, but Linux tries
to
>> detect a hypervisor by searching CPUID leaves 0x400xyz00 for
>> "KVMKVMKVM\0\0\0", so at least Linux can handle the KVM
leaves being
>> in a somewhat variable location.
>>
>> Do we consider this mechanism to work across all hypervisors and
>> guests?  That is, could we put something like "CrossHVPara\0"
>> somewhere in that range, where each hypervisor would be free to decide
>> exactly where it ends up?
>
> That's also possible, but extending the hypervisor CPUID range
> beywond 400000FFH is not officially sanctioned by Intel.
>
> Xen started doing that in order to expose both Hyper-V and Xen
> CPUID leaves, and KVM followed the practice.
>
Whoops.

Might Intel be willing to extend that range to 0x40000000 -
0x400fffff?  And would Microsoft be okay with using this mechanism for
discovery?

Do we have anyone from VMware in this thread?  I don't have any VMware
contacts.

--Andy

On Thu, Sep 18, 2014 at 12:07 PM, Andy Lutomirski <luto at amacapital.net>
wrote:
> Might Intel be willing to extend that range to 0x40000000 -
> 0x400fffff?  And would Microsoft be okay with using this mechanism for
> discovery?
So, for CPUID, the SDM (Table 3-17. Information Returned by CPUID) says today:
"No existing or future CPU will return processor identification or
feature information if the initial EAX value is in the range 40000000H
to 4FFFFFFFH."

We can define a cross-VM CPUID range from there. The CPUID can return
the index of the MSR if needed.

-- 
Jun
Intel Open Source Technology Center

On Thu, Sep 18, 2014 at 2:21 PM, Nakajima, Jun <jun.nakajima at intel.com>
wrote:
> On Thu, Sep 18, 2014 at 12:07 PM, Andy Lutomirski <luto at
amacapital.net> wrote:
>
>> Might Intel be willing to extend that range to 0x40000000 -
>> 0x400fffff?  And would Microsoft be okay with using this mechanism for
>> discovery?
>
> So, for CPUID, the SDM (Table 3-17. Information Returned by CPUID) says
today:
> "No existing or future CPU will return processor identification or
> feature information if the initial EAX value is in the range 40000000H
> to 4FFFFFFFH."
>
> We can define a cross-VM CPUID range from there. The CPUID can return
> the index of the MSR if needed.
Right, sorry.  I was looking at this sentence in SDM Volume 3 Section 35.1:

MSR address range between 40000000H - 400000FFH is marked as a
specially reserved range. All existing and
future processors will not implement any features using any MSR in this range.

That's not really a large enough range for us to reserve an MSR for
this.  However, KVM, is already using MSRs outside that range: it uses
0x4b564d00-0x4b564d04 or so.  I wonder whether KVM got confused by the
differing ranges for cpuid leaves and MSR indices.

Any chance that Intel could reserve a larger range to include the KVM
MSRs?  It would also be easier if the MSR indices for cross-HV
features were constants.

Thanks,
Andy

I'm not sure what you mean by "this mechanism?"  Are you
suggesting that each hypervisor put "CrossHVPara\0" somewhere in the
0x40000000 - 0x400fffff CPUID range, and an OS has to do a full scan of this
CPUID range on boot to find it?  That seems pretty inefficient.  An OS will take
1000's of hypervisor intercepts on every boot just to search this CPUID
range.

I suggest we come to consensus on a specific CPUID leaf where an OS needs to
look to determine if a hypervisor supports this capability.  We could define a
new CPUID leaf range at a well-defined location, or we could just use one of the
existing CPUID leaf ranges implemented by an existing hypervisor.  I'm not
familiar with the KVM CPUID leaf range, but in the case of Hyper-V, the Hyper-V
CPUID leaf range was architected to allow for other hypervisors to implement it
and just show through specific capabilities supported by the hypervisor.  So, we
could define a bit in the Hyper-V CPUID leaf range (since Xen and KVM also
implement this range), but that would require Linux to look in that range on
boot to discover this capability.

Thanks...

David

-----Original Message-----
From: Andy Lutomirski [mailto:luto at amacapital.net] 
Sent: Thursday, September 18, 2014 12:07 PM
To: Paolo Bonzini
Cc: Jun Nakajima; KY Srinivasan; Mathew John; Theodore Ts'o; John Starks;
kvm list; Gleb Natapov; Niels Ferguson; David Hepkin; H. Peter Anvin; Jake
Oshins; Linux Virtualization
Subject: Re: Standardizing an MSR or other hypercall to get an RNG seed?

On Thu, Sep 18, 2014 at 11:58 AM, Paolo Bonzini <pbonzini at redhat.com>
wrote:
>
>> > Actually, that MSR address range has been reserved for that 
>> > purpose, along
>> > with:
>> > - CPUID.EAX=1 -> ECX bit 31 (always returns 0 on bare metal)
>> > - CPUID.EAX=4000_00xxH leaves (i.e. HYPERVISOR CPUID)
>>
>> I don't know whether this is documented anywhere, but Linux tries
to
>> detect a hypervisor by searching CPUID leaves 0x400xyz00 for 
>> "KVMKVMKVM\0\0\0", so at least Linux can handle the KVM
leaves being
>> in a somewhat variable location.
>>
>> Do we consider this mechanism to work across all hypervisors and 
>> guests?  That is, could we put something like "CrossHVPara\0"
>> somewhere in that range, where each hypervisor would be free to 
>> decide exactly where it ends up?
>
> That's also possible, but extending the hypervisor CPUID range beywond 
> 400000FFH is not officially sanctioned by Intel.
>
> Xen started doing that in order to expose both Hyper-V and Xen CPUID 
> leaves, and KVM followed the practice.
>
Whoops.

Might Intel be willing to extend that range to 0x40000000 - 0x400fffff?  And
would Microsoft be okay with using this mechanism for discovery?

Do we have anyone from VMware in this thread?  I don't have any VMware
contacts.

--Andy

On 09/18/2014 02:46 PM, David Hepkin wrote:
> I'm not sure what you mean by "this mechanism?"  Are you
suggesting that each hypervisor put "CrossHVPara\0" somewhere in the
0x40000000 - 0x400fffff CPUID range, and an OS has to do a full scan of this
CPUID range on boot to find it?  That seems pretty inefficient.  An OS will take
1000's of hypervisor intercepts on every boot just to search this CPUID
range.
> 
> I suggest we come to consensus on a specific CPUID leaf where an OS needs
to look to determine if a hypervisor supports this capability.  We could define
a new CPUID leaf range at a well-defined location, or we could just use one of
the existing CPUID leaf ranges implemented by an existing hypervisor.  I'm
not familiar with the KVM CPUID leaf range, but in the case of Hyper-V, the
Hyper-V CPUID leaf range was architected to allow for other hypervisors to
implement it and just show through specific capabilities supported by the
hypervisor.  So, we could define a bit in the Hyper-V CPUID leaf range (since
Xen and KVM also implement this range), but that would require Linux to look in
that range on boot to discover this capability.
> 
Yes, I would agree that if anything we should define a new range unique
to this cross-VM interface, e.g. 0x48000000.

	-hpa

On Thu, Sep 18, 2014 at 2:46 PM, David Hepkin <davidhep at microsoft.com>
wrote:
> I'm not sure what you mean by "this mechanism?"  Are you
suggesting that each hypervisor put "CrossHVPara\0" somewhere in the
0x40000000 - 0x400fffff CPUID range, and an OS has to do a full scan of this
CPUID range on boot to find it?  That seems pretty inefficient.  An OS will take
1000's of hypervisor intercepts on every boot just to search this CPUID
range.
Linux already does this, which is arguably unfortunate.  But it's not
quite that bad; the KVM and Xen code is only scanning at increments of
0x100.

I think that Linux as a guest would have no problem with checking the
Hyper-V range or some new range.  I don't think that Linux would want
to have to set a guest OS identity, and it's not entirely clear to me
whether this would be necessary to use the Hyper-V mechanism.
>
> I suggest we come to consensus on a specific CPUID leaf where an OS needs
to look to determine if a hypervisor supports this capability.  We could define
a new CPUID leaf range at a well-defined location, or we could just use one of
the existing CPUID leaf ranges implemented by an existing hypervisor.  I'm
not familiar with the KVM CPUID leaf range, but in the case of Hyper-V, the
Hyper-V CPUID leaf range was architected to allow for other hypervisors to
implement it and just show through specific capabilities supported by the
hypervisor.  So, we could define a bit in the Hyper-V CPUID leaf range (since
Xen and KVM also implement this range), but that would require Linux to look in
that range on boot to discover this capability.
I also don't know whether QEMU and KVM would be okay with implementing
the host side of the Hyper-V mechanism by default.  They would have to
implement at least leaves 0x40000001 and 0x4000002, plus correctly
reporting zeros through whatever leaf is used for this new feature.
Gleb?  Paolo?

--Andy