Linux Virtualization - Oct 2018 - [patch 00/11] x86/vdso: Cleanups, simmplifications and CLOCK_TAI support

Andy Lutomirski <luto at amacapital.net> writes:
>> On Oct 3, 2018, at 2:22 AM, Vitaly Kuznetsov <vkuznets at
redhat.com> wrote:
>> 
>> Andy Lutomirski <luto at kernel.org> writes:
>> 
>>> Hi Vitaly, Paolo, Radim, etc.,
>>> 
>> The notification you're talking about exists, it is called
>> Reenligntenment, see 0092e4346f49 "x86/kvm: Support Hyper-V
>> reenlightenment"). When TSC page changes (and this only happens
when L1
>> is migrated to a different host with a different TSC frequency and TSC
>> scaling is not supported by the CPU) we receive an interrupt in L1 (at
>> this moment all TSC accesses are emulated which guarantees the
>> correctness of the readings), pause all L2 guests, update their
kvmclock
>> structures with new data (we already know the new TSC frequency) and
>> then tell L0 that we're done and it can stop emulating TSC
accesses.
>
> That?s delightful!  Does the emulation magic also work for L1 user
> mode?
As far as I understand - yes, all rdtsc* calls will trap into L0.
>  If so, couldn?t we drop the HyperV vclock entirely and just
> fold the adjustment into the core timekeeping data?  (Preferably the
> actual core data, which would require core changes, but it could
> plausibly be done in arch code, too.)
Not all Hyper-V hosts support reenlightenment notifications (and, if I'm
not mistaken, you need to enable nesting for the VM to get the feature -
and most VMs don't have this) so I think we'll have to keep Hyper-V
vclock for the time being.

-- 
Vitaly

> On Oct 3, 2018, at 5:01 AM, Vitaly Kuznetsov <vkuznets at redhat.com>
wrote:
> 
> Andy Lutomirski <luto at amacapital.net> writes:
> 
>>> On Oct 3, 2018, at 2:22 AM, Vitaly Kuznetsov <vkuznets at
redhat.com> wrote:
>>> 
>>> Andy Lutomirski <luto at kernel.org> writes:
>>> 
>>>> Hi Vitaly, Paolo, Radim, etc.,
>>>> 
>>> The notification you're talking about exists, it is called
>>> Reenligntenment, see 0092e4346f49 "x86/kvm: Support Hyper-V
>>> reenlightenment"). When TSC page changes (and this only
happens when L1
>>> is migrated to a different host with a different TSC frequency and
TSC
>>> scaling is not supported by the CPU) we receive an interrupt in L1
(at
>>> this moment all TSC accesses are emulated which guarantees the
>>> correctness of the readings), pause all L2 guests, update their
kvmclock
>>> structures with new data (we already know the new TSC frequency)
and
>>> then tell L0 that we're done and it can stop emulating TSC
accesses.
>> 
>> That?s delightful!  Does the emulation magic also work for L1 user
>> mode?
> 
> As far as I understand - yes, all rdtsc* calls will trap into L0.
> 
>> If so, couldn?t we drop the HyperV vclock entirely and just
>> fold the adjustment into the core timekeeping data?  (Preferably the
>> actual core data, which would require core changes, but it could
>> plausibly be done in arch code, too.)
> 
> Not all Hyper-V hosts support reenlightenment notifications (and, if
I'm
> not mistaken, you need to enable nesting for the VM to get the feature -
> and most VMs don't have this) so I think we'll have to keep Hyper-V
> vclock for the time being.
> 
> 
But this does suggest that the correct way to pass a clock through to an L2
guest where L0 is HV is to make L1 use the ?tsc? clock and L2 use kvmclock (or
something newer and better).  This would require adding support for atomic
frequency changes all the way through the timekeeping and arch code.

John, tglx, would that be okay or crazy?

On Wed, 3 Oct 2018, Andy Lutomirski wrote:
> > On Oct 3, 2018, at 5:01 AM, Vitaly Kuznetsov <vkuznets at
redhat.com> wrote:
> > Not all Hyper-V hosts support reenlightenment notifications (and, if
I'm
> > not mistaken, you need to enable nesting for the VM to get the feature
-
> > and most VMs don't have this) so I think we'll have to keep
Hyper-V
> > vclock for the time being.
> > 
> But this does suggest that the correct way to pass a clock through to an
> L2 guest where L0 is HV is to make L1 use the ?tsc? clock and L2 use
> kvmclock (or something newer and better).  This would require adding
> support for atomic frequency changes all the way through the timekeeping
> and arch code.
>
> John, tglx, would that be okay or crazy?
Not sure what you mean. I think I lost you somewhere on the way.

Thanks,

	tglx