Linux Virtualization - Sep 2018 - [patch 09/11] x86/vdso: Simplify the invalid vclock case

> On Sep 18, 2018, at 3:46 PM, Thomas Gleixner <tglx at linutronix.de>
wrote:
> 
> On Tue, 18 Sep 2018, Andy Lutomirski wrote:
>>> On Sep 18, 2018, at 12:52 AM, Thomas Gleixner <tglx at
linutronix.de> wrote:
>>> 
>>>>> On Mon, 17 Sep 2018, John Stultz wrote:
>>>>> On Mon, Sep 17, 2018 at 12:25 PM, Andy Lutomirski <luto
at kernel.org> wrote:
>>>>> Also, I'm not entirely convinced that this
"last" thing is needed at
>>>>> all.  John, what's the scenario under which we need it?
>>>> 
>>>> So my memory is probably a bit foggy, but I recall that as we
>>>> accelerated gettimeofday, we found that even on systems that
claimed
>>>> to have synced TSCs, they were actually just slightly out of
sync.
>>>> Enough that right after cycles_last had been updated, a read on
>>>> another cpu could come in just behind cycles_last, resulting in
a
>>>> negative interval causing lots of havoc.
>>>> 
>>>> So the sanity check is needed to avoid that case.
>>> 
>>> Your memory serves you right. That's indeed observable on CPUs
which
>>> lack TSC_ADJUST.
>>> 
>>> @Andy: Welcome to the wonderful world of TSC.
>>> 
>> 
>> Do we do better if we use signed arithmetic for the whole calculation?
>> Then a small backwards movement would result in a small backwards
result.
>> Or we could offset everything so that we?d have to go back several
>> hundred ms before we cross zero.
> 
> That would be probably the better solution as signed math would be
> problematic when the resulting ns value becomes negative. As the delta is
> really small, otherwise the TSC sync check would have caught it, the caller
> should never be able to observe time going backwards.
> 
> I'll have a look into that. It needs some thought vs. the fractional
part
> of the base time, but it should be not rocket science to get that
> correct. Famous last words...
> 
It?s also fiddly to tune. If you offset it too much, then the fancy
divide-by-repeated-subtraction loop will hurt more than the comparison to last.

On Tue, 18 Sep 2018, Andy Lutomirski wrote:
> > On Sep 18, 2018, at 3:46 PM, Thomas Gleixner <tglx at
linutronix.de> wrote:
> > On Tue, 18 Sep 2018, Andy Lutomirski wrote:
> >> Do we do better if we use signed arithmetic for the whole
calculation?
> >> Then a small backwards movement would result in a small backwards
result.
> >> Or we could offset everything so that we?d have to go back several
> >> hundred ms before we cross zero.
> > 
> > That would be probably the better solution as signed math would be
> > problematic when the resulting ns value becomes negative. As the delta
is
> > really small, otherwise the TSC sync check would have caught it, the
caller
> > should never be able to observe time going backwards.
> > 
> > I'll have a look into that. It needs some thought vs. the
fractional part
> > of the base time, but it should be not rocket science to get that
> > correct. Famous last words...
> > 
> 
> It?s also fiddly to tune. If you offset it too much, then the fancy
> divide-by-repeated-subtraction loop will hurt more than the comparison to
> last.
Not really. It's sufficient to offset it by at max. 1000 cycles or so. That
won't hurt the magic loop, but it will definitely cover that slight offset
case.

Thanks,

	tglx

On Wed, 19 Sep 2018, Thomas Gleixner wrote:
> On Tue, 18 Sep 2018, Andy Lutomirski wrote:
> > > On Sep 18, 2018, at 3:46 PM, Thomas Gleixner <tglx at
linutronix.de> wrote:
> > > On Tue, 18 Sep 2018, Andy Lutomirski wrote:
> > >> Do we do better if we use signed arithmetic for the whole
calculation?
> > >> Then a small backwards movement would result in a small
backwards result.
> > >> Or we could offset everything so that we?d have to go back
several
> > >> hundred ms before we cross zero.
> > > 
> > > That would be probably the better solution as signed math would
be
> > > problematic when the resulting ns value becomes negative. As the
delta is
> > > really small, otherwise the TSC sync check would have caught it,
the caller
> > > should never be able to observe time going backwards.
> > > 
> > > I'll have a look into that. It needs some thought vs. the
fractional part
> > > of the base time, but it should be not rocket science to get that
> > > correct. Famous last words...
> > > 
> > 
> > It?s also fiddly to tune. If you offset it too much, then the fancy
> > divide-by-repeated-subtraction loop will hurt more than the comparison
to
> > last.
> 
> Not really. It's sufficient to offset it by at max. 1000 cycles or so.
That
> won't hurt the magic loop, but it will definitely cover that slight
offset
> case.
I got it working, but first of all the gain is close to 0.

There is this other subtle issue that we've seen TSCs slowly drifting apart
which is caught by the TSC watchdog eventually, but if it exeeds the offset
_before_ the watchdog triggers, we're back to square one.

So I rather stay on the safe side and just accept that we have to deal with
that. Sigh.

Thanks,

	tglx