search for: gtod

...CLOCK_MAX sanity check - Remove the magic clock masking and extend the storage array Thanks, tglx 8<------------------- arch/x86/Kconfig | 1 arch/x86/entry/vdso/vclock_gettime.c | 199 ++++++++------------------------ arch/x86/entry/vsyscall/vsyscall_gtod.c | 55 ++++---- arch/x86/include/asm/vgtod.h | 42 +++--- arch/x86/kernel/time.c | 22 +++ include/linux/clocksource.h | 5 kernel/time/Kconfig | 4 kernel/time/clocksource.c | 2 8 files changed, 140 inser...

...s code. The resulting performance depends heavily on the micro architecture and the compiler. Thanks, tglx 8<------------------- arch/x86/Kconfig | 1 arch/x86/entry/vdso/vclock_gettime.c | 199 ++++++++------------------------ arch/x86/entry/vsyscall/vsyscall_gtod.c | 55 ++++---- arch/x86/include/asm/vgtod.h | 46 ++++--- arch/x86/kernel/time.c | 22 +++ include/linux/clocksource.h | 5 kernel/time/Kconfig | 4 kernel/time/clocksource.c | 2 8 files changed, 144 inse...

...s code. The resulting performance depends heavily on the micro architecture and the compiler. Thanks, tglx 8<------------------- arch/x86/Kconfig | 1 arch/x86/entry/vdso/vclock_gettime.c | 199 ++++++++------------------------ arch/x86/entry/vsyscall/vsyscall_gtod.c | 55 ++++---- arch/x86/include/asm/vgtod.h | 46 ++++--- arch/x86/kernel/time.c | 22 +++ include/linux/clocksource.h | 5 kernel/time/Kconfig | 4 kernel/time/clocksource.c | 2 8 files changed, 144 inse...

...> On Tue, 18 Sep 2018, Thomas Gleixner wrote: > > > On Tue, 18 Sep 2018, Peter Zijlstra wrote: > > > > > Your memory serves you right. That's indeed observable on CPUs which > > > > > lack TSC_ADJUST. > > > > > > > > But, if the gtod code can observe this, then why doesn't the code that > > > > checks the sync? > > > > > > Because it depends where the involved CPUs are in the topology. The sync > > > code might just run on the same package an simply not see it. Yes, w/o > > >...

...> On Tue, 18 Sep 2018, Thomas Gleixner wrote: > > > On Tue, 18 Sep 2018, Peter Zijlstra wrote: > > > > > Your memory serves you right. That's indeed observable on CPUs which > > > > > lack TSC_ADJUST. > > > > > > > > But, if the gtod code can observe this, then why doesn't the code that > > > > checks the sync? > > > > > > Because it depends where the involved CPUs are in the topology. The sync > > > code might just run on the same package an simply not see it. Yes, w/o > > >...

Dereferencing gtod->cycle_last all over the place and foing the cycles < last comparison in the vclock read functions generates horrible code. Doing it at the call site is much better and gains a few cycles both for TSC and pvclock. Caveat: This adds the comparison to the hyperv vclock as well, but I have no w...

The code flow for the vclocks is convoluted as it requires the vclocks which can be invalidated separately from the vsyscall_gtod_data sequence to store the fact in a separate variable. That's inefficient. Restructure the code so the vclock readout returns cycles and the conversion to nanoseconds is handled at the call site. If the clock gets invalidated or vclock is already VCLOCK_NONE, return U64_MAX as the cycle valu...

On Tue, 18 Sep 2018, Thomas Gleixner wrote: > On Tue, 18 Sep 2018, Peter Zijlstra wrote: > > > Your memory serves you right. That's indeed observable on CPUs which > > > lack TSC_ADJUST. > > > > But, if the gtod code can observe this, then why doesn't the code that > > checks the sync? > > Because it depends where the involved CPUs are in the topology. The sync > code might just run on the same package an simply not see it. Yes, w/o > TSC_ADJUST the TSC sync code can just fail to see...

On Tue, 18 Sep 2018, Thomas Gleixner wrote: > On Tue, 18 Sep 2018, Peter Zijlstra wrote: > > > Your memory serves you right. That's indeed observable on CPUs which > > > lack TSC_ADJUST. > > > > But, if the gtod code can observe this, then why doesn't the code that > > checks the sync? > > Because it depends where the involved CPUs are in the topology. The sync > code might just run on the same package an simply not see it. Yes, w/o > TSC_ADJUST the TSC sync code can just fail to see...

...update this bizarro seqlock? What exactly does sequence==0 mean? > + > + *mode = VCLOCK_NONE; > + return 0; > +} > +#endif > > notrace static u64 vread_tsc(void) > { > @@ -173,6 +217,10 @@ notrace static inline u64 vgetsns(int *mode) > else if (gtod->vclock_mode == VCLOCK_PVCLOCK) > cycles = vread_pvclock(mode); > #endif > +#ifdef CONFIG_HYPERV_CLOCK > + else if (gtod->vclock_mode == VCLOCK_HVCLOCK) > + cycles = vread_hvclock(mode); > +#endif > else > re...

...update this bizarro seqlock? What exactly does sequence==0 mean? > + > + *mode = VCLOCK_NONE; > + return 0; > +} > +#endif > > notrace static u64 vread_tsc(void) > { > @@ -173,6 +217,10 @@ notrace static inline u64 vgetsns(int *mode) > else if (gtod->vclock_mode == VCLOCK_PVCLOCK) > cycles = vread_pvclock(mode); > #endif > +#ifdef CONFIG_HYPERV_CLOCK > + else if (gtod->vclock_mode == VCLOCK_HVCLOCK) > + cycles = vread_hvclock(mode); > +#endif > else > re...

Hi, Hyper-V TSC page clocksource is suitable for vDSO, however, the protocol defined by the hypervisor is different from VCLOCK_PVCLOCK. I implemented the required support re-using pvclock_page VVAR. Simple sysbench test shows the following results: Before: # time sysbench --test=memory --max-requests=500000 run ... real 1m22.618s user 0m50.193s sys 0m32.268s After: # time sysbench

Hi, Hyper-V TSC page clocksource is suitable for vDSO, however, the protocol defined by the hypervisor is different from VCLOCK_PVCLOCK. I implemented the required support re-using pvclock_page VVAR. Simple sysbench test shows the following results: Before: # time sysbench --test=memory --max-requests=500000 run ... real 1m22.618s user 0m50.193s sys 0m32.268s After: # time sysbench

...still have one of the machines which is affected by this. > > Are we sure this isn't a load vs rdtsc reorder? Because if I look at the > current code: The load order of last vs. rdtsc does not matter at all. CPU0 CPU1 .... now0 = rdtsc_ordered(); ... tk->cycle_last = now0; gtod->seq++; gtod->cycle_last = tk->cycle_last; ... gtod->seq++; seq_begin(gtod->seq); now1 = rdtsc_ordered(); So if the TSC on CPU1 is slightly behind the TSC on CPU0 then now1 can be smaller than cycle_last. The TSC sync stuff does not catch the small delta for unknown raisin...

...nt_tick = mul64x64_hi(cur_tsc, scale) + offset; + + if (READ_ONCE(tsc_pg->tsc_sequence) == sequence) + return current_tick; + } + + *mode = VCLOCK_NONE; + return 0; +} +#endif notrace static u64 vread_tsc(void) { @@ -173,6 +217,10 @@ notrace static inline u64 vgetsns(int *mode) else if (gtod->vclock_mode == VCLOCK_PVCLOCK) cycles = vread_pvclock(mode); #endif +#ifdef CONFIG_HYPERV_CLOCK + else if (gtod->vclock_mode == VCLOCK_HVCLOCK) + cycles = vread_hvclock(mode); +#endif else return 0; v = (cycles - gtod->cycle_last) & gtod->mask; diff --git a/arch/x86/ent...

...= mul_u64_u64_shr(cur_tsc, scale, 64) + offset; + + if (READ_ONCE(tsc_pg->tsc_sequence) == sequence) + return current_tick; + } + + *mode = VCLOCK_NONE; + return 0; +} +#endif notrace static u64 vread_tsc(void) { @@ -173,6 +205,10 @@ notrace static inline u64 vgetsns(int *mode) else if (gtod->vclock_mode == VCLOCK_PVCLOCK) cycles = vread_pvclock(mode); #endif +#ifdef CONFIG_HYPERV_TSCPAGE + else if (gtod->vclock_mode == VCLOCK_HVCLOCK) + cycles = vread_hvclock(mode); +#endif else return 0; v = (cycles - gtod->cycle_last) & gtod->mask; diff --git a/arch/x86/e...

...cpu could come in just behind cycles_last, resulting in a > > negative interval causing lots of havoc. > > > > So the sanity check is needed to avoid that case. > > Your memory serves you right. That's indeed observable on CPUs which > lack TSC_ADJUST. But, if the gtod code can observe this, then why doesn't the code that checks the sync?

...cpu could come in just behind cycles_last, resulting in a > > negative interval causing lots of havoc. > > > > So the sanity check is needed to avoid that case. > > Your memory serves you right. That's indeed observable on CPUs which > lack TSC_ADJUST. But, if the gtod code can observe this, then why doesn't the code that checks the sync?

...he basetime array lookup and set up the CLOCK_TAI base time in update_vsyscall(). The performance impact of the mask operation is within the noise. Signed-off-by: Thomas Gleixner <tglx at linutronix.de> --- arch/x86/entry/vdso/vclock_gettime.c | 2 +- arch/x86/entry/vsyscall/vsyscall_gtod.c | 4 ++++ arch/x86/include/asm/vgtod.h | 6 +++++- 3 files changed, 10 insertions(+), 2 deletions(-) --- a/arch/x86/entry/vdso/vclock_gettime.c +++ b/arch/x86/entry/vdso/vclock_gettime.c @@ -140,7 +140,7 @@ notrace static inline u64 vgetcyc(int mo notrace static int do_hres(c...

Hi, Hyper-V TSC page clocksource is suitable for vDSO, however, the protocol defined by the hypervisor is different from VCLOCK_PVCLOCK. Implemented the required support. Simple sysbench test shows the following results: Before: # time sysbench --test=memory --max-requests=500000 run ... real 1m22.618s user 0m50.193s sys 0m32.268s After: # time sysbench --test=memory