This series (with some toolstack updates) corrects a number of cases where a guest can see an incorrect wallclock time. 1. Systems with NTP won''t periodically synchronize the hardware RTC so the wallclock may be incorrect after a reboot. 2. The wallclock is always ~500 ms behind the correct time. 3. If the system time was stepped and NTP isn''t synchronized yet, the wallclock will still have the incorrect time. The fix for this requires the toolstack to synchronize the wallclock -- patch #3 provides the mechanism for this. David
David Vrabel
2012-Oct-12 12:57 UTC
[PATCH 1/3] xen: sync the CMOS RTC as well as the Xen wallclock
From: David Vrabel <david.vrabel@citrix.com> If NTP is used in dom0 and it is synchronized to its clock source, then the kernel will periodically synchronize the Xen wallclock with the system time. Updates to the Xen wallclock do not persist across reboots, so also synchronize the CMOS RTC (as on bare metal). Signed-off-by: David Vrabel <david.vrabel@citrix.com> --- arch/x86/xen/time.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 0296a95..5e7f536 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -14,6 +14,7 @@ #include <linux/kernel_stat.h> #include <linux/math64.h> #include <linux/gfp.h> +#include <linux/mc146818rtc.h> #include <asm/pvclock.h> #include <asm/xen/hypervisor.h> @@ -208,6 +209,10 @@ static int xen_set_wallclock(unsigned long now) if (!xen_initial_domain()) return -1; + /* Set the hardware RTC. */ + mach_set_rtc_mmss(now); + + /* Set the Xen wallclock. */ op.cmd = XENPF_settime; op.u.settime.secs = now; op.u.settime.nsecs = 0; -- 1.7.2.5
David Vrabel
2012-Oct-12 12:57 UTC
[PATCH 2/3] xen: add correct 500 ms offset when setting Xen wallclock
From: David Vrabel <david.vrabel@citrix.com> update_persistent_wallclock() (and hence xet_set_wallclock()) is called 500 ms after the second. xen_set_wallclock() was not considering this so the Xen wallclock would end up ~500 ms behind the correct time. Signed-off-by: David Vrabel <david.vrabel@citrix.com> --- arch/x86/xen/time.c | 9 +++++++-- 1 files changed, 7 insertions(+), 2 deletions(-) diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 5e7f536..11770d0 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -212,10 +212,15 @@ static int xen_set_wallclock(unsigned long now) /* Set the hardware RTC. */ mach_set_rtc_mmss(now); - /* Set the Xen wallclock. */ + /* + * Set the Xen wallclock. + * + * update_persistent_wallclock() is called ~500 ms after ''now'' + * so add an extra 500 ms. + */ op.cmd = XENPF_settime; op.u.settime.secs = now; - op.u.settime.nsecs = 0; + op.u.settime.nsecs = NSEC_PER_SEC / 2; op.u.settime.system_time = xen_clocksource_read(); rc = HYPERVISOR_dom0_op(&op); -- 1.7.2.5
David Vrabel
2012-Oct-12 12:57 UTC
[PATCH 3/3] xen/privcmd: add IOCTL_PRIVCMD_SYNC_WALLCLOCK to sync Xen''s wallclock
From: David Vrabel <david.vrabel@citrix.com> Add a new ioctl to synchronize Xen''s wallclock with the current system time. This may be used by the tools to ensure that newly created domains see the correct wallclock time if NTP is not used in dom0 or if domains are started before NTP has synchronized. Signed-off-by: David Vrabel <david.vrabel@citrix.com> --- arch/x86/xen/time.c | 25 ++++++++++++++++++------- drivers/xen/privcmd.c | 12 ++++++++++++ include/xen/privcmd.h | 8 ++++++++ include/xen/xen-ops.h | 2 ++ 4 files changed, 40 insertions(+), 7 deletions(-) diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 11770d0..d481ac9 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -8,6 +8,7 @@ * Jeremy Fitzhardinge <jeremy@xensource.com>, XenSource Inc, 2007 */ #include <linux/kernel.h> +#include <linux/export.h> #include <linux/interrupt.h> #include <linux/clocksource.h> #include <linux/clockchips.h> @@ -192,6 +193,19 @@ static void xen_read_wallclock(struct timespec *ts) put_cpu_var(xen_vcpu); } +int xen_write_wallclock(const struct timespec *ts) +{ + struct xen_platform_op op; + + op.cmd = XENPF_settime; + op.u.settime.secs = ts->tv_sec; + op.u.settime.nsecs = ts->tv_nsec; + op.u.settime.system_time = xen_clocksource_read(); + + return HYPERVISOR_dom0_op(&op); +} +EXPORT_SYMBOL_GPL(xen_write_wallclock); + static unsigned long xen_get_wallclock(void) { struct timespec ts; @@ -202,7 +216,7 @@ static unsigned long xen_get_wallclock(void) static int xen_set_wallclock(unsigned long now) { - struct xen_platform_op op; + struct timespec ts; int rc; /* do nothing for domU */ @@ -218,12 +232,9 @@ static int xen_set_wallclock(unsigned long now) * update_persistent_wallclock() is called ~500 ms after ''now'' * so add an extra 500 ms. */ - op.cmd = XENPF_settime; - op.u.settime.secs = now; - op.u.settime.nsecs = NSEC_PER_SEC / 2; - op.u.settime.system_time = xen_clocksource_read(); - - rc = HYPERVISOR_dom0_op(&op); + ts.tv_sec = now; + ts.tv_nsec = NSEC_PER_SEC / 2; + rc = xen_write_wallclock(&ts); WARN(rc != 0, "XENPF_settime failed: now=%ld\n", now); return rc; diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c index ccee0f1..ed2caf3 100644 --- a/drivers/xen/privcmd.c +++ b/drivers/xen/privcmd.c @@ -338,6 +338,14 @@ out: return ret; } +static long privcmd_ioctl_sync_wallclock(void) +{ + struct timespec ts; + + getnstimeofday(&ts); + return xen_write_wallclock(&ts); +} + static long privcmd_ioctl(struct file *file, unsigned int cmd, unsigned long data) { @@ -357,6 +365,10 @@ static long privcmd_ioctl(struct file *file, ret = privcmd_ioctl_mmap_batch(udata); break; + case IOCTL_PRIVCMD_SYNC_WALLCLOCK: + ret = privcmd_ioctl_sync_wallclock(); + break; + default: ret = -EINVAL; break; diff --git a/include/xen/privcmd.h b/include/xen/privcmd.h index 17857fb..d17d087 100644 --- a/include/xen/privcmd.h +++ b/include/xen/privcmd.h @@ -66,6 +66,12 @@ struct privcmd_mmapbatch { * @cmd: IOCTL_PRIVCMD_HYPERCALL * @arg: &privcmd_hypercall_t * Return: Value returned from execution of the specified hypercall. + * + * @cmd: IOCTL_PRIVCMD_SYNC_WALLCLOCK + * @arg: Unused. + * Synchronizes the Xen wallclock with the current system time. + * Return: 0 on success, or -1 on error with errno set to EPERM or + * EACCES. */ #define IOCTL_PRIVCMD_HYPERCALL \ _IOC(_IOC_NONE, ''P'', 0, sizeof(struct privcmd_hypercall)) @@ -73,5 +79,7 @@ struct privcmd_mmapbatch { _IOC(_IOC_NONE, ''P'', 2, sizeof(struct privcmd_mmap)) #define IOCTL_PRIVCMD_MMAPBATCH \ _IOC(_IOC_NONE, ''P'', 3, sizeof(struct privcmd_mmapbatch)) +#define IOCTL_PRIVCMD_SYNC_WALLCLOCK \ + _IOC(_IOC_NONE, ''P'', 5, 0) #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */ diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index 6a198e4..eefed22 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -29,4 +29,6 @@ int xen_remap_domain_mfn_range(struct vm_area_struct *vma, unsigned long mfn, int nr, pgprot_t prot, unsigned domid); +int xen_write_wallclock(const struct timespec *ts); + #endif /* INCLUDE_XEN_OPS_H */ -- 1.7.2.5
Konrad Rzeszutek Wilk
2012-Oct-12 13:41 UTC
Re: [PATCH 3/3] xen/privcmd: add IOCTL_PRIVCMD_SYNC_WALLCLOCK to sync Xen''s wallclock
On Fri, Oct 12, 2012 at 01:57:14PM +0100, David Vrabel wrote:> From: David Vrabel <david.vrabel@citrix.com> > > Add a new ioctl to synchronize Xen''s wallclock with the current system > time. > > This may be used by the tools to ensure that newly created domains see > the correct wallclock time if NTP is not used in dom0 or if domains > are started before NTP has synchronized.So... how does this work with NTPD? As in does ntpd _not_ update the hwclock enough?> > Signed-off-by: David Vrabel <david.vrabel@citrix.com> > --- > arch/x86/xen/time.c | 25 ++++++++++++++++++------- > drivers/xen/privcmd.c | 12 ++++++++++++ > include/xen/privcmd.h | 8 ++++++++ > include/xen/xen-ops.h | 2 ++ > 4 files changed, 40 insertions(+), 7 deletions(-) > > diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c > index 11770d0..d481ac9 100644 > --- a/arch/x86/xen/time.c > +++ b/arch/x86/xen/time.c > @@ -8,6 +8,7 @@ > * Jeremy Fitzhardinge <jeremy@xensource.com>, XenSource Inc, 2007 > */ > #include <linux/kernel.h> > +#include <linux/export.h> > #include <linux/interrupt.h> > #include <linux/clocksource.h> > #include <linux/clockchips.h> > @@ -192,6 +193,19 @@ static void xen_read_wallclock(struct timespec *ts) > put_cpu_var(xen_vcpu); > } > > +int xen_write_wallclock(const struct timespec *ts) > +{ > + struct xen_platform_op op; > + > + op.cmd = XENPF_settime; > + op.u.settime.secs = ts->tv_sec; > + op.u.settime.nsecs = ts->tv_nsec; > + op.u.settime.system_time = xen_clocksource_read(); > + > + return HYPERVISOR_dom0_op(&op); > +} > +EXPORT_SYMBOL_GPL(xen_write_wallclock); > + > static unsigned long xen_get_wallclock(void) > { > struct timespec ts; > @@ -202,7 +216,7 @@ static unsigned long xen_get_wallclock(void) > > static int xen_set_wallclock(unsigned long now) > { > - struct xen_platform_op op; > + struct timespec ts; > int rc; > > /* do nothing for domU */ > @@ -218,12 +232,9 @@ static int xen_set_wallclock(unsigned long now) > * update_persistent_wallclock() is called ~500 ms after ''now'' > * so add an extra 500 ms. > */ > - op.cmd = XENPF_settime; > - op.u.settime.secs = now; > - op.u.settime.nsecs = NSEC_PER_SEC / 2; > - op.u.settime.system_time = xen_clocksource_read(); > - > - rc = HYPERVISOR_dom0_op(&op); > + ts.tv_sec = now; > + ts.tv_nsec = NSEC_PER_SEC / 2; > + rc = xen_write_wallclock(&ts); > WARN(rc != 0, "XENPF_settime failed: now=%ld\n", now); > > return rc; > diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c > index ccee0f1..ed2caf3 100644 > --- a/drivers/xen/privcmd.c > +++ b/drivers/xen/privcmd.c > @@ -338,6 +338,14 @@ out: > return ret; > } > > +static long privcmd_ioctl_sync_wallclock(void) > +{ > + struct timespec ts; > + > + getnstimeofday(&ts); > + return xen_write_wallclock(&ts); > +} > + > static long privcmd_ioctl(struct file *file, > unsigned int cmd, unsigned long data) > { > @@ -357,6 +365,10 @@ static long privcmd_ioctl(struct file *file, > ret = privcmd_ioctl_mmap_batch(udata); > break; > > + case IOCTL_PRIVCMD_SYNC_WALLCLOCK: > + ret = privcmd_ioctl_sync_wallclock(); > + break; > + > default: > ret = -EINVAL; > break; > diff --git a/include/xen/privcmd.h b/include/xen/privcmd.h > index 17857fb..d17d087 100644 > --- a/include/xen/privcmd.h > +++ b/include/xen/privcmd.h > @@ -66,6 +66,12 @@ struct privcmd_mmapbatch { > * @cmd: IOCTL_PRIVCMD_HYPERCALL > * @arg: &privcmd_hypercall_t > * Return: Value returned from execution of the specified hypercall. > + * > + * @cmd: IOCTL_PRIVCMD_SYNC_WALLCLOCK > + * @arg: Unused. > + * Synchronizes the Xen wallclock with the current system time. > + * Return: 0 on success, or -1 on error with errno set to EPERM or > + * EACCES. > */ > #define IOCTL_PRIVCMD_HYPERCALL \ > _IOC(_IOC_NONE, ''P'', 0, sizeof(struct privcmd_hypercall)) > @@ -73,5 +79,7 @@ struct privcmd_mmapbatch { > _IOC(_IOC_NONE, ''P'', 2, sizeof(struct privcmd_mmap)) > #define IOCTL_PRIVCMD_MMAPBATCH \ > _IOC(_IOC_NONE, ''P'', 3, sizeof(struct privcmd_mmapbatch)) > +#define IOCTL_PRIVCMD_SYNC_WALLCLOCK \ > + _IOC(_IOC_NONE, ''P'', 5, 0) > > #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */ > diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h > index 6a198e4..eefed22 100644 > --- a/include/xen/xen-ops.h > +++ b/include/xen/xen-ops.h > @@ -29,4 +29,6 @@ int xen_remap_domain_mfn_range(struct vm_area_struct *vma, > unsigned long mfn, int nr, > pgprot_t prot, unsigned domid); > > +int xen_write_wallclock(const struct timespec *ts); > + > #endif /* INCLUDE_XEN_OPS_H */ > -- > 1.7.2.5 > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >
David Vrabel
2012-Oct-12 14:02 UTC
Re: [PATCH 3/3] xen/privcmd: add IOCTL_PRIVCMD_SYNC_WALLCLOCK to sync Xen''s wallclock
On 12/10/12 14:41, Konrad Rzeszutek Wilk wrote:> On Fri, Oct 12, 2012 at 01:57:14PM +0100, David Vrabel wrote: >> From: David Vrabel <david.vrabel@citrix.com> >> >> Add a new ioctl to synchronize Xen''s wallclock with the current system >> time. >> >> This may be used by the tools to ensure that newly created domains see >> the correct wallclock time if NTP is not used in dom0 or if domains >> are started before NTP has synchronized. > > So... how does this work with NTPD? As in does ntpd _not_ update the > hwclock enough?Once NTPD is synchronized then the kernel updates the wallclock (and the RTC with patch #1) every 11 mins. I assume this is often enough given how NTP adjusts the system time. You only really need the tools to sync wallclock if system time was stepped at start of day. e.g., init scripts could do something like: ntpdate pool.ntp.org hwclock --systohc xen-wallclock --systowc David>> Signed-off-by: David Vrabel <david.vrabel@citrix.com> >> --- >> arch/x86/xen/time.c | 25 ++++++++++++++++++------- >> drivers/xen/privcmd.c | 12 ++++++++++++ >> include/xen/privcmd.h | 8 ++++++++ >> include/xen/xen-ops.h | 2 ++ >> 4 files changed, 40 insertions(+), 7 deletions(-) >> >> diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c >> index 11770d0..d481ac9 100644 >> --- a/arch/x86/xen/time.c >> +++ b/arch/x86/xen/time.c >> @@ -8,6 +8,7 @@ >> * Jeremy Fitzhardinge <jeremy@xensource.com>, XenSource Inc, 2007 >> */ >> #include <linux/kernel.h> >> +#include <linux/export.h> >> #include <linux/interrupt.h> >> #include <linux/clocksource.h> >> #include <linux/clockchips.h> >> @@ -192,6 +193,19 @@ static void xen_read_wallclock(struct timespec *ts) >> put_cpu_var(xen_vcpu); >> } >> >> +int xen_write_wallclock(const struct timespec *ts) >> +{ >> + struct xen_platform_op op; >> + >> + op.cmd = XENPF_settime; >> + op.u.settime.secs = ts->tv_sec; >> + op.u.settime.nsecs = ts->tv_nsec; >> + op.u.settime.system_time = xen_clocksource_read(); >> + >> + return HYPERVISOR_dom0_op(&op); >> +} >> +EXPORT_SYMBOL_GPL(xen_write_wallclock); >> + >> static unsigned long xen_get_wallclock(void) >> { >> struct timespec ts; >> @@ -202,7 +216,7 @@ static unsigned long xen_get_wallclock(void) >> >> static int xen_set_wallclock(unsigned long now) >> { >> - struct xen_platform_op op; >> + struct timespec ts; >> int rc; >> >> /* do nothing for domU */ >> @@ -218,12 +232,9 @@ static int xen_set_wallclock(unsigned long now) >> * update_persistent_wallclock() is called ~500 ms after ''now'' >> * so add an extra 500 ms. >> */ >> - op.cmd = XENPF_settime; >> - op.u.settime.secs = now; >> - op.u.settime.nsecs = NSEC_PER_SEC / 2; >> - op.u.settime.system_time = xen_clocksource_read(); >> - >> - rc = HYPERVISOR_dom0_op(&op); >> + ts.tv_sec = now; >> + ts.tv_nsec = NSEC_PER_SEC / 2; >> + rc = xen_write_wallclock(&ts); >> WARN(rc != 0, "XENPF_settime failed: now=%ld\n", now); >> >> return rc; >> diff --git a/drivers/xen/privcmd.c b/drivers/xen/privcmd.c >> index ccee0f1..ed2caf3 100644 >> --- a/drivers/xen/privcmd.c >> +++ b/drivers/xen/privcmd.c >> @@ -338,6 +338,14 @@ out: >> return ret; >> } >> >> +static long privcmd_ioctl_sync_wallclock(void) >> +{ >> + struct timespec ts; >> + >> + getnstimeofday(&ts); >> + return xen_write_wallclock(&ts); >> +} >> + >> static long privcmd_ioctl(struct file *file, >> unsigned int cmd, unsigned long data) >> { >> @@ -357,6 +365,10 @@ static long privcmd_ioctl(struct file *file, >> ret = privcmd_ioctl_mmap_batch(udata); >> break; >> >> + case IOCTL_PRIVCMD_SYNC_WALLCLOCK: >> + ret = privcmd_ioctl_sync_wallclock(); >> + break; >> + >> default: >> ret = -EINVAL; >> break; >> diff --git a/include/xen/privcmd.h b/include/xen/privcmd.h >> index 17857fb..d17d087 100644 >> --- a/include/xen/privcmd.h >> +++ b/include/xen/privcmd.h >> @@ -66,6 +66,12 @@ struct privcmd_mmapbatch { >> * @cmd: IOCTL_PRIVCMD_HYPERCALL >> * @arg: &privcmd_hypercall_t >> * Return: Value returned from execution of the specified hypercall. >> + * >> + * @cmd: IOCTL_PRIVCMD_SYNC_WALLCLOCK >> + * @arg: Unused. >> + * Synchronizes the Xen wallclock with the current system time. >> + * Return: 0 on success, or -1 on error with errno set to EPERM or >> + * EACCES. >> */ >> #define IOCTL_PRIVCMD_HYPERCALL \ >> _IOC(_IOC_NONE, ''P'', 0, sizeof(struct privcmd_hypercall)) >> @@ -73,5 +79,7 @@ struct privcmd_mmapbatch { >> _IOC(_IOC_NONE, ''P'', 2, sizeof(struct privcmd_mmap)) >> #define IOCTL_PRIVCMD_MMAPBATCH \ >> _IOC(_IOC_NONE, ''P'', 3, sizeof(struct privcmd_mmapbatch)) >> +#define IOCTL_PRIVCMD_SYNC_WALLCLOCK \ >> + _IOC(_IOC_NONE, ''P'', 5, 0) >> >> #endif /* __LINUX_PUBLIC_PRIVCMD_H__ */ >> diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h >> index 6a198e4..eefed22 100644 >> --- a/include/xen/xen-ops.h >> +++ b/include/xen/xen-ops.h >> @@ -29,4 +29,6 @@ int xen_remap_domain_mfn_range(struct vm_area_struct *vma, >> unsigned long mfn, int nr, >> pgprot_t prot, unsigned domid); >> >> +int xen_write_wallclock(const struct timespec *ts); >> + >> #endif /* INCLUDE_XEN_OPS_H */ >> -- >> 1.7.2.5 >> >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel >>
Konrad Rzeszutek Wilk
2012-Oct-12 14:29 UTC
Re: [PATCH 3/3] xen/privcmd: add IOCTL_PRIVCMD_SYNC_WALLCLOCK to sync Xen''s wallclock
On Fri, Oct 12, 2012 at 10:02 AM, David Vrabel <david.vrabel@citrix.com> wrote:> On 12/10/12 14:41, Konrad Rzeszutek Wilk wrote: >> On Fri, Oct 12, 2012 at 01:57:14PM +0100, David Vrabel wrote: >>> From: David Vrabel <david.vrabel@citrix.com> >>> >>> Add a new ioctl to synchronize Xen''s wallclock with the current system >>> time. >>> >>> This may be used by the tools to ensure that newly created domains see >>> the correct wallclock time if NTP is not used in dom0 or if domains >>> are started before NTP has synchronized. >> >> So... how does this work with NTPD? As in does ntpd _not_ update the >> hwclock enough? > > Once NTPD is synchronized then the kernel updates the wallclock (and the > RTC with patch #1) every 11 mins. I assume this is often enough given > how NTP adjusts the system time. > > You only really need the tools to sync wallclock if system time was > stepped at start of day. e.g., init scripts could do something like: > > ntpdate pool.ntp.org > hwclock --systohc > xen-wallclock --systowcI think I am missing something. The hwclock should end up in the xen_set_wallclock call. And from there on, the ntpd would update the wallclock if it got skewed enough? Or is the system time not calling the wall-clock enough? If that is the case, would just adding this in the crontab be enough: */11 * * * * hwclock --systohc ?
Konrad Rzeszutek Wilk
2012-Oct-12 14:57 UTC
Re: [PATCH 1/3] xen: sync the CMOS RTC as well as the Xen wallclock
On Fri, Oct 12, 2012 at 01:57:12PM +0100, David Vrabel wrote:> From: David Vrabel <david.vrabel@citrix.com> > > If NTP is used in dom0 and it is synchronized to its clock source, > then the kernel will periodically synchronize the Xen wallclock with > the system time. Updates to the Xen wallclock do not persist across > reboots, so also synchronize the CMOS RTC (as on bare metal). > > Signed-off-by: David Vrabel <david.vrabel@citrix.com> > --- > arch/x86/xen/time.c | 5 +++++ > 1 files changed, 5 insertions(+), 0 deletions(-) > > diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c > index 0296a95..5e7f536 100644 > --- a/arch/x86/xen/time.c > +++ b/arch/x86/xen/time.c > @@ -14,6 +14,7 @@ > #include <linux/kernel_stat.h> > #include <linux/math64.h> > #include <linux/gfp.h> > +#include <linux/mc146818rtc.h> > > #include <asm/pvclock.h> > #include <asm/xen/hypervisor.h> > @@ -208,6 +209,10 @@ static int xen_set_wallclock(unsigned long now) > if (!xen_initial_domain()) > return -1; > > + /* Set the hardware RTC. */ > + mach_set_rtc_mmss(now);So how does this work? Is the hypervisor traping on the RTC CMOS clock?> + > + /* Set the Xen wallclock. */ > op.cmd = XENPF_settime; > op.u.settime.secs = now; > op.u.settime.nsecs = 0; > -- > 1.7.2.5
David Vrabel
2012-Oct-12 14:59 UTC
Re: [PATCH 3/3] xen/privcmd: add IOCTL_PRIVCMD_SYNC_WALLCLOCK to sync Xen''s wallclock
On 12/10/12 15:29, Konrad Rzeszutek Wilk wrote:> On Fri, Oct 12, 2012 at 10:02 AM, David Vrabel <david.vrabel@citrix.com> wrote: >> On 12/10/12 14:41, Konrad Rzeszutek Wilk wrote: >>> On Fri, Oct 12, 2012 at 01:57:14PM +0100, David Vrabel wrote: >>>> From: David Vrabel <david.vrabel@citrix.com> >>>> >>>> Add a new ioctl to synchronize Xen''s wallclock with the current system >>>> time. >>>> >>>> This may be used by the tools to ensure that newly created domains see >>>> the correct wallclock time if NTP is not used in dom0 or if domains >>>> are started before NTP has synchronized. >>> >>> So... how does this work with NTPD? As in does ntpd _not_ update the >>> hwclock enough? >> >> Once NTPD is synchronized then the kernel updates the wallclock (and the >> RTC with patch #1) every 11 mins. I assume this is often enough given >> how NTP adjusts the system time. >> >> You only really need the tools to sync wallclock if system time was >> stepped at start of day. e.g., init scripts could do something like: >> >> ntpdate pool.ntp.org >> hwclock --systohc >> xen-wallclock --systowc > > I think I am missing something. The hwclock should end up in the > xen_set_wallclock call. And from there on, the ntpd would update the > wallclock if it got skewed enough? Or is the system time not calling > the wall-clock enough? If that is the case, would just adding this in > the crontab be enough:hwclock talks to /dev/rtc which writes to the CMOS directly and does not call update_persistent_clock() (or xen_set_wallclock()). David
Konrad Rzeszutek Wilk
2012-Oct-12 15:02 UTC
Re: [PATCH 3/3] xen/privcmd: add IOCTL_PRIVCMD_SYNC_WALLCLOCK to sync Xen''s wallclock
On Fri, Oct 12, 2012 at 03:59:25PM +0100, David Vrabel wrote:> On 12/10/12 15:29, Konrad Rzeszutek Wilk wrote: > > On Fri, Oct 12, 2012 at 10:02 AM, David Vrabel <david.vrabel@citrix.com> wrote: > >> On 12/10/12 14:41, Konrad Rzeszutek Wilk wrote: > >>> On Fri, Oct 12, 2012 at 01:57:14PM +0100, David Vrabel wrote: > >>>> From: David Vrabel <david.vrabel@citrix.com> > >>>> > >>>> Add a new ioctl to synchronize Xen''s wallclock with the current system > >>>> time. > >>>> > >>>> This may be used by the tools to ensure that newly created domains see > >>>> the correct wallclock time if NTP is not used in dom0 or if domains > >>>> are started before NTP has synchronized. > >>> > >>> So... how does this work with NTPD? As in does ntpd _not_ update the > >>> hwclock enough? > >> > >> Once NTPD is synchronized then the kernel updates the wallclock (and the > >> RTC with patch #1) every 11 mins. I assume this is often enough given > >> how NTP adjusts the system time. > >> > >> You only really need the tools to sync wallclock if system time was > >> stepped at start of day. e.g., init scripts could do something like: > >> > >> ntpdate pool.ntp.org > >> hwclock --systohc > >> xen-wallclock --systowc > > > > I think I am missing something. The hwclock should end up in the > > xen_set_wallclock call. And from there on, the ntpd would update the > > wallclock if it got skewed enough? Or is the system time not calling > > the wall-clock enough? If that is the case, would just adding this in > > the crontab be enough: > > hwclock talks to /dev/rtc which writes to the CMOS directly and does not > call update_persistent_clock() (or xen_set_wallclock())./me scratches his head. I recall that the update_persistent_clock() was being called.. with hwclock -w (which is the same as --systohc). That seems like a bug in the generic code - I would think that hwclock would update the wallclock, not just the RTC. Oh, it is whoever calls ''adjtimex'' syscall ends up calling in update_persistent_clock(). So .. ntpdate or ntpd don''t call that? Somebody does.. I hope?
David Vrabel
2012-Oct-12 16:13 UTC
Re: [PATCH 1/3] xen: sync the CMOS RTC as well as the Xen wallclock
On 12/10/12 15:57, Konrad Rzeszutek Wilk wrote:> On Fri, Oct 12, 2012 at 01:57:12PM +0100, David Vrabel wrote: >> From: David Vrabel <david.vrabel@citrix.com> >> >> If NTP is used in dom0 and it is synchronized to its clock source, >> then the kernel will periodically synchronize the Xen wallclock with >> the system time. Updates to the Xen wallclock do not persist across >> reboots, so also synchronize the CMOS RTC (as on bare metal). >> >> Signed-off-by: David Vrabel <david.vrabel@citrix.com> >> --- >> arch/x86/xen/time.c | 5 +++++ >> 1 files changed, 5 insertions(+), 0 deletions(-) >> >> diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c >> index 0296a95..5e7f536 100644 >> --- a/arch/x86/xen/time.c >> +++ b/arch/x86/xen/time.c >> @@ -14,6 +14,7 @@ >> #include <linux/kernel_stat.h> >> #include <linux/math64.h> >> #include <linux/gfp.h> >> +#include <linux/mc146818rtc.h> >> >> #include <asm/pvclock.h> >> #include <asm/xen/hypervisor.h> >> @@ -208,6 +209,10 @@ static int xen_set_wallclock(unsigned long now) >> if (!xen_initial_domain()) >> return -1; >> >> + /* Set the hardware RTC. */ >> + mach_set_rtc_mmss(now); > > So how does this work? Is the hypervisor traping on the RTC CMOS clock?It''s works the same as with the RTC driver. I think the hypervisor allows dom0 access to the relevant I/O ports. It''s worth pointing out that mach_set_rtc_mmss() only sets the minutes and seconds so it doesn''t handle big step changes which is why update_persistent_clock() is only called once NTP is synchronized. David
David Vrabel
2012-Oct-12 16:14 UTC
Re: [PATCH 3/3] xen/privcmd: add IOCTL_PRIVCMD_SYNC_WALLCLOCK to sync Xen''s wallclock
On 12/10/12 16:02, Konrad Rzeszutek Wilk wrote:> On Fri, Oct 12, 2012 at 03:59:25PM +0100, David Vrabel wrote: >> On 12/10/12 15:29, Konrad Rzeszutek Wilk wrote: >>> On Fri, Oct 12, 2012 at 10:02 AM, David Vrabel <david.vrabel@citrix.com> wrote: >>>> On 12/10/12 14:41, Konrad Rzeszutek Wilk wrote: >>>>> On Fri, Oct 12, 2012 at 01:57:14PM +0100, David Vrabel wrote: >>>>>> From: David Vrabel <david.vrabel@citrix.com> >>>>>> >>>>>> Add a new ioctl to synchronize Xen''s wallclock with the current system >>>>>> time. >>>>>> >>>>>> This may be used by the tools to ensure that newly created domains see >>>>>> the correct wallclock time if NTP is not used in dom0 or if domains >>>>>> are started before NTP has synchronized. >>>>> >>>>> So... how does this work with NTPD? As in does ntpd _not_ update the >>>>> hwclock enough? >>>> >>>> Once NTPD is synchronized then the kernel updates the wallclock (and the >>>> RTC with patch #1) every 11 mins. I assume this is often enough given >>>> how NTP adjusts the system time. >>>> >>>> You only really need the tools to sync wallclock if system time was >>>> stepped at start of day. e.g., init scripts could do something like: >>>> >>>> ntpdate pool.ntp.org >>>> hwclock --systohc >>>> xen-wallclock --systowc >>> >>> I think I am missing something. The hwclock should end up in the >>> xen_set_wallclock call. And from there on, the ntpd would update the >>> wallclock if it got skewed enough? Or is the system time not calling >>> the wall-clock enough? If that is the case, would just adding this in >>> the crontab be enough: >> >> hwclock talks to /dev/rtc which writes to the CMOS directly and does not >> call update_persistent_clock() (or xen_set_wallclock()). > > /me scratches his head. > > I recall that the update_persistent_clock() was being called.. with > hwclock -w (which is the same as --systohc).No,> That seems like a bug in the generic code - I would think that > hwclock would update the wallclock, not just the RTC.If we wanted hwclock to also update the xen wallclock we would need a xen-specific rtc driver that updated both rtc and wallclock.> Oh, it is whoever calls ''adjtimex'' syscall ends up calling in > update_persistent_clock(). So .. ntpdate or ntpd don''t call that?ntpd does call adjtimex so if you are running ntpd and it is synchronized to its clock source you do get the periodic sync of the wallclock. David
Konrad Rzeszutek Wilk
2012-Oct-12 16:16 UTC
Re: [PATCH 3/3] xen/privcmd: add IOCTL_PRIVCMD_SYNC_WALLCLOCK to sync Xen''s wallclock
On Fri, Oct 12, 2012 at 05:14:16PM +0100, David Vrabel wrote:> On 12/10/12 16:02, Konrad Rzeszutek Wilk wrote: > > On Fri, Oct 12, 2012 at 03:59:25PM +0100, David Vrabel wrote: > >> On 12/10/12 15:29, Konrad Rzeszutek Wilk wrote: > >>> On Fri, Oct 12, 2012 at 10:02 AM, David Vrabel <david.vrabel@citrix.com> wrote: > >>>> On 12/10/12 14:41, Konrad Rzeszutek Wilk wrote: > >>>>> On Fri, Oct 12, 2012 at 01:57:14PM +0100, David Vrabel wrote: > >>>>>> From: David Vrabel <david.vrabel@citrix.com> > >>>>>> > >>>>>> Add a new ioctl to synchronize Xen''s wallclock with the current system > >>>>>> time. > >>>>>> > >>>>>> This may be used by the tools to ensure that newly created domains see > >>>>>> the correct wallclock time if NTP is not used in dom0 or if domains > >>>>>> are started before NTP has synchronized. > >>>>> > >>>>> So... how does this work with NTPD? As in does ntpd _not_ update the > >>>>> hwclock enough? > >>>> > >>>> Once NTPD is synchronized then the kernel updates the wallclock (and the > >>>> RTC with patch #1) every 11 mins. I assume this is often enough given > >>>> how NTP adjusts the system time. > >>>> > >>>> You only really need the tools to sync wallclock if system time was > >>>> stepped at start of day. e.g., init scripts could do something like: > >>>> > >>>> ntpdate pool.ntp.org > >>>> hwclock --systohc > >>>> xen-wallclock --systowc > >>> > >>> I think I am missing something. The hwclock should end up in the > >>> xen_set_wallclock call. And from there on, the ntpd would update the > >>> wallclock if it got skewed enough? Or is the system time not calling > >>> the wall-clock enough? If that is the case, would just adding this in > >>> the crontab be enough: > >> > >> hwclock talks to /dev/rtc which writes to the CMOS directly and does not > >> call update_persistent_clock() (or xen_set_wallclock()). > > > > /me scratches his head. > > > > I recall that the update_persistent_clock() was being called.. with > > hwclock -w (which is the same as --systohc). > > No, > > > That seems like a bug in the generic code - I would think that > > hwclock would update the wallclock, not just the RTC. > > If we wanted hwclock to also update the xen wallclock we would need a > xen-specific rtc driver that updated both rtc and wallclock. > > > Oh, it is whoever calls ''adjtimex'' syscall ends up calling in > > update_persistent_clock(). So .. ntpdate or ntpd don''t call that? > > ntpd does call adjtimex so if you are running ntpd and it is > synchronized to its clock source you do get the periodic sync of the > wallclock.So, having both ntpd (which calls call adjtimex) - which would run in the background; and hwclock run at startup - (which updates the RTC, which ends - I hope - being trapped by the hypervisor), we end up with the correct time, right?
On 12/10/12 13:57, David Vrabel wrote:> This series (with some toolstack updates) corrects a number of cases > where a guest can see an incorrect wallclock time. > > 1. Systems with NTP won''t periodically synchronize the hardware RTC so > the wallclock may be incorrect after a reboot. > > 2. The wallclock is always ~500 ms behind the correct time. > > 3. If the system time was stepped and NTP isn''t synchronized yet, the > wallclock will still have the incorrect time. The fix for this > requires the toolstack to synchronize the wallclock -- patch #3 > provides the mechanism for this.These tables should help. Before: | Updates Process | System Time? Xen Wallclock? Hardware Clock? ------------------------------------------------------------- date -s | X hwclock -w | X ntpd | X X[*] ntpdate | X After this (and the toolstack) patches: | Updates Process | System Time? Xen Wallclock? Hardware Clock? ------------------------------------------------------------- date -s | X hwclock -w | X ntpd | X X[*] X[*] ntpdate | X xen-wallclock | X [*] every 11 minutes. David
David Vrabel
2012-Oct-12 16:30 UTC
Re: [PATCH 3/3] xen/privcmd: add IOCTL_PRIVCMD_SYNC_WALLCLOCK to sync Xen''s wallclock
On 12/10/12 17:16, Konrad Rzeszutek Wilk wrote:> > So, having both ntpd (which calls call adjtimex) - which would run in the > background; and hwclock run at startup - (which updates the RTC, which ends > - I hope - being trapped by the hypervisor), we end up with the correct > time, right?Yes, unless you start a guest before ntpd has synchronized. David
Ian Campbell
2012-Oct-15 09:25 UTC
Re: [PATCH 2/3] xen: add correct 500 ms offset when setting Xen wallclock
On Fri, 2012-10-12 at 13:57 +0100, David Vrabel wrote:> From: David Vrabel <david.vrabel@citrix.com> > > update_persistent_wallclock() (and hence xet_set_wallclock()) is > called 500 ms after the second.The comment in sync_cmos_clock says it is called 500ms before the next second. I only mention it to double check that the right semantics for set_wallclock are being implemented, i.e. that we are compensating in the right direction.> xen_set_wallclock() was not > considering this so the Xen wallclock would end up ~500 ms behind the > correct time. > > Signed-off-by: David Vrabel <david.vrabel@citrix.com> > --- > arch/x86/xen/time.c | 9 +++++++-- > 1 files changed, 7 insertions(+), 2 deletions(-) > > diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c > index 5e7f536..11770d0 100644 > --- a/arch/x86/xen/time.c > +++ b/arch/x86/xen/time.c > @@ -212,10 +212,15 @@ static int xen_set_wallclock(unsigned long now) > /* Set the hardware RTC. */ > mach_set_rtc_mmss(now); > > - /* Set the Xen wallclock. */ > + /* > + * Set the Xen wallclock. > + * > + * update_persistent_wallclock() is called ~500 ms after ''now'' > + * so add an extra 500 ms. > + */ > op.cmd = XENPF_settime; > op.u.settime.secs = now; > - op.u.settime.nsecs = 0; > + op.u.settime.nsecs = NSEC_PER_SEC / 2; > op.u.settime.system_time = xen_clocksource_read(); > > rc = HYPERVISOR_dom0_op(&op);
On Fri, 2012-10-12 at 17:25 +0100, David Vrabel wrote:> On 12/10/12 13:57, David Vrabel wrote: > > This series (with some toolstack updates) corrects a number of cases > > where a guest can see an incorrect wallclock time. > > > > 1. Systems with NTP won''t periodically synchronize the hardware RTC so > > the wallclock may be incorrect after a reboot. > > > > 2. The wallclock is always ~500 ms behind the correct time. > > > > 3. If the system time was stepped and NTP isn''t synchronized yet, the > > wallclock will still have the incorrect time. The fix for this > > requires the toolstack to synchronize the wallclock -- patch #3 > > provides the mechanism for this. > > These tables should help. > > Before: > > | Updates > Process | System Time? Xen Wallclock? Hardware Clock? > ------------------------------------------------------------- > date -s | X > hwclock -w | X > ntpd | X X[*] > ntpdate | XWhat does the equivalent table for native look like? Is the "updated hardware clock" column in that caseis union of xen wallclock and hardware clock columns here? Ian.
David Vrabel
2012-Oct-15 12:21 UTC
Re: [PATCH 2/3] xen: add correct 500 ms offset when setting Xen wallclock
On 15/10/12 10:25, Ian Campbell wrote:> On Fri, 2012-10-12 at 13:57 +0100, David Vrabel wrote: >> From: David Vrabel <david.vrabel@citrix.com> >> >> update_persistent_wallclock() (and hence xet_set_wallclock()) is >> called 500 ms after the second. > > The comment in sync_cmos_clock says it is called 500ms before the next > second.This is the same thing, right?> I only mention it to double check that the right semantics for > set_wallclock are being implemented, i.e. that we are compensating in > the right direction.I''ll add some debug code to make sure. David
On 15/10/12 10:26, Ian Campbell wrote:> On Fri, 2012-10-12 at 17:25 +0100, David Vrabel wrote: >> On 12/10/12 13:57, David Vrabel wrote: >>> This series (with some toolstack updates) corrects a number of cases >>> where a guest can see an incorrect wallclock time. >>> >>> 1. Systems with NTP won''t periodically synchronize the hardware RTC so >>> the wallclock may be incorrect after a reboot. >>> >>> 2. The wallclock is always ~500 ms behind the correct time. >>> >>> 3. If the system time was stepped and NTP isn''t synchronized yet, the >>> wallclock will still have the incorrect time. The fix for this >>> requires the toolstack to synchronize the wallclock -- patch #3 >>> provides the mechanism for this. >> >> These tables should help. >> >> Before: >> >> | Updates >> Process | System Time? Xen Wallclock? Hardware Clock? >> ------------------------------------------------------------- >> date -s | X >> hwclock -w | X >> ntpd | X X[*] >> ntpdate | X > > What does the equivalent table for native look like? > > Is the "updated hardware clock" column in that caseis union of xen > wallclock and hardware clock columns here?Yes. | Updates Process | System Time? Hardware Clock? --------------------------------------------- date -s | X hwclock -w | X ntpd | X X[*] ntpdate | X David
Ian Campbell
2012-Oct-15 12:27 UTC
Re: [PATCH 2/3] xen: add correct 500 ms offset when setting Xen wallclock
On Mon, 2012-10-15 at 13:21 +0100, David Vrabel wrote:> On 15/10/12 10:25, Ian Campbell wrote: > > On Fri, 2012-10-12 at 13:57 +0100, David Vrabel wrote: > >> From: David Vrabel <david.vrabel@citrix.com> > >> > >> update_persistent_wallclock() (and hence xet_set_wallclock()) is > >> called 500 ms after the second. > > > > The comment in sync_cmos_clock says it is called 500ms before the next > > second. > > This is the same thing, right?It matters in the native case, where, If I''m reading it right, things are setup such thatthe time change happens on the next second tick over. I just wanted to check it did/didn''t matter here as well.> > > I only mention it to double check that the right semantics for > > set_wallclock are being implemented, i.e. that we are compensating in > > the right direction. > > I''ll add some debug code to make sure. > > David