Hi,
using SLES10 I''m unable to synchronize the time of DomU with that of
Dom0. There
is a persistent offset of about 3 seconds!
Here''s a small history (not actual output):
remote refid st t when poll reach delay offset jitter
rkdvmso1.dvm.kl 192.168.0.11 5 u - 64 1 0.136 -2977.1 0.099
*rkdvmso1.dvm.kl 192.168.0.11 5 u 2 64 3 0.092 -2980.9 1.245
*rkdvmso1.dvm.kl 192.168.0.11 5 u 1 64 7 0.073 -2986.6 0.274
*rkdvmso1.dvm.kl 192.168.0.11 5 u 8 64 37 0.107 -2992.4 0.271
*rkdvmso1.dvm.kl 192.168.0.11 5 u 39 64 177 0.064 -2998.4 0.341
*rkdvmso1.dvm.kl 192.168.0.11 5 u 9 64 377 0.108 -3001.4 0.415
*rkdvmso1.dvm.kl 192.168.0.11 5 u - 64 77 0.104 -3036.6 1.050
rkdvmso1.dvm.kl 192.168.0.11 5 u - 64 1 0.133 -3039.7 0.001
*rkdvmso1.dvm.kl 192.168.0.11 5 u 10 64 1 0.080 -3043.2 0.858
*rkdvmso1.dvm.kl 192.168.0.11 5 u 24 64 17 0.067 -3052.3 0.236
*rkdvmso1.dvm.kl 192.168.0.11 5 u 1 64 37 0.067 -3052.3 2.093
*rkdvmso1.dvm.kl 192.168.0.11 5 u 6 64 177 0.096 -3061.2 0.230
Dom0 is properly synchronized however:
+pctick1a.dvm.kl 132.199.176.153 4 u 358 512 377 0.257 -1.471 0.752
*rkdvmhp2.dvm.kl 192.168.0.16 4 u 317 512 377 0.266 0.635 0.057
Can anybody explain, while the clock sticks at a 3 second offset for DomU?
/var/log/ntp says things ike these:
11 Oct 10:08:27 ntpd[12453]: synchronized to 192.168.0.61, stratum 5
11 Oct 10:08:27 ntpd[12453]: time reset -2.977411 s
11 Oct 10:09:39 ntpd[12453]: synchronized to 192.168.0.61, stratum 5
11 Oct 10:23:29 ntpd[12453]: time reset -3.018544 s
11 Oct 10:31:23 ntpd[12625]: synchronized to 192.168.0.61, stratum 5
11 Oct 10:31:23 ntpd[12625]: time reset -3.040169 s
Regards,
Ulrich
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Thomas Harold
2006-Oct-11 09:55 UTC
Re: [Xen-users] time synchronization problem (using NTP)
Ulrich Windl wrote:> Hi, > > using SLES10 I''m unable to synchronize the time of DomU with that of Dom0. There > is a persistent offset of about 3 seconds!(laughs) I just happen to be working on this myself. All of my DomUs are about 15 seconds fast compared to the Dom0 and I''ve been searching the archives to figure out why this happens. ------------------------------------------- According to the wiki: http://wiki.xensource.com/xenwiki/InstallationNotes xenU services The following services are not needed anymore: * ntpd the xenU uses the dom0 time If you want to run ntp in the domU, try: echo 1 > /proc/sys/xen/independent_wallclock ------------------------------------------- But if I look through news.gmane.org''s archive of xen-users (gmane.comp.emulators.xen.user), there''s conflicting advice given back in April 2006 (and Nov 2005... and May 2005). One camp says that we should run ntpd in Dom0, leave independent_wallclock set to zero and the DomUs should take their time from Dom0 and stay in sync. The other camp says that we should set independent_wallclock to 1 and run ntpd within the DomUs. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Thomas Schneider
2006-Oct-11 10:17 UTC
Re: [Xen-users] time synchronization problem (using NTP)
Hi! --On 11. Oktober 2006 05:55:44 -0400 Thomas Harold <tgh@tgharold.com> wrote:> Ulrich Windl wrote: >> Hi, >> >> using SLES10 I''m unable to synchronize the time of DomU with that of >> Dom0. There is a persistent offset of about 3 seconds! > > (laughs) I just happen to be working on this myself. All of my DomUs are > about 15 seconds fast compared to the Dom0 and I''ve been searching the > archives to figure out why this happens. >As far as my experience shows, none of the ntp / independent_wallclock combinations work. We got the same problem here with xen 3.0.2 and kernel 2.6.16. It looks like ntpd can''t set the time in DomUs with independent_wallclock set to 1 but the DomUs aren''t using the exact time of Dom0 with independent_wallclock set to 0. With knowing that we decided to wait for kernel release 2.6.18 because some stuff regarding hwclock und system-clock was changed. Maybe someone else can give us a better answer what to do with that? Thanks, Tom -- -- So Long, and Thanks for all the Fish - Douglas Adams, Hitchhikers Guide to the Galaxy -- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Roger Lucas
2006-Oct-11 13:09 UTC
RE: [Xen-users] time synchronization problem (using NTP)
Have you tried chrony ? We use chrony on our Xen-3.0.2-2 servers and it is happily running on both Dom-0 and the Dom-Us. If you are running (K)Ubuntu and have "universe" repositories enabled, installation is as simple as "apt-get install chrony". A quick change to "/etc/chrony/chrony.conf" to set the NTP servers you want to use and (if you are reliably connected to the Internet) removing the "offline" parameter for them will have you ready to (re-)start the chrony daemon and sync the clocks. FWIW, we have "/proc/sys/xen/independent_wallclock" set to 0 on the Dom0 and the DomU''s. Regards, Roger> -----Original Message----- > From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On > Behalf Of Thomas Schneider > Sent: 11 October 2006 11:17 > Cc: xen-users@lists.xensource.com > Subject: Re: [Xen-users] time synchronization problem (using NTP) > > Hi! > > --On 11. Oktober 2006 05:55:44 -0400 Thomas Harold <tgh@tgharold.com> wrote: > > > Ulrich Windl wrote: > >> Hi, > >> > >> using SLES10 I''m unable to synchronize the time of DomU with that of > >> Dom0. There is a persistent offset of about 3 seconds! > > > > (laughs) I just happen to be working on this myself. All of my DomUs are > > about 15 seconds fast compared to the Dom0 and I''ve been searching the > > archives to figure out why this happens. > > > > As far as my experience shows, none of the ntp / independent_wallclock > combinations work. We got the same problem here with xen 3.0.2 and kernel > 2.6.16. It looks like ntpd can''t set the time in DomUs with > independent_wallclock set to 1 but the DomUs aren''t using the exact time of > Dom0 with independent_wallclock set to 0. > > With knowing that we decided to wait for kernel release 2.6.18 because some > stuff regarding hwclock und system-clock was changed. > > Maybe someone else can give us a better answer what to do with that? > > Thanks, > > Tom > > -- > -- So Long, and Thanks for all the Fish > - Douglas Adams, Hitchhikers Guide to the Galaxy -- > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Christoph Purrucker
2006-Oct-11 17:02 UTC
Re: [Xen-users] time synchronization problem (using NTP)
Hi Thomas,> As far as my experience shows, none of the ntp / independent_wallclock > combinations work. We got the same problem here with xen 3.0.2 and > kernel 2.6.16. It looks like ntpd can''t set the time in DomUs with > independent_wallclock set to 1 but the DomUs aren''t using the exact time > of Dom0 with independent_wallclock set to 0.I''m running 2.6.16-2-xen-k7 with XEN 3.0 on both Dom0 and DomUs. A cron job on Dom0 runs "ntpdate ptbtime1.ptb.de" every night (shifting the time some milliseconds). All DomUs are perfectly in sync. /proc/sys/xen/independent_wallclock = 0 in all DomUs. cu cp _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ulrich Windl
2006-Oct-12 08:37 UTC
Re: [Xen-users] time synchronization problem (using NTP)
On 11 Oct 2006 at 5:55, Thomas Harold wrote:> Ulrich Windl wrote: > > Hi, > > > > using SLES10 I''m unable to synchronize the time of DomU with that of Dom0. There > > is a persistent offset of about 3 seconds! > > (laughs) I just happen to be working on this myself. All of my DomUs > are about 15 seconds fast compared to the Dom0 and I''ve been searching > the archives to figure out why this happens.Hi! I''ve dumped VMware server because it was unable to keep the time. XEN looked much better, but now I''m puzzled: Each DomU has it''s own kernel with a software clock: Why shouldn''t it be possible to tune that software clock? Using Dom0''s clock (IMHO) takes away some of the freedoms DomU offers: Testing new kernels with probably different time settings. OK, normally you want to have "just the correct time".> > ------------------------------------------- > > According to the wiki: > > http://wiki.xensource.com/xenwiki/InstallationNotes > > xenU services > > The following services are not needed anymore: > > * ntpd > > the xenU uses the dom0 timeAre there any statements how close both times (Dom0 and DomU) are together? Three seconds seems way too far apart for me. (It seems another DomU has a different constant offset)> > If you want to run ntp in the domU, try: echo 1 > > /proc/sys/xen/independent_wallclockOK, trying that. So far ntpd could sync at least. I''ll have to monitor offset and stability however before I make any conclusion. Thanks for the hint!> > ------------------------------------------- > > But if I look through news.gmane.org''s archive of xen-users > (gmane.comp.emulators.xen.user), there''s conflicting advice given back > in April 2006 (and Nov 2005... and May 2005). > > One camp says that we should run ntpd in Dom0, leave > independent_wallclock set to zero and the DomUs should take their time > from Dom0 and stay in sync.It''s probably simpler and needs less resources IF IT WORKS.> > The other camp says that we should set independent_wallclock to 1 and > run ntpd within the DomUs.This will add more flexibility (e.g. testing different versions or configurations of NTP) IF IT WORKS. Now I''m looking for a version that simply works. Regards, Ulrich _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Roger Lucas
2006-Oct-12 10:32 UTC
RE: [Xen-users] time synchronization problem (using NTP)
Firstly, I am really not a clock expert....> On 11 Oct 2006 at 14:09, Roger Lucas wrote: > > > Have you tried chrony ? > > Hi! > > As I sayid months before, NTP is not designed fo fix a bad clock implementation, > so does chrony? I''m tempted to say: if NTP is unable to work inside XEN, but > succeeds outside, there''s a problem with XEN. >As I understand it, chrony will constantly adjust the system clock time to keep it in line with the remote NTP servers. It can track frequency drift etc so your system clock changes are as smooth as possible.> > > > We use chrony on our Xen-3.0.2-2 servers and it is happily running on both Dom-0 and the Dom-Us. > > > > If you are running (K)Ubuntu and have "universe" repositories enabled, installation is as simple > as "apt-get install chrony". A > > quick change to "/etc/chrony/chrony.conf" to set the NTP servers you want to use and (if you are > reliably connected to the Internet) > > removing the "offline" parameter for them will have you ready to (re-)start the chrony daemon and > sync the clocks. > > > > FWIW, we have "/proc/sys/xen/independent_wallclock" set to 0 on the Dom0 and the DomU''s. > > That seemed to have helped (just running for a short time now): > > remote refid st t when poll reach delay offset jitter > =============================================================================> *rkdvmso1.dvm.kl 192.168.0.11 5 u 17 64 377 0.072 -6.540 0.009 > +rksapas01.dvm.k 192.168.0.61 6 u 8 64 377 0.123 -2.231 0.006 > *rkdvmso1.dvm.kl 192.168.0.11 5 u 1 64 377 0.080 -4.302 0.053 >We have tried several different NTP-based solutions to keep all our servers in sync and chrony is the only one that has worked painlessly on every platform we have tried including several different linux distributions and VMs. Don''t ask me to list the ones that didn''t work - it was a long time ago and I can''t remember the details any more. IMHO, chrony is one of the wonderful *nix tools that "just works". Don''t forget, however, that chrony takes some time (IIRC about 15 mins) to synchronise up to the remote servers and accurately calculate the system time error. Only when it has an accurate estimate of the error will it start to adjust the local system clock. BR, Roger _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Thomas Harold
2006-Oct-12 11:25 UTC
Re: [Xen-users] time synchronization problem (using NTP)
Roger Lucas wrote:> As I understand it, chrony will constantly adjust the system clock > time to keep it in line with the remote NTP servers. It can track > frequency drift etc so your system clock changes are as smooth as > possible.That might not be too bad in a DomU. NTP is considered by most to be the "gold standard" of keeping clocks in sync, but as long as chrony isn''t jumping the clock forward/backward constantly, it would be acceptable. For my uses, I simply want the DomUs to track Dom0 time within a few dozen msec (NTP typically keeps the Dom0s all within 2-3 msec on our network).> Don''t forget, however, that chrony takes some time (IIRC about 15 > mins) to synchronise up to the remote servers and accurately > calculate the system time error. Only when it has an accurate > estimate of the error will it start to adjust the local system clock.That happens with NTP as well. You can speed up the synchronization and stabilization time period by using "iburst" at the end of the server lines in ntp.conf. NTP, when it starts up, will then send off a volley of packets to the time servers instead of just one, which gives it enough data to start correcting the clock sooner. The "iburst" parameter seems to cause the server clocks to be fairly correct within a few minutes. But since the DomUs get their time from Dom0 when they startup, it''s a bit of a moot point (the DomU clocks are hopefully already accurate, chrony/ntp just need to keep the DomU clocks from drifting). ... We originally started with OpenNTPd on our servers (other servers, not the Xen boxes), but I''ve spent a lot of time reading the old archives of comp.protocols.time.ntp and the lengthy online documentation about time keeping, so we tackled the learning curve of NTP. So when it came time to setup the Xen servers, NTP in Dom0 seemed like a natural fit. Accurate time can be a bit engrossing, there''s always the question in the back of your mind... "can I get it just a little more accurate?". While I haven''t built a GPS reference clock yet for our network, it''s only going to be a matter of time ($85 + misc parts + soldering skills). ... Once I accomplish a few other things this week, I''ll try to come back around and play with chrony or the independent_wallclock setting. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Ulrich Windl
2006-Oct-13 08:59 UTC
Re: [Xen-users] time synchronization problem (using NTP)
On 11 Oct 2006 at 19:02, Christoph Purrucker wrote:> Hi Thomas, > > > As far as my experience shows, none of the ntp / independent_wallclock > > combinations work. We got the same problem here with xen 3.0.2 and > > kernel 2.6.16. It looks like ntpd can''t set the time in DomUs with > > independent_wallclock set to 1 but the DomUs aren''t using the exact time > > of Dom0 with independent_wallclock set to 0. > > I''m running 2.6.16-2-xen-k7 with XEN 3.0 on both Dom0 and DomUs. A cron > job on Dom0 runs "ntpdate ptbtime1.ptb.de" every night (shifting the > time some milliseconds). All DomUs are perfectly in sync. > /proc/sys/xen/independent_wallclock = 0 in all DomUs. >A plain "ntpd" also does the job once XEN''s own synchronization (wich must be buggy) is disabled (sysctl -w xen.independent_wallclock=1): (2 DomUs): rksapas01:~ # ntpq -p remote refid st t when poll reach delay offset jitter =============================================================================*rkdvmso1.dvm.kl 192.168.0.11 5 u 60 128 377 0.081 0.004 0.431 rksapas01:~ # ntpq -p rkdvmsod remote refid st t when poll reach delay offset jitter =============================================================================*rkdvmso1.dvm.kl 192.168.0.11 5 u 233 256 377 0.088 0.038 0.010 +rkdvmas01.dvm.k 192.168.0.61 6 u 10 256 377 0.117 0.033 0.248 (Dom0): rksapas01:~ # ntpq -p rkdvmso1 remote refid st t when poll reach delay offset jitter =============================================================================+pctick1a.dvm.kl 132.199.176.153 4 u 731 1024 377 0.164 -0.840 0.772 *rkdvmhp2.dvm.kl 192.168.0.16 4 u 712 1024 377 0.338 1.489 0.034 (All offsets are milliseconds, Configuration not complete yet) Regards, Ulrich _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Thomas Harold
2006-Dec-27 18:06 UTC
Re: [Xen-users] time synchronization problem (using NTP)
Ulrich Windl wrote:> On 11 Oct 2006 at 19:02, Christoph Purrucker wrote: > >> Hi Thomas, >> >>> As far as my experience shows, none of the ntp / independent_wallclock >>> combinations work. We got the same problem here with xen 3.0.2 and >>> kernel 2.6.16. It looks like ntpd can''t set the time in DomUs with >>> independent_wallclock set to 1 but the DomUs aren''t using the exact time >>> of Dom0 with independent_wallclock set to 0. >> I''m running 2.6.16-2-xen-k7 with XEN 3.0 on both Dom0 and DomUs. A cron >> job on Dom0 runs "ntpdate ptbtime1.ptb.de" every night (shifting the >> time some milliseconds). All DomUs are perfectly in sync. >> /proc/sys/xen/independent_wallclock = 0 in all DomUs. >> > > A plain "ntpd" also does the job once XEN''s own synchronization (wich must be > buggy) is disabled (sysctl -w xen.independent_wallclock=1):Just a small update and a question. Before setting wallclock=1 (mine was set to 0), the ntpd in my DomU was running about 2 minutes fast. There were entries in the ntpd.log file indicating that it was setting the clock back by 141 sec, but the "ntpq -p" output still showed an offset of 141 sec (-141000 msec). Now it seems to be working properly (ntpd is now able to adjust the clock). The ntp.conf file in the DomUs looks like: server time1.example.com iburst server time2.example.com iburst server time3.example.com iburst server time4.example.com iburst server time5.example.com iburst driftfile /var/lib/ntp/ntp.drift restrict default nomodify nopeer restrict 127.0.0.1 (The timeN.example.com are our internal time servers. These are usually Dom0s. They currently access public servers but may eventually talk to a GPS clock directly.) Still using Xen 3.0.2 here w/ the latest NTP version. ... Will the sysctl command make that a permanent change? Or do I need to edit /etc/sysctl.conf and add that line at the end? (I''d guess that I need to add that in my DomUs.) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users