Hi, Just wondering what is the best practice for timekeeping in Xen, on HVMs and PVMs. I found lots of information that DomUs should sync with it''s Dom0. However, there are also alot of posts for VMs timedrifting even if the Dom0 is properly synced. So it in the end, it seems it is better to treat the VM as an independent timekeeper and sync it using NTP. What are your thoughts? Is there an official answer out there? Should it be assumed that somehow a VM is improperly configured with regard to its Dom0 when it timedrifts? http://support.ntp.org/bin/view/Support/KnownOsIssues#Section_9.2.2.2. http://www.brookstevens.org/2010/06/xen-time-drift-and-ntp.html regards, Lloyd _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Lloyd Dizon wrote:> Hi, > Just wondering what is the best practice for timekeeping in Xen, on > HVMs and PVMs. > > I found lots of information that DomUs should sync with it''s Dom0. > > However, there are also alot of posts for VMs timedrifting even if the > Dom0 is properly synced. > > So it in the end, it seems it is better to treat the VM as an > independent timekeeper and sync it using NTP. > > What are your thoughts? Is there an official answer out there? > Should it be assumed that somehow a VM is improperly configured with > regard to its Dom0 when it timedrifts? >There''s a pretty good discussion on the Debian Xen Wiki, at http://wiki.debian.org/Xen scroll down or search the page for the section starting: ''clocksource/0: Time went backwards'' - which lists several workarounds note that some of them no longer work with Xen 4 (I think xen.independent_wallclock=1 is no longer recognized) -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
Hi, Miles Fidelman wrote:> There''s a pretty good discussion on the Debian Xen Wiki, at > http://wiki.debian.org/Xen scroll down or search the page for the > section starting: ''clocksource/0: Time went backwards'' - which lists > several workarounds note that some of them no longer work with Xen 4 (I > think xen.independent_wallclock=1 is no longer recognized)I second that. While it was possible to get a pretty accurate time in Xen 3 by using independent_wallclock, I have been unable to do so with Xen 4 so far. We have several Xen 4.0.1 production servers, and even by using NTP in dom0 AND domU we have a recurring problem where the clock goes out of sync for an hour or so. Using NTP in each domU with the "disable kernel" option seems to help a bit, but for some users needing less than a few seconds of time drift it is a real issue. I am currently thinking of downgrading some users to Xen 3 to work around this.. -- Regards, Remi Gacogne
On Fri, Apr 13, 2012 at 2:49 PM, Remi Gacogne <rgacogne-xen@valombre.net>wrote:> > Miles Fidelman wrote: > >> There''s a pretty good discussion on the Debian Xen Wiki, at >> http://wiki.debian.org/Xen scroll down or search the page for the >> section starting: ''clocksource/0: Time went backwards'' - which lists >> several workarounds note that some of them no longer work with Xen 4 (I >> think xen.independent_wallclock=1 is no longer recognized) >> > > I second that. While it was possible to get a pretty accurate time in Xen > 3 by using independent_wallclock, I have been unable to do so with Xen 4 so > far. > > We have several Xen 4.0.1 production servers, and even by using NTP in > dom0 AND domU we have a recurring problem where the clock goes out of sync > for an hour or so. Using NTP in each domU with the "disable kernel" option > seems to help a bit, but for some users needing less than a few seconds of > time drift it is a real issue. > > I am currently thinking of downgrading some users to Xen 3 to work around > this..This is annoying, as I am on the verge of migrating production machines from Xen 3 to 4. I am currently stuck because of time issues with the test machines I''ve already migrated. In our environnement 200 ms of time drift is unacceptable. Has anybody had feedback from Xen development teams? Wondering what will it take in order to fix this issue... _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On Fri, Apr 13, 2012 at 9:26 AM, Lloyd Dizon <thecarrionkind@gmail.com> wrote:> > > On Fri, Apr 13, 2012 at 2:49 PM, Remi Gacogne <rgacogne-xen@valombre.net> > wrote: >> >> >> Miles Fidelman wrote: >>> >>> There''s a pretty good discussion on the Debian Xen Wiki, at >>> http://wiki.debian.org/Xen scroll down or search the page for the section >>> starting: ''clocksource/0: Time went backwards'' - which lists several >>> workarounds note that some of them no longer work with Xen 4 (I think >>> xen.independent_wallclock=1 is no longer recognized) >> >> >> I second that. While it was possible to get a pretty accurate time in Xen >> 3 by using independent_wallclock, I have been unable to do so with Xen 4 so >> far. >> >> We have several Xen 4.0.1 production servers, and even by using NTP in >> dom0 AND domU we have a recurring problem where the clock goes out of sync >> for an hour or so. Using NTP in each domU with the "disable kernel" option >> seems to help a bit, but for some users needing less than a few seconds of >> time drift it is a real issue. >> >> I am currently thinking of downgrading some users to Xen 3 to work around >> this.. > > > This is annoying, as I am on the verge of migrating production machines from > Xen 3 to 4. > I am currently stuck because of time issues with the test machines I''ve > already migrated. In our environnement 200 ms of time drift is unacceptable. > > Has anybody had feedback from Xen development teams? Wondering what will it > take in order to fix this issue... >Does this help at all: http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3F Thanks, Todd -- Todd Deshane http://www.linkedin.com/in/deshantm http://blog.xen.org/ http://wiki.xen.org/
> Does this help at all: > http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3FWell, thank you but independent_wallclock doesn''t exist anymore on pvops kernel, and the link to blogspot seems broken so, no, not very much :) -- Regards, Remi Gacogne
Adding Xen-devel. What are the current timekeeping best practices now? Thanks, Todd ---------- Forwarded message ---------- From: Remi Gacogne <rgacogne-xen@valombre.net> Date: Fri, Apr 13, 2012 at 12:31 PM Subject: Re: [Xen-users] Xen timekeeping best practices To: Todd Deshane <todd.deshane@xen.org> Cc: Lloyd Dizon <thecarrionkind@gmail.com>, xen-users@lists.xen.org> Does this help at all: > http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3FWell, thank you but independent_wallclock doesn''t exist anymore on pvops kernel, and the link to blogspot seems broken so, no, not very much :) -- Regards, Remi Gacogne -- Todd Deshane http://www.linkedin.com/in/deshantm http://blog.xen.org/ http://wiki.xen.org/
Adding Xen-devel. What are the current timekeeping best practices now? Thanks, Todd ---------- Forwarded message ---------- From: Remi Gacogne <rgacogne-xen@valombre.net> Date: Fri, Apr 13, 2012 at 12:31 PM Subject: Re: [Xen-users] Xen timekeeping best practices To: Todd Deshane <todd.deshane@xen.org> Cc: Lloyd Dizon <thecarrionkind@gmail.com>, xen-users@lists.xen.org> Does this help at all: > http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3FWell, thank you but independent_wallclock doesn''t exist anymore on pvops kernel, and the link to blogspot seems broken so, no, not very much :) -- Regards, Remi Gacogne -- Todd Deshane http://www.linkedin.com/in/deshantm http://blog.xen.org/ http://wiki.xen.org/
On Apr 13, 2012, at 10:22 AM, Todd Deshane wrote:> Adding Xen-devel. > > What are the current timekeeping best practices now? >You may want to start here: xen-tarball-upacked-here/docs/misc/tscmode.txt I''m not sure there''s a silver-bullet solution, but that README is a very detailed description of high-performance timekeeping that discusses hardware caps, migration, and application time-sensitivity. Once you''re familiar with all that, then there will be questions about how your preferred synchronization mechanism (NTP?) interacts with the mechanisms discussed in the TSC README. Q
On Fri, Apr 13, 2012 at 12:21 PM, Qrux <qrux.qed@gmail.com> wrote:> On Apr 13, 2012, at 10:22 AM, Todd Deshane wrote: > >> Adding Xen-devel. >> What are the current timekeeping best practices now? > > You may want to start here: > xen-tarball-upacked-here/docs/misc/tscmode.txtIs that file available online anywhere? It sounds like an interesting read. -- Freddie Cash fjwcash@gmail.com _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On Apr 13, 2012, at 12:41 PM, Freddie Cash wrote:> On Fri, Apr 13, 2012 at 12:21 PM, Qrux <qrux.qed@gmail.com> wrote: >> On Apr 13, 2012, at 10:22 AM, Todd Deshane wrote: >> >>> Adding Xen-devel. >>> What are the current timekeeping best practices now? >> >> You may want to start here: >> xen-tarball-upacked-here/docs/misc/tscmode.txt > > Is that file available online anywhere? It sounds like an interesting read.[Just to be absolutely clear--I did not write this. This is an article written by Dan Magenheimer, whom I believe to be a Xen developer from Oracle.] I''ve posted it here: http://xlapp.org/xen-timekeeping/tscmode.txt I''ve been meaning to start a collection of information for Xen Timekeeping. If anyone else has any docs or images they''d like to contribute, I''d be happy to collect them here as reference material for when someone takes the plunge to update the Xen wiki. Q
On Apr 13, 2012, at 7:15 PM, Todd Deshane wrote:> On Fri, Apr 13, 2012 at 3:51 PM, Qrux <qrux.qed@gmail.com> wrote: >> >> I''ve been meaning to start a collection of information for Xen Timekeeping. If anyone else has any docs or images they''d like to contribute, I''d be happy to collect them here as reference material for when someone takes the plunge to update the Xen wiki. >> > > This type of cleanup would be great for a Xen Document Day1) I have no idea what kind of "cleanup" you''re referring to. I''m just trying to fill a void; specifically, the lack of a single place to look for timekeeping information. I''m not trying to summarize, even; I''m just looking to collect information. 2) As for a Xen Doc Day...I "attended" one of those once, for the better part of a working day. There were a few people on that IRC channel, and it felt like you had to know the "magic password"; the channel was completely silent, other than someone telling me that it was "just quiet" after I asked what the heck was going on (or supposed to go on). That didn''t seem a particularly good use of time. Personally, I''d rather just collect some information for now. Q
On Fri, Apr 13, 2012 at 10:48 PM, Qrux <qrux.qed@gmail.com> wrote:> > On Apr 13, 2012, at 7:15 PM, Todd Deshane wrote: > >> On Fri, Apr 13, 2012 at 3:51 PM, Qrux <qrux.qed@gmail.com> wrote: >>> >>> I''ve been meaning to start a collection of information for Xen Timekeeping. If anyone else has any docs or images they''d like to contribute, I''d be happy to collect them here as reference material for when someone takes the plunge to update the Xen wiki. >>> >> >> This type of cleanup would be great for a Xen Document Day > > 1) I have no idea what kind of "cleanup" you''re referring to. I''m just trying to fill a void; specifically, the lack of a single place to look for timekeeping information. I''m not trying to summarize, even; I''m just looking to collect information. >I was referring to the FAQ (http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3F) that turned out to be not useful to answer the question. The cleanup would be to actually provide an accurate answer on the xen wiki for a common question like this.> 2) As for a Xen Doc Day...I "attended" one of those once, for the better part of a working day. There were a few people on that IRC channel, and it felt like you had to know the "magic password"; the channel was completely silent, other than someone telling me that it was "just quiet" after I asked what the heck was going on (or supposed to go on). That didn''t seem a particularly good use of time. >Thanks for the feedback. I''ve added Lars to the CC. He may not have been around for that particular doc day to help coordinate. Lars can give you a better sense of how they run and what to expect.> Personally, I''d rather just collect some information for now.Great. We just like to try to coordinate these efforts so that the information lands (and/or is linked) on the appropriate wikis, developer docs, websites, etc. Thanks, Todd -- Todd Deshane http://www.linkedin.com/in/deshantm http://blog.xen.org/ http://wiki.xen.org/
On Fri, 2012-04-13 at 18:22 +0100, Todd Deshane wrote:> Adding Xen-devel. > > What are the current timekeeping best practices now?pvops kernels behave as if independent_wallclock == 1, so the existing advice: set /proc/sys/xen/independent_wallclock to 1 and run ntp on domU. still applies. Ian.> > Thanks, > Todd > > ---------- Forwarded message ---------- > From: Remi Gacogne <rgacogne-xen@valombre.net> > Date: Fri, Apr 13, 2012 at 12:31 PM > Subject: Re: [Xen-users] Xen timekeeping best practices > To: Todd Deshane <todd.deshane@xen.org> > Cc: Lloyd Dizon <thecarrionkind@gmail.com>, xen-users@lists.xen.org > > > > > Does this help at all: > > http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3F > > > Well, thank you but independent_wallclock doesn''t exist anymore on > pvops kernel, and the link to blogspot seems broken so, no, not very > much :) > > -- > > Regards, > > Remi Gacogne > > >
On Fri, 2012-04-13 at 18:22 +0100, Todd Deshane wrote:> Adding Xen-devel. > > What are the current timekeeping best practices now?pvops kernels behave as if independent_wallclock == 1, so the existing advice: set /proc/sys/xen/independent_wallclock to 1 and run ntp on domU. still applies. Ian.> > Thanks, > Todd > > ---------- Forwarded message ---------- > From: Remi Gacogne <rgacogne-xen@valombre.net> > Date: Fri, Apr 13, 2012 at 12:31 PM > Subject: Re: [Xen-users] Xen timekeeping best practices > To: Todd Deshane <todd.deshane@xen.org> > Cc: Lloyd Dizon <thecarrionkind@gmail.com>, xen-users@lists.xen.org > > > > > Does this help at all: > > http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3F > > > Well, thank you but independent_wallclock doesn''t exist anymore on > pvops kernel, and the link to blogspot seems broken so, no, not very > much :) > > -- > > Regards, > > Remi Gacogne > > >
Hi Ian,>> What are the current timekeeping best practices now? > > pvops kernels behave as if independent_wallclock == 1, so the existing > advice: > set /proc/sys/xen/independent_wallclock to 1 and run ntp on > domU. > still applies.O, thanks! The thing is, we have several servers in Xen 4.0.1 using NTP in dom0 and domU. We didn''t have any problem keeping a correct time with Xen 3 on the same servers, but with Xen 4 we have a recurring problem where the clock goes out of sync for an hour or so, in dom0 and domU at the same time. We have done more tests since friday, and it seems that this issue does not occur on servers having a Intel Xeon L5630, but on older servers having E5405 or L5410 processors. Looking at the output from xm dmesg, olders servers have : (XEN) checking TSC synchronization across 8 CPUs: passed. (XEN) Platform timer appears to have unexpectedly wrapped 10 or more times. (XEN) Platform timer is 3.579MHz ACPI PM Timer (XEN) CPUIDLE: disabled due to no HPET. Force enable with ''cpuidle''. where newer have : (XEN) TSC is reliable, synchronization unnecessary (XEN) Platform timer is 14.318MHz HPET I''m wondering if my issue could simply be linked to the fact that HPET is not available ? On newer servers, the ouput of # xm debug-key s; xm dmesg | tail is : (XEN) TSC marked as reliable, warp = 0 (count=4) (XEN) dom3: mode=0,ofs=0x42f57c62a36,khz=2133464,inc=1 (XEN) dom4: mode=0,ofs=0x430b996b94f,khz=2133464,inc=1 (XEN) dom5(hvm): mode=0,ofs=0x470ba656d26,khz=2133464,inc=1 (XEN) dom10(hvm): mode=0,ofs=0x6baa148c81f66,khz=2133464,inc=1 (XEN) No domains have emulated TSC whereas it is on olders servers : (XEN) Platform timer appears to have unexpectedly wrapped 10 or more times (XEN) TSC has constant rate, deep Cstates possible, so not reliable, warp=2310 (count=8) (XEN) dom1: mode=0,ofs=0x4379529ffa3,khz=2000068,inc=1,vtsc count: 5579056116 kernel, 18906545 user (XEN) dom2: mode=0,ofs=0x43a55ddb79e,khz=2000068,inc=1,vtsc count: 12684066480 kernel, 25277162 user (XEN) dom3: mode=0,ofs=0x43bd882b29e,khz=2000068,inc=1,vtsc count: 35568040315 kernel, 27356321 user Any idea would be appreciated ! Thanks,
On Mon, Apr 16, 2012 at 11:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> > On Fri, 2012-04-13 at 18:22 +0100, Todd Deshane wrote: > > Adding Xen-devel. > > > > What are the current timekeeping best practices now? > > pvops kernels behave as if independent_wallclock == 1, so the existing > advice: > set /proc/sys/xen/independent_wallclock to 1 and run ntp on > domU. > still applies.This also applies to Xen 4.0 (4.0.0) if I understood correctly? If so and based on previous posts can we resume that this is what is recommended: | Xen3 | Xen4 | --------------------------------------------------------------------- PV | independent_wallclock = 0 or 1 | independent_wallclock = 1 | --------------------------------------------------------------------- HVM | independent_wallclock = 0 | independent_wallclock = 1 | --------------------------------------------------------------------- With priority to independent_wallclock=1 and if activated, ntp required.
On Mon, 2012-04-16 at 15:45 +0100, Lloyd Dizon wrote:> On Mon, Apr 16, 2012 at 11:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > > On Fri, 2012-04-13 at 18:22 +0100, Todd Deshane wrote: > > > Adding Xen-devel. > > > > > > What are the current timekeeping best practices now? > > > > pvops kernels behave as if independent_wallclock == 1, so the existing > > advice: > > set /proc/sys/xen/independent_wallclock to 1 and run ntp on > > domU. > > still applies. > > This also applies to Xen 4.0 (4.0.0) if I understood correctly?This is entirely a function of the guest kernel version and not the hypervisor> If so and based on previous posts can we resume that this is what is > recommended:If you s/Xen3/Classic XenoLinux Kernels/ and s/Xen4/Paravirt Ops Linux Kernels/g then I think it is somewhat accurate, at least for the PV line. I must confess I don''t really know for the HVM line. I suspect that it is actually: Classic XenoLinux -- no PV time source, independent_wallclock means nothing, run NTP in the guest Paravirt ops kernels _without_ PV time PVHVM extensions -- no PV time source, independent_wallclock means nothing, run NTP in the guest Paravirt ops kernels _with_ PV time PVHVM extensions -- PV time source, independent_wallclock always == 1, run NTP. So in short always run NTP in an HVM guest, regardless of the presence or absence of PV time extensions.> | Xen3 | Xen4 | > --------------------------------------------------------------------- > PV | independent_wallclock = 0 or 1 | independent_wallclock = 1 | > --------------------------------------------------------------------- > HVM | independent_wallclock = 0 | independent_wallclock = 1 | > --------------------------------------------------------------------- > > With priority to independent_wallclock=1 and if activated, ntp required.
On Mon, 2012-04-16 at 15:45 +0100, Lloyd Dizon wrote:> On Mon, Apr 16, 2012 at 11:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > > On Fri, 2012-04-13 at 18:22 +0100, Todd Deshane wrote: > > > Adding Xen-devel. > > > > > > What are the current timekeeping best practices now? > > > > pvops kernels behave as if independent_wallclock == 1, so the existing > > advice: > > set /proc/sys/xen/independent_wallclock to 1 and run ntp on > > domU. > > still applies. > > This also applies to Xen 4.0 (4.0.0) if I understood correctly?This is entirely a function of the guest kernel version and not the hypervisor> If so and based on previous posts can we resume that this is what is > recommended:If you s/Xen3/Classic XenoLinux Kernels/ and s/Xen4/Paravirt Ops Linux Kernels/g then I think it is somewhat accurate, at least for the PV line. I must confess I don''t really know for the HVM line. I suspect that it is actually: Classic XenoLinux -- no PV time source, independent_wallclock means nothing, run NTP in the guest Paravirt ops kernels _without_ PV time PVHVM extensions -- no PV time source, independent_wallclock means nothing, run NTP in the guest Paravirt ops kernels _with_ PV time PVHVM extensions -- PV time source, independent_wallclock always == 1, run NTP. So in short always run NTP in an HVM guest, regardless of the presence or absence of PV time extensions.> | Xen3 | Xen4 | > --------------------------------------------------------------------- > PV | independent_wallclock = 0 or 1 | independent_wallclock = 1 | > --------------------------------------------------------------------- > HVM | independent_wallclock = 0 | independent_wallclock = 1 | > --------------------------------------------------------------------- > > With priority to independent_wallclock=1 and if activated, ntp required.
On Mon, Apr 16, 2012 at 4:57 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Mon, 2012-04-16 at 15:45 +0100, Lloyd Dizon wrote: >> On Mon, Apr 16, 2012 at 11:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> > >> > On Fri, 2012-04-13 at 18:22 +0100, Todd Deshane wrote: >> > > Adding Xen-devel. >> > > >> > > What are the current timekeeping best practices now? >> > >> > pvops kernels behave as if independent_wallclock == 1, so the existing >> > advice: >> > set /proc/sys/xen/independent_wallclock to 1 and run ntp on >> > domU. >> > still applies. >> >> This also applies to Xen 4.0 (4.0.0) if I understood correctly? > > This is entirely a function of the guest kernel version and not the > hypervisor > >> If so and based on previous posts can we resume that this is what is >> recommended: > > If you s/Xen3/Classic XenoLinux Kernels/ and s/Xen4/Paravirt Ops Linux > Kernels/g then I think it is somewhat accurate, at least for the PV > line. > > I must confess I don''t really know for the HVM line. I suspect that it > is actually: > > Classic XenoLinux -- no PV time source, independent_wallclock means > nothing, run NTP in the guest > > Paravirt ops kernels _without_ PV time PVHVM extensions -- no PV time > source, independent_wallclock means nothing, run NTP in the guest > > Paravirt ops kernels _with_ PV time PVHVM extensions -- PV time source, > independent_wallclock always == 1, run NTP. > > So in short always run NTP in an HVM guest, regardless of the presence > or absence of PV time extensionsI do not fully understand the difference yet between and Classic XenoLinux and Paravirt Ops Kernel (will dig the info later) but here is the resume: ------------------------------------------------------------------------------------------------------------------- | Xen3: Classic XenoLinux Kern | Xen4: Paravirt Ops Linux Kern | -------------------------------------------------------------------------------------------------------------------------- PV | independent_wallclock = 0 or 1 | independent_wallclock = 1 | -------------------------------------------------------------------------------------------------------------------------- HVM | -> no PV timesource, no ind. wallclock, run NTP | -> w/o PV time PVHVM extensions, no ind. wallclock, run NTP | | | -> PV time source, ind. wallclock always 1, run NTP | -------------------------------------------------------------------------------------------------------------------------- Should we put this somewhere (wiki) until somebody else proves this is inaccurate ?
On Tue, 2012-04-17 at 08:41 +0100, Lloyd Dizon wrote:> On Mon, Apr 16, 2012 at 4:57 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > On Mon, 2012-04-16 at 15:45 +0100, Lloyd Dizon wrote: > >> On Mon, Apr 16, 2012 at 11:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > >> > > >> > On Fri, 2012-04-13 at 18:22 +0100, Todd Deshane wrote: > >> > > Adding Xen-devel. > >> > > > >> > > What are the current timekeeping best practices now? > >> > > >> > pvops kernels behave as if independent_wallclock == 1, so the existing > >> > advice: > >> > set /proc/sys/xen/independent_wallclock to 1 and run ntp on > >> > domU. > >> > still applies. > >> > >> This also applies to Xen 4.0 (4.0.0) if I understood correctly? > > > > This is entirely a function of the guest kernel version and not the > > hypervisor > > > >> If so and based on previous posts can we resume that this is what is > >> recommended: > > > > If you s/Xen3/Classic XenoLinux Kernels/ and s/Xen4/Paravirt Ops Linux > > Kernels/g then I think it is somewhat accurate, at least for the PV > > line. > > > > I must confess I don''t really know for the HVM line. I suspect that it > > is actually: > > > > Classic XenoLinux -- no PV time source, independent_wallclock means > > nothing, run NTP in the guest > > > > Paravirt ops kernels _without_ PV time PVHVM extensions -- no PV time > > source, independent_wallclock means nothing, run NTP in the guest > > > > Paravirt ops kernels _with_ PV time PVHVM extensions -- PV time source, > > independent_wallclock always == 1, run NTP. > > > > So in short always run NTP in an HVM guest, regardless of the presence > > or absence of PV time extensions > > I do not fully understand the difference yet between and Classic > XenoLinux and Paravirt Ops Kernel (will dig the info later)Roughly: classic == old out-of-tree Xen patches, static compile time support for Xen. This is the old "2.6.18-xen.hg" tree and includes the "SuSE forward port kernels" which are the classic patches kept up to date with recent kernels. pvops == patches in upstream Linux, dynamic support for Xen enabled at kernel boot time. pvops supported domU from ~2.6.24 and dom0 from 3.0 onwards.> but here > is the resume: > > ------------------------------------------------------------------------------------------------------------------- > | Xen3: Classic XenoLinux Kern | > Xen4: Paravirt Ops Linux Kern | > -------------------------------------------------------------------------------------------------------------------------- > PV | independent_wallclock = 0 or 1 | > independent_wallclock = 1 | > -------------------------------------------------------------------------------------------------------------------------- > HVM | -> no PV timesource, no ind. wallclock, run NTP | -> w/o PV > time PVHVM extensions, no ind. wallclock, run NTP | > | | -> PV > time source, ind. wallclock always 1, run NTP | > --------------------------------------------------------------------------------------------------------------------------It got a bit mangled in transit but it looks reasonable. You should remove "Xen3" and "Xen4" from the titles since the hypervisor version has no bearing on any of this. You can just as easily run a pvops kernel on Xen3.x or a classic kernel on Xen4.x (and of course you can mix and match those on a single host) The 3->4 major number transition was mostly just a version number bump, it didn''t indicate any major change in the API or anything like that -- any PV kernel should run on any Xen from 3.0 onwards, right through to 4.2 (and beyond).> Should we put this somewhere (wiki) until somebody else proves this is > inaccurate ?Please do put it on the wiki, I think there was a reference to a page (a FAQ?) further up thread which seems like as good a place as any to add it. Ian.
On Tue, 2012-04-17 at 08:41 +0100, Lloyd Dizon wrote:> On Mon, Apr 16, 2012 at 4:57 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > On Mon, 2012-04-16 at 15:45 +0100, Lloyd Dizon wrote: > >> On Mon, Apr 16, 2012 at 11:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > >> > > >> > On Fri, 2012-04-13 at 18:22 +0100, Todd Deshane wrote: > >> > > Adding Xen-devel. > >> > > > >> > > What are the current timekeeping best practices now? > >> > > >> > pvops kernels behave as if independent_wallclock == 1, so the existing > >> > advice: > >> > set /proc/sys/xen/independent_wallclock to 1 and run ntp on > >> > domU. > >> > still applies. > >> > >> This also applies to Xen 4.0 (4.0.0) if I understood correctly? > > > > This is entirely a function of the guest kernel version and not the > > hypervisor > > > >> If so and based on previous posts can we resume that this is what is > >> recommended: > > > > If you s/Xen3/Classic XenoLinux Kernels/ and s/Xen4/Paravirt Ops Linux > > Kernels/g then I think it is somewhat accurate, at least for the PV > > line. > > > > I must confess I don''t really know for the HVM line. I suspect that it > > is actually: > > > > Classic XenoLinux -- no PV time source, independent_wallclock means > > nothing, run NTP in the guest > > > > Paravirt ops kernels _without_ PV time PVHVM extensions -- no PV time > > source, independent_wallclock means nothing, run NTP in the guest > > > > Paravirt ops kernels _with_ PV time PVHVM extensions -- PV time source, > > independent_wallclock always == 1, run NTP. > > > > So in short always run NTP in an HVM guest, regardless of the presence > > or absence of PV time extensions > > I do not fully understand the difference yet between and Classic > XenoLinux and Paravirt Ops Kernel (will dig the info later)Roughly: classic == old out-of-tree Xen patches, static compile time support for Xen. This is the old "2.6.18-xen.hg" tree and includes the "SuSE forward port kernels" which are the classic patches kept up to date with recent kernels. pvops == patches in upstream Linux, dynamic support for Xen enabled at kernel boot time. pvops supported domU from ~2.6.24 and dom0 from 3.0 onwards.> but here > is the resume: > > ------------------------------------------------------------------------------------------------------------------- > | Xen3: Classic XenoLinux Kern | > Xen4: Paravirt Ops Linux Kern | > -------------------------------------------------------------------------------------------------------------------------- > PV | independent_wallclock = 0 or 1 | > independent_wallclock = 1 | > -------------------------------------------------------------------------------------------------------------------------- > HVM | -> no PV timesource, no ind. wallclock, run NTP | -> w/o PV > time PVHVM extensions, no ind. wallclock, run NTP | > | | -> PV > time source, ind. wallclock always 1, run NTP | > --------------------------------------------------------------------------------------------------------------------------It got a bit mangled in transit but it looks reasonable. You should remove "Xen3" and "Xen4" from the titles since the hypervisor version has no bearing on any of this. You can just as easily run a pvops kernel on Xen3.x or a classic kernel on Xen4.x (and of course you can mix and match those on a single host) The 3->4 major number transition was mostly just a version number bump, it didn''t indicate any major change in the API or anything like that -- any PV kernel should run on any Xen from 3.0 onwards, right through to 4.2 (and beyond).> Should we put this somewhere (wiki) until somebody else proves this is > inaccurate ?Please do put it on the wiki, I think there was a reference to a page (a FAQ?) further up thread which seems like as good a place as any to add it. Ian.
On Fri, Apr 13, 2012 at 6:31 PM, Remi Gacogne <rgacogne-xen@valombre.net> wrote:> >> Does this help at all: >> >> http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3F > > > Well, thank you but independent_wallclock doesn''t exist anymore on pvops > kernel,I think it still exists, I found the info below in the last SUSE 11 SP 2 release (13.04.2012). This versions runs the 3.0.10 linux kernel with includes pvops. http://www.suse.com/releasenotes/x86_64/SUSE-SLES/11-SP2/ 11.6.6.10. Time Synchronization in titlevirtualized Domains with NTP Paravirtualized (PV) DomUs usually receive the time from the hypervisor. If you want to run "ntp" in PV DomUs, the DomU must be decoupled from the Dom0''s time. At runtime, this is done with: echo 1 > /proc/sys/xen/independent_wallclock To set this at boot time: either append "independent_wallclock=1" to kernel cmd line in DomU''s grub configuration file or append "xen.independent_wallclock = 1" to /etc/sysctl.conf in the DomU. If you encounter time synchronization issues with Paravirtualized Domains, we encourage you to use NTP. http://www.suse.com/releasenotes/x86_64/SUSE-SLES/11-SP2/
On Tue, Apr 17, 2012 at 10:16 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Tue, 2012-04-17 at 08:41 +0100, Lloyd Dizon wrote: >> On Mon, Apr 16, 2012 at 4:57 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> > On Mon, 2012-04-16 at 15:45 +0100, Lloyd Dizon wrote: >> >> On Mon, Apr 16, 2012 at 11:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> >> > >> >> > On Fri, 2012-04-13 at 18:22 +0100, Todd Deshane wrote: >> >> > > Adding Xen-devel. >> >> > > >> >> > > What are the current timekeeping best practices now? >> >> > >> >> > pvops kernels behave as if independent_wallclock == 1, so the existing >> >> > advice: >> >> > set /proc/sys/xen/independent_wallclock to 1 and run ntp on >> >> > domU. >> >> > still applies. >> >> >> >> This also applies to Xen 4.0 (4.0.0) if I understood correctly? >> > >> > This is entirely a function of the guest kernel version and not the >> > hypervisor >> > >> >> If so and based on previous posts can we resume that this is what is >> >> recommended: >> > >> > If you s/Xen3/Classic XenoLinux Kernels/ and s/Xen4/Paravirt Ops Linux >> > Kernels/g then I think it is somewhat accurate, at least for the PV >> > line. >> > >> > I must confess I don''t really know for the HVM line. I suspect that it >> > is actually: >> > >> > Classic XenoLinux -- no PV time source, independent_wallclock means >> > nothing, run NTP in the guest >> > >> > Paravirt ops kernels _without_ PV time PVHVM extensions -- no PV time >> > source, independent_wallclock means nothing, run NTP in the guest >> > >> > Paravirt ops kernels _with_ PV time PVHVM extensions -- PV time source, >> > independent_wallclock always == 1, run NTP. >> > >> > So in short always run NTP in an HVM guest, regardless of the presence >> > or absence of PV time extensions >> >> I do not fully understand the difference yet between and Classic >> XenoLinux and Paravirt Ops Kernel (will dig the info later) > > Roughly: > > classic == old out-of-tree Xen patches, static compile time support for > Xen. This is the old "2.6.18-xen.hg" tree and includes the "SuSE forward > port kernels" which are the classic patches kept up to date with recent > kernels. > > pvops == patches in upstream Linux, dynamic support for Xen enabled at > kernel boot time. pvops supported domU from ~2.6.24 and dom0 from 3.0 > onwards.Thanks also found info here: http://wiki.xensource.com/xenwiki/XenDom0Kernels.html>> but here >> is the resume: >> >> ------------------------------------------------------------------------------------------------------------------- >> | Xen3: Classic XenoLinux Kern | >> Xen4: Paravirt Ops Linux Kern | >> -------------------------------------------------------------------------------------------------------------------------- >> PV | independent_wallclock = 0 or 1 | >> independent_wallclock = 1 | >> -------------------------------------------------------------------------------------------------------------------------- >> HVM | -> no PV timesource, no ind. wallclock, run NTP | -> w/o PV >> time PVHVM extensions, no ind. wallclock, run NTP | >> | | -> PV >> time source, ind. wallclock always 1, run NTP | >> -------------------------------------------------------------------------------------------------------------------------- > > It got a bit mangled in transit but it looks reasonable. > > You should remove "Xen3" and "Xen4" from the titles since the hypervisor > version has no bearing on any of this. You can just as easily run a > pvops kernel on Xen3.x or a classic kernel on Xen4.x (and of course you > can mix and match those on a single host) > > The 3->4 major number transition was mostly just a version number bump, > it didn''t indicate any major change in the API or anything like that -- > any PV kernel should run on any Xen from 3.0 onwards, right through to > 4.2 (and beyond). > >> Should we put this somewhere (wiki) until somebody else proves this is >> inaccurate ? > > Please do put it on the wiki, I think there was a reference to a page (a > FAQ?) further up thread which seems like as good a place as any to add > it.Done. Anybody please correct the information if inaccurate. http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3F
Hi LLoyd,>> Well, thank you but independent_wallclock doesn''t exist anymore on pvops >> kernel, > > I think it still exists, I found the info below in the last SUSE 11 SP > 2 release (13.04.2012). > This versions runs the 3.0.10 linux kernel with includes pvops.I don''t have access to a running Suse SLES, but I am wondering if the documentation is up to date. What I know is that pv_ops kernel in Debian squeeze do not have independent_wallclock. I have not been able to find something related to an independent_wallclock parameter in a vanilla Linux 3.2 kernel either. In addition, there is a lot of ressources available on Internet stating that independent_wallclock does not exist on pv_ops kernel anymore, starting with http://wiki.debian.org/Xen.> Done. > Anybody please correct the information if inaccurate. > http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3FThanks. However, I think the table headers a bit misleading, because in> Classic XenoLinux Kern (up to 2.6.37)2.6.37 relates to the kernel version, whereas in> Paravirt Ops Linux Kern (from 3.0)it seems to be related to Xen version, if I am not mistaken. -- Regards, Remi Gacogne
On Tue, Apr 17, 2012 at 4:53 PM, Remi Gacogne <rgacogne-xen@valombre.net> wrote:>> http://wiki.xensource.com/wiki/Xen_FAQ_DomU#How_can_i_synchronize_a_dom0_clock.3F > > > Thanks. However, I think the table headers a bit misleading, because in > >> Classic XenoLinux Kern (up to 2.6.37) > > > 2.6.37 relates to the kernel version, whereas in > >> Paravirt Ops Linux Kern (from 3.0) > > > it seems to be related to Xen version, if I am not mistaken.No. Why would you think that? vanilla kernel can be used as dom0 starting from linux-3.0 (it''s currently at 3.3.2) -- Fajar
>> it seems to be related to Xen version, if I am not mistaken. > > No. Why would you think that? > > vanilla kernel can be used as dom0 starting from linux-3.0 (it''s > currently at 3.3.2)Ok, sorry. I guess I have been reading too quickly and mixed it up with Ian''s reponse : "The 3->4 major number transition was mostly just a version number bump, it didn''t indicate any major change in the API or anything like that -- any PV kernel should run on any Xen from 3.0 onwards, right through to 4.2 (and beyond). "
On Tue, Apr 17, 2012 at 11:53 AM, Remi Gacogne <rgacogne-xen@valombre.net> wrote:>>> Well, thank you but independent_wallclock doesn''t exist anymore on pvops >>> kernel, >> >> >> I think it still exists, I found the info below in the last SUSE 11 SP >> 2 release (13.04.2012). >> This versions runs the 3.0.10 linux kernel with includes pvops. > > > I don''t have access to a running Suse SLES, but I am wondering if the > documentation is up to date. > > What I know is that pv_ops kernel in Debian squeeze do not have > independent_wallclock. > I have not been able to find something related to an independent_wallclock > parameter in a vanilla Linux 3.2 kernel either.I think you should run ntp on DomU anyway. Found this info from http://wiki.debian.org/Xen#A.27clocksource.2BAC8-0.3ATimewentbackwards.27 xen.independent_wallclock sysctl setting is not available for newer squeeze kernels supporting pvops. If you have relied on it, you would have to run ntpd in dom0 and domUs. source If you do can you please post your results? Thanks. Lloyd
Hi,> I think you should run ntp on DomU anyway. > Found this info from > http://wiki.debian.org/Xen#A.27clocksource.2BAC8-0.3ATimewentbackwards.27 > > xen.independent_wallclock sysctl setting is not available for newer > squeeze kernels supporting pvops. If you have relied on it, you would > have to run ntpd in dom0 and domUs. source > > If you do can you please post your results?I am already running NTP in dom0 AND domUs. We are currently thinking that our problem may be related to the fact that the ACPI of the related hardware does not advertise HPET support, and we are investigating whether booting the kernel with hpet=force and cpuidle would fix this issue.
Has there been any resolution on this? I''ve been playing with a new installation of Xen (Xen 4 on Debian Squeeze), in preparation of migrating from Xen 3 on Debian Lenny. I''ve been having a devil of a time figuring out how to get time to work on domUs (or not figuring it out): - independent wallclock is no longer an available setting - doesn''t seem to matter if I install a separate ntpd on a domU or not, either way time on the domU seems to run very fast, just running further and further ahead of time on the dom0 and the ntp source Something is weird. Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
Miles Fidelman wrote:> Has there been any resolution on this? > > I''ve been playing with a new installation of Xen (Xen 4 on Debian > Squeeze), in preparation of migrating from Xen 3 on Debian Lenny. > > I''ve been having a devil of a time figuring out how to get time to > work on domUs (or not figuring it out): > - independent wallclock is no longer an available setting > - doesn''t seem to matter if I install a separate ntpd on a domU or > not, either way time on the domU seems to run very fast, just running > further and further ahead of time on the dom0 and the ntp source > > Something is weird.SOLVED IT: Took *clocksource=jiffies * out of my <vm>.cfg file and now everything is in sync. Looks like all the things needed to make time work in Xen 3/Lenny, break things in Xen 4/Squeeze. -- In theory, there is no difference between theory and practice. In practice, there is. .... Yogi Berra
On Thu, 12 Jul 2012, James Triplett wrote:> Did you ever get this resolved? I''m having, I think, the same problem...Hi James, I realize I didn''t keep the list updated of my hpet=force try, so I am sending this mail to xen-users. I am putting you in Bcc so your email @ doesn''t leak on a public mailing list if you didn''t intend it to. I''m afraid I did not. Booting the kernel with hpet=force and cpuidle didn''t help and, as we were running out of time, the time-sensitive domUs were moved to a dom0 where the HPET support is correctly working. For what it''s worth, I am seeing this issue on every Intel E5405 powered Xen 4 I have, and it is correctly working on every Intel L5630 powered Xen 4. I had no issue with the Intel E5405 hosts in Xen 3, the timekeeping issue happened after migrating to Xen 4. Hope this helps. Cheers, Remi Gacogne