thr3ads.net - Xen users - [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel. [Jan 2010]

If this information is useful, please help other people find it:
Share via:

mail ignored

2010-Jan-13 02:14 UTC

[Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On a selection of boxes, ntpd running in Xen Dom0 reproducibly
exhibits extermely high noise/jitter.
Switching back to -default, non-xen kernel ntpd runs with very low jitter/noise.

Question -- how can I ''tame'' ntpd noise & jitter when
running in Dom0?
 Is the problem a config issue, or a bug?

Already reported this downstream; everybody''s "stumped".

Running

 uname -a
  Linux test 2.6.31.8-0.1 #1 SMP 2009-12-15 23:55:40 +0100 x86_64
x86_64 x86_64 GNU/Linux

with @ grub,

 kernel /vmlinuz root=/dev/V0/ROOT rootfstype=ext4
rootflags=journal_checksum resume=/dev/V0/SWAP showopts splash=silent
selinux=0 SELINUX_INIT=NO elevator=cfq vga=0x31a console=tty0
console=ttyS0,57600n8

 cat /sys/devices/system/clocksource/clocksource0/current_clocksource
   tsc

on 5 different mobos, all with AMD Phenom II X4 Deneb Cores
(http://en.wikipedia.org/wiki/List_of_AMD_Phenom_microprocessors#.22Deneb.22_.28C2.2C_45_nm.2C_quad-core.29)

e.g.,

 grep "model name" /proc/cpuinfo
   model name      : AMD Phenom(tm) II X4 920 Processor
 grep flags /proc/cpuinfo
   flags           : fpu de tsc msr pae mce cx8 apic mtrr mca cmov pat
clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext
3dnow constant_tsc rep_good nonstop_tsc pni cx16 popcnt hypervisor
lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse
3dnowprefetch

running ntpd, within ~ 5 minutes of launch,

 ntpq -p -c rv

returns expected low-jitter, low-offset, full-reach results,

	       remote           refid      st t when poll reach   delay   offset
	jitter
	=============================================================================	 
*clock.fmt.he.ne .PPS.            1 u    2  128  377   14.472    6.176
	21.235
	  +otc2.psu.edu    128.4.1.1        2 u   68  128  377   98.808   17.478
	33.898
	  +clock.isc.org   216.200.141.8    2 u   67  128  377   15.491   -4.421
	22.236
	  +clock.sjc.he.ne .CDMA.           1 u   66  128  377   14.973   14.981
	23.539
	  +zorro.sf-bay.or 216.218.254.202  2 u   77  128  377   14.861   -5.981
	14.786
	  +rrcs-64-183-56- .GPS.            1 u  124  128  377   34.705   11.295
	36.253
	  +nist1.aol-ca.tr .ACTS.           1 u   62  128  377   16.302    8.190
	24.812

rebooting to,

 uname -a
  Linux test 2.6.31.8-0.1-xen #1 SMP 2009-12-15 23:55:40 +0100 x86_64
x86_64 x86_64 GNU/Linux

with @ grub,

 kernel /xen.gz loglvl=all loglvl_guest=all dom0_mem=1024M
vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1 iommu=1
cpufreq=xen:performance cpuidle
 module /vmlinuz-xen root=/dev/V0/ROOT rootfstype=ext4
rootflags=journal_checksum resume=/dev/V0/SWAP showopts splash=silent
selinux=0 SELINUX_INIT=NO elevator=cfq vga=0x31a console=tty0
console=xvc0,57600 iommu=off noirqdebug clocksource=xen

and,

 cat /sys/devices/system/clocksource/clocksource0/current_clocksource
   xen

 cat /etc/init.d/boot.local
   xenpm set-scaling-governor ondemand
   xenpm set-scaling-governor performance

after boot,

 cat /proc/cpuinfo | grep MHz
  cpu MHz         : 2800.000
  cpu MHz         : 2800.000
  cpu MHz         : 2800.000
  cpu MHz         : 2800.000

 xenpm get-cpufreq-para | grep governor
  current_governor     : performance
  current_governor     : performance
  current_governor     : performance
  current_governor     : performance

ntpd @ Dom0 return extremely high offset & jitter, never dropping to
''normal'', non-xen levels; e.g., even after 2-3 days of
running,

	      remote           refid      st t when poll reach   delay   offset  jitter
	=============================================================================	 
+clock.fmt.he.ne .PPS.            1 u   60   64  377   21.185  -1690.8
	658.866
	  +otc2.psu.edu    128.4.1.1        2 u   33   64  377  109.992  -1754.1
	659.825
	  +clock.isc.org   216.200.141.8    2 u    2   64  377   13.304  -1827.8
	676.431
	  *clock.sjc.he.ne .CDMA.           1 u   46   64  377   21.193  -1724.5
	655.151
	   zorro.sf-bay.or 216.218.254.202  2 u   53   64  377   15.396  -1711.7
	653.852
	  +rrcs-64-183-56- .GPS.            1 u   28   64  377   53.359  -1766.9
	657.547
	  +nist1.aol-ca.tr .ACTS.           1 u   37   64  377   22.613  -1742.4
	665.713

Also, with ntpd *off*,

@ boot to kernel-default
  sntp clock.isc.org && date && hwclock --show
      2009 Dec 28 10:38:49.000867 + 0.250226 +/- 0.023099 secs
      Mon Dec 28 10:38:50 PST 2009
      Mon 28 Dec 2009 10:38:51 AM PST  -0.998094 seconds

@ boot to kernel-xen
  sntp clock.isc.org && date && hwclock --show
!     2009 Dec 28 10:31:32.000206 + 28800.725203 +/- 0.013941 secs
!     Mon Dec 28 18:31:32 PST 2009
      Mon 28 Dec 2009 10:31:33 AM PST  -0.638684 seconds

note the dramatic differences in ''sntp'' &
''date'' results.  These
results can be reproduced swithicn back and forth between -xen &
-default kernel boots.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-13 07:13 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Tue, Jan 12, 2010 at 06:14:57PM -0800, mail ignored
wrote:> On a selection of boxes, ntpd running in Xen Dom0 reproducibly
> exhibits extermely high noise/jitter.
> Switching back to -default, non-xen kernel ntpd runs with very low
jitter/noise.
> 
> Question -- how can I ''tame'' ntpd noise & jitter when
running in Dom0?
>  Is the problem a config issue, or a bug?
> 
> Already reported this downstream; everybody''s "stumped".
> 
How many vcpus does your dom0 have? Have you configured domain weights
(ie. guaranteed cpu time for dom0?)

Have you tried dedicating a single cpu core only for dom0? 

-- Pasi
> Running
> 
>  uname -a
>   Linux test 2.6.31.8-0.1 #1 SMP 2009-12-15 23:55:40 +0100 x86_64
> x86_64 x86_64 GNU/Linux
> 
> with @ grub,
> 
>  kernel /vmlinuz root=/dev/V0/ROOT rootfstype=ext4
> rootflags=journal_checksum resume=/dev/V0/SWAP showopts splash=silent
> selinux=0 SELINUX_INIT=NO elevator=cfq vga=0x31a console=tty0
> console=ttyS0,57600n8
> 
>  cat /sys/devices/system/clocksource/clocksource0/current_clocksource
>    tsc
> 
> on 5 different mobos, all with AMD Phenom II X4 Deneb Cores
>
(http://en.wikipedia.org/wiki/List_of_AMD_Phenom_microprocessors#.22Deneb.22_.28C2.2C_45_nm.2C_quad-core.29)
> 
> e.g.,
> 
>  grep "model name" /proc/cpuinfo
>    model name      : AMD Phenom(tm) II X4 920 Processor
>  grep flags /proc/cpuinfo
>    flags           : fpu de tsc msr pae mce cx8 apic mtrr mca cmov pat
> clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext
> 3dnow constant_tsc rep_good nonstop_tsc pni cx16 popcnt hypervisor
> lahf_lm cmp_legacy extapic cr8_legacy abm sse4a misalignsse
> 3dnowprefetch
> 
> running ntpd, within ~ 5 minutes of launch,
> 
>  ntpq -p -c rv
> 
> returns expected low-jitter, low-offset, full-reach results,
> 
> 	       remote           refid      st t when poll reach   delay   offset
> 	jitter
> 
=============================================================================>
*clock.fmt.he.ne .PPS.            1 u    2  128  377   14.472    6.176
> 	21.235
> 	  +otc2.psu.edu    128.4.1.1        2 u   68  128  377   98.808   17.478
> 	33.898
> 	  +clock.isc.org   216.200.141.8    2 u   67  128  377   15.491   -4.421
> 	22.236
> 	  +clock.sjc.he.ne .CDMA.           1 u   66  128  377   14.973   14.981
> 	23.539
> 	  +zorro.sf-bay.or 216.218.254.202  2 u   77  128  377   14.861   -5.981
> 	14.786
> 	  +rrcs-64-183-56- .GPS.            1 u  124  128  377   34.705   11.295
> 	36.253
> 	  +nist1.aol-ca.tr .ACTS.           1 u   62  128  377   16.302    8.190
> 	24.812
> 
> rebooting to,
> 
>  uname -a
>   Linux test 2.6.31.8-0.1-xen #1 SMP 2009-12-15 23:55:40 +0100 x86_64
> x86_64 x86_64 GNU/Linux
> 
> with @ grub,
> 
>  kernel /xen.gz loglvl=all loglvl_guest=all dom0_mem=1024M
> vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1 iommu=1
> cpufreq=xen:performance cpuidle
>  module /vmlinuz-xen root=/dev/V0/ROOT rootfstype=ext4
> rootflags=journal_checksum resume=/dev/V0/SWAP showopts splash=silent
> selinux=0 SELINUX_INIT=NO elevator=cfq vga=0x31a console=tty0
> console=xvc0,57600 iommu=off noirqdebug clocksource=xen
> 
> and,
> 
>  cat /sys/devices/system/clocksource/clocksource0/current_clocksource
>    xen
> 
>  cat /etc/init.d/boot.local
>    xenpm set-scaling-governor ondemand
>    xenpm set-scaling-governor performance
> 
> after boot,
> 
>  cat /proc/cpuinfo | grep MHz
>   cpu MHz         : 2800.000
>   cpu MHz         : 2800.000
>   cpu MHz         : 2800.000
>   cpu MHz         : 2800.000
> 
>  xenpm get-cpufreq-para | grep governor
>   current_governor     : performance
>   current_governor     : performance
>   current_governor     : performance
>   current_governor     : performance
> 
> ntpd @ Dom0 return extremely high offset & jitter, never dropping to
> ''normal'', non-xen levels; e.g., even after 2-3 days of
running,
> 
> 	      remote           refid      st t when poll reach   delay   offset 
jitter
> 
=============================================================================>
+clock.fmt.he.ne .PPS.            1 u   60   64  377   21.185  -1690.8
> 	658.866
> 	  +otc2.psu.edu    128.4.1.1        2 u   33   64  377  109.992  -1754.1
> 	659.825
> 	  +clock.isc.org   216.200.141.8    2 u    2   64  377   13.304  -1827.8
> 	676.431
> 	  *clock.sjc.he.ne .CDMA.           1 u   46   64  377   21.193  -1724.5
> 	655.151
> 	   zorro.sf-bay.or 216.218.254.202  2 u   53   64  377   15.396  -1711.7
> 	653.852
> 	  +rrcs-64-183-56- .GPS.            1 u   28   64  377   53.359  -1766.9
> 	657.547
> 	  +nist1.aol-ca.tr .ACTS.           1 u   37   64  377   22.613  -1742.4
> 	665.713
> 
> Also, with ntpd *off*,
> 
> @ boot to kernel-default
>   sntp clock.isc.org && date && hwclock --show
>       2009 Dec 28 10:38:49.000867 + 0.250226 +/- 0.023099 secs
>       Mon Dec 28 10:38:50 PST 2009
>       Mon 28 Dec 2009 10:38:51 AM PST  -0.998094 seconds
> 
> @ boot to kernel-xen
>   sntp clock.isc.org && date && hwclock --show
> !     2009 Dec 28 10:31:32.000206 + 28800.725203 +/- 0.013941 secs
> !     Mon Dec 28 18:31:32 PST 2009
>       Mon 28 Dec 2009 10:31:33 AM PST  -0.638684 seconds
> 
> note the dramatic differences in ''sntp'' &
''date'' results.  These
> results can be reproduced swithicn back and forth between -xen &
> -default kernel boots.
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Fajar A. Nugraha

2010-Jan-13 07:37 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 9:14 AM, mail ignored <0.bugs.only.0@gmail.com>
wrote:> On a selection of boxes, ntpd running in Xen Dom0 reproducibly
> exhibits extermely high noise/jitter.
> Switching back to -default, non-xen kernel ntpd runs with very low
jitter/noise.
>  uname -a
>  Linux test 2.6.31.8-0.1 #1 SMP 2009-12-15 23:55:40 +0100 x86_64
> x86_64 x86_64 GNU/Linux
which kernel is it? vanilla? distro-provided? gentoo-xen kernel?

FWIW, a quick workaround is to use kernel 2.6.18. I''ve been using
RHEL''s kernel-xen (2.6.18-164.2.1.el5xen) and it works great WRT
time/ntp.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 16:44 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

hi

On Tue, Jan 12, 2010 at 11:13 PM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> How many vcpus does your dom0 have?
4, atm,

xm vcpu-list Domain-0
 Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
 Domain-0                             0     0     0   r--     273.0 any cpu
 Domain-0                             0     1     1   -b-     202.4 any cpu
 Domain-0                             0     2     3   -b-     179.1 any cpu
 Domain-0                             0     3     2   -b-     199.9 any cpu
> Have you configured domain weights (ie. guaranteed cpu time for dom0?)
no, i didn''t realize that was available.  how is that explicitly done?
 is that done using the CreditScheduler?
> Have you tried dedicating a single cpu core only for dom0?
no, not yet.  clearly, i could set maxcpus=1 @ grub.  would i also
need to pin the Dom0 cpu to a specific cpu, shomehow?

On Tue, Jan 12, 2010 at 11:37 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:> which kernel is it? vanilla? distro-provided? gentoo-xen kernel?
it''s distro-provided -- opensuse 11.2''s,

 uname -a
  Linux test 2.6.31.8-0.1-xen #1 SMP 2009-12-15 23:55:40 +0100 x86_64
x86_64 x86_64 GNU/Linux

 lsb_release -a
  LSB Version:
core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:graphics-4.0-noarch
  Distributor ID: SUSE LINUX
  Description:    openSUSE 11.2 (x86_64)
  Release:        11.2
  Codename:       n/a

 rpm -qa | grep -i ^xen-3
  xen-3.4.1_19718_04-28.1.x86_64
> FWIW, a quick workaround is to use kernel 2.6.18. I''ve been using
> RHEL''s kernel-xen (2.6.18-164.2.1.el5xen) and it works great WRT
> time/ntp.
good to know. although @downstream, they''re unable to reproduce this
problem, i''m able to on, as above, 5 different mobos from 3 different
vendors.

commonalities are (1) opensuse 11.2, (2) similar install profiles, (3)
AMD Deneb cores (Phenom II X4s)

whether this is an issue due to my local environmentS, opensuse or
xen, i''d like to find/fix it.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-13 17:13 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 08:44:08AM -0800, mail ignored
wrote:> hi
> 
> On Tue, Jan 12, 2010 at 11:13 PM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:
> > How many vcpus does your dom0 have?
> 
> 4, atm,
> 
> xm vcpu-list Domain-0
>  Name                                ID  VCPU   CPU State   Time(s) CPU
Affinity
>  Domain-0                             0     0     0   r--     273.0 any cpu
>  Domain-0                             0     1     1   -b-     202.4 any cpu
>  Domain-0                             0     2     3   -b-     179.1 any cpu
>  Domain-0                             0     3     2   -b-     199.9 any cpu
> 
> > Have you configured domain weights (ie. guaranteed cpu time for dom0?)
> 
> no, i didn''t realize that was available.  how is that explicitly
done?
>  is that done using the CreditScheduler?
> 
Yes, it''s done using Xen Credit Scheduler:
http://wiki.xensource.com/xenwiki/CreditScheduler

Use this to check the current settings:
	xm sched-credit -d Domain-0

And this to configure the weight:
	xm sched-credit -d Domain-0 -w <weight>

You might want to give dom0 for example weight of 384 or 512, 
so it''ll have more weight than the guests (the default weight is 256).

> > Have you tried dedicating a single cpu core only for dom0?
> 
> no, not yet.  clearly, i could set maxcpus=1 @ grub.  would i also
> need to pin the Dom0 cpu to a specific cpu, shomehow?
> 
You can use this for xen.gz in grub.conf:
dom0_vcpus=1 dom0_vcpus_pin

And then make sure the other domains (guests) don''t use pcpu 0, using
cpus=1-3 parameter in /etc/xen/<guest> cfgfiles.

Also you might want to dedicate dom0 fixed amount of memory, for example
512MB using dom0_mem=512M options for xen.gz, and then disable dom0
ballooning totally by specifying (dom0-min-mem 512) in /etc/xen/xend-config.sxp 


-- Pasi
> On Tue, Jan 12, 2010 at 11:37 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:
> > which kernel is it? vanilla? distro-provided? gentoo-xen kernel?
> 
> it''s distro-provided -- opensuse 11.2''s,
> 
>  uname -a
>   Linux test 2.6.31.8-0.1-xen #1 SMP 2009-12-15 23:55:40 +0100 x86_64
> x86_64 x86_64 GNU/Linux
> 
>  lsb_release -a
>   LSB Version:
>
core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:graphics-4.0-noarch
>   Distributor ID: SUSE LINUX
>   Description:    openSUSE 11.2 (x86_64)
>   Release:        11.2
>   Codename:       n/a
> 
>  rpm -qa | grep -i ^xen-3
>   xen-3.4.1_19718_04-28.1.x86_64
> 
> > FWIW, a quick workaround is to use kernel 2.6.18. I''ve been
using
> > RHEL''s kernel-xen (2.6.18-164.2.1.el5xen) and it works great
WRT
> > time/ntp.
> 
> good to know. although @downstream, they''re unable to reproduce
this
> problem, i''m able to on, as above, 5 different mobos from 3
different
> vendors.
> 
> commonalities are (1) opensuse 11.2, (2) similar install profiles, (3)
> AMD Deneb cores (Phenom II X4s)
> 
> whether this is an issue due to my local environmentS, opensuse or
> xen, i''d like to find/fix it.
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 17:23 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

hi

On Wed, Jan 13, 2010 at 9:13 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> Yes, it''s done using Xen Credit Scheduler:
> http://wiki.xensource.com/xenwiki/CreditScheduler
already reading up!
> You might want to give dom0 for example weight of 384 or 512,
> so it''ll have more weight than the guests (the default weight is
256).
missed the default value.  that explains that. thanks.
> dom0_vcpus=1 dom0_vcpus_pin
does that *presume* pin to cpu=0?

iiuc, i *should* be also able to

  dom0_vcpus=2 dom0_vcpus_pin

and specify that it pins to cpus=0&1?  or doe the pinning always use
the lowest CPU #s?
> And then make sure the other domains (guests) don''t use pcpu 0,
using
> cpus=1-3 parameter in /etc/xen/<guest> cfgfiles.
got it.
> Also you might want to dedicate dom0 fixed amount of memory, for example
> 512MB using dom0_mem=512M options for xen.gz, and then disable dom0
> ballooning totally by specifying (dom0-min-mem 512) in
/etc/xen/xend-config.sxp
that''s already done in my case, @ 1024.

thanks!

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 17:27 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

forgot to ask ...
> And this to configure the weight:
>        xm sched-credit -d Domain-0 -w <weight>
where/what is the "right" place/method to set those for survive across
boot?

grub? in /etc/init.d/(late boot script)?
> You might want to give dom0 for example weight of 384 or 512,
> so it''ll have more weight than the guests (the default weight is
256).
> You can use this for xen.gz in grub.conf:
> dom0_vcpus=1 dom0_vcpus_pin
how is maxcpus=N different?  is that a ''normal'' kernel
construct?  do
i still need it, if using dom0_vcpus?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-13 17:37 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 09:27:22AM -0800, mail ignored
wrote:> forgot to ask ...
> 
> > And this to configure the weight:
> >        xm sched-credit -d Domain-0 -w <weight>
> 
> where/what is the "right" place/method to set those for survive
across boot?
> 
> grub? in /etc/init.d/(late boot script)?
> 
/etc/rc.local maybe.. you need to do it after xend has started.
> > You might want to give dom0 for example weight of 384 or 512,
> > so it''ll have more weight than the guests (the default weight
is 256).
> 
> > You can use this for xen.gz in grub.conf:
> > dom0_vcpus=1 dom0_vcpus_pin
> 
> how is maxcpus=N different?  is that a ''normal'' kernel
construct?  do
> i still need it, if using dom0_vcpus?
>
you don''t need Linux kernel maxcpus= option when you specify Xen
hypervisor dom0_vcpus=X.
Dom0 Linux kernel won''t see more than X CPUs then.

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-13 17:37 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 09:23:46AM -0800, mail ignored
wrote:> 
> > You might want to give dom0 for example weight of 384 or 512,
> > so it''ll have more weight than the guests (the default weight
is 256).
> 
> missed the default value.  that explains that. thanks.
> 
> > dom0_vcpus=1 dom0_vcpus_pin
> 
> does that *presume* pin to cpu=0?
> 
> iiuc, i *should* be also able to
> 
>   dom0_vcpus=2 dom0_vcpus_pin
> 
> and specify that it pins to cpus=0&1?  or doe the pinning always use
> the lowest CPU #s?
> 
Yeah, "dom0_vcpus=2 dom0_vcpus_pin" should work.
You can verify it by running "xm vcpu-list" after reboot.

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 17:42 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

ok, almost there.
> On Wed, Jan 13, 2010 at 9:13 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:
>> And then make sure the other domains (guests) don''t use pcpu
0, using
>> cpus=1-3 parameter in /etc/xen/<guest> cfgfiles.
that goes on the extra=''... cpus=1-3 ...'' line, yes?

as a standalone entry in the cfg, it''s certainly not happy ...

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-13 17:45 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 09:42:53AM -0800, mail ignored
wrote:> ok, almost there.
> 
> > On Wed, Jan 13, 2010 at 9:13 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:
> >> And then make sure the other domains (guests) don''t use
pcpu 0, using
> >> cpus=1-3 parameter in /etc/xen/<guest> cfgfiles.
> 
> that goes on the extra=''... cpus=1-3 ...'' line, yes?
> 
> as a standalone entry in the cfg, it''s certainly not happy ...
>
No, it doesn''t go to extra.

cpus="1-3" 
vcpus=1

That should do it.

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 17:51 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 9:45 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> cpus="1-3"
agh!  quotes ...

now to set it all up, and watch ntpq for a bit ...

thanks.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 18:06 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 9:13 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> You can use this for xen.gz in grub.conf:
> dom0_vcpus=1 dom0_vcpus_pin
booting to,

    kernel /xen.gz ... dom0_vcpus=1 dom0_vcpus_pin sched=credit ...

i check,

 xm list
  Name                                        ID   Mem VCPUs
State   Time(s)
  Domain-0                                     0  1022     4     r-----     79.4

 xm vcpu-list Domain-0
  Name                                ID  VCPU   CPU State   Time(s)
CPU Affinity
  Domain-0                             0     0     0   -b-      37.4 0
  Domain-0                             0     1     1   -b-      13.0 1
  Domain-0                             0     2     2   r--      21.6 2
  Domain-0                             0     3     3   -b-      15.1 3

but,

 xm vcpu-set --help
  Usage: xm vcpu-set <Domain> <vCPUs>

  Set the number of active VCPUs for allowed for the domain.

then,

 xm vcpu-set Domain-0 1
 xm vcpu-list Domain-0

 (locks up the ssh session ... no response)

ssh in another session, try again,

 xm vcpu-list Domain-0

 (locks up the ssh session ... no response)


so, not sticking across boot ... and this odd lockup. thoughts?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-13 18:08 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 10:06:26AM -0800, mail ignored
wrote:> On Wed, Jan 13, 2010 at 9:13 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:
> > You can use this for xen.gz in grub.conf:
> > dom0_vcpus=1 dom0_vcpus_pin
> 
> booting to,
> 
>     kernel /xen.gz ... dom0_vcpus=1 dom0_vcpus_pin sched=credit ...
> 
> i check,
> 
>  xm list
>   Name                                        ID   Mem VCPUs
> State   Time(s)
>   Domain-0                                     0  1022     4     r-----    
79.4
> 
>  xm vcpu-list Domain-0
>   Name                                ID  VCPU   CPU State   Time(s)
> CPU Affinity
>   Domain-0                             0     0     0   -b-      37.4 0
>   Domain-0                             0     1     1   -b-      13.0 1
>   Domain-0                             0     2     2   r--      21.6 2
>   Domain-0                             0     3     3   -b-      15.1 3
> 
.. so something went wrong. You should only see 1 vcpus for dom0 now.
> but,
> 
>  xm vcpu-set --help
>   Usage: xm vcpu-set <Domain> <vCPUs>
> 
>   Set the number of active VCPUs for allowed for the domain.
> 
> then,
> 
>  xm vcpu-set Domain-0 1
>  xm vcpu-list Domain-0
> 
>  (locks up the ssh session ... no response)
> 
> ssh in another session, try again,
> 
>  xm vcpu-list Domain-0
> 
>  (locks up the ssh session ... no response)
> 
> 
> so, not sticking across boot ... and this odd lockup. thoughts?
>
What dom0 kernel you were running again? I can''t remember anymore.
Looks like it didn''t like when cpus were removed on-the-fly.

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 18:10 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 10:08 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> What dom0 kernel you were running again? I can''t remember anymore.
np.

it''s distro-provided -- opensuse 11.2''s,

 uname -a
 Linux test 2.6.31.8-0.1-xen #1 SMP 2009-12-15 23:55:40 +0100 x86_64
x86_64 x86_64 GNU/Linux

 lsb_release -a
 LSB Version:
core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:graphics-4.0-noarch
 Distributor ID: SUSE LINUX
 Description:    openSUSE 11.2 (x86_64)
 Release:        11.2
 Codename:       n/a

 rpm -qa | grep -i ^xen-3
 xen-3.4.1_19718_04-28.1.x86_64
> Looks like it didn''t like when cpus were removed on-the-fly.
which is fine, i suppose, as long as we can get it to stick at boot.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-13 18:12 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 10:10:32AM -0800, mail ignored
wrote:> On Wed, Jan 13, 2010 at 10:08 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:
> > What dom0 kernel you were running again? I can''t remember
anymore.
> 
> np.
> 
> it''s distro-provided -- opensuse 11.2''s,
> 
OK.. so you might want to report a bug to the opensuse bugzilla.
>  uname -a
>  Linux test 2.6.31.8-0.1-xen #1 SMP 2009-12-15 23:55:40 +0100 x86_64
> x86_64 x86_64 GNU/Linux
> 
>  lsb_release -a
>  LSB Version:
>
core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:graphics-4.0-noarch
>  Distributor ID: SUSE LINUX
>  Description:    openSUSE 11.2 (x86_64)
>  Release:        11.2
>  Codename:       n/a
> 
>  rpm -qa | grep -i ^xen-3
>  xen-3.4.1_19718_04-28.1.x86_64
> 
> > Looks like it didn''t like when cpus were removed on-the-fly.
> 
> which is fine, i suppose, as long as we can get it to stick at boot.
>
"dom0_vcpus=1 dom0_vcpus=pin" options should do it.
Verify with "xm dmesg" that there parameters are really there.

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 18:25 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 10:12 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> OK.. so you might want to report a bug to the opensuse bugzilla.
so it *is* a bug.  ok, i''ll file @ opensuse. thanks.
> "dom0_vcpus=1 dom0_vcpus=pin" options should do it.
> Verify with "xm dmesg" that there parameters are really there.
seem to be,

xm dmesg | egrep -i "dom0|vcp"
 (XEN) Command line: loglvl=all loglvl_guest=all dom0_mem=1024M
vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1 iommu=1
cpufreq=xen:performance cpuidle dom0_vcpus=1 dom0_vcpus_pin
sched=credit
 (XEN)  Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x8e1000
 (XEN)  Dom0 alloc.:   0000000216000000->0000000218000000 (253952
pages to be allocated)
 (XEN) Dom0 has maximum 4 VCPUs
 (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times
to switch
input to Xen)

but it is, then.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-13 18:39 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 10:25:45AM -0800, mail ignored
wrote:> On Wed, Jan 13, 2010 at 10:12 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:
> > OK.. so you might want to report a bug to the opensuse bugzilla.
> 
> so it *is* a bug.  ok, i''ll file @ opensuse. thanks.
> 
I think it''s actually two different bugs :)

1) You should be able to limit the number of dom0 vcpus by dom0_vcpus option
2) It shouldn''t crash when you remove vcpus from dom0 using vcpu-set.
> > "dom0_vcpus=1 dom0_vcpus=pin" options should do it.
> > Verify with "xm dmesg" that there parameters are really
there.
> 
> seem to be,
> 
> xm dmesg | egrep -i "dom0|vcp"
>  (XEN) Command line: loglvl=all loglvl_guest=all dom0_mem=1024M
> vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1 iommu=1
> cpufreq=xen:performance cpuidle dom0_vcpus=1 dom0_vcpus_pin
> sched=credit
>  (XEN)  Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x8e1000
>  (XEN)  Dom0 alloc.:   0000000216000000->0000000218000000 (253952
> pages to be allocated)
>  (XEN) Dom0 has maximum 4 VCPUs
>  (XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three
times to switch
> input to Xen)
> 
> but it is, then.
>
I''ll test dom0_vcpus option on my testbox at some point, with Xen
3.4.x.
I''ve been using it successfully with Xen 3.1.x, on RHEL5/CentOS5.

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 18:58 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 10:39 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> I think it''s actually two different bugs :)
>
> 1) You should be able to limit the number of dom0 vcpus by dom0_vcpus
option
> 2) It shouldn''t crash when you remove vcpus from dom0 using
vcpu-set.
thanks.

two bugs filed. fyi,

 "Bug 570432 - xen ignores cmd line params to limit/pin Dom0 cpu
utilization" https://bugzilla.novell.com/show_bug.cgi?id=570432
 "Bug 570443 - CRASH @ attempt to manually remove vcpus from dom0
using vcpu-set" https://bugzilla.novell.com/show_bug.cgi?id=570443
> I''ll test dom0_vcpus option on my testbox at some point, with Xen
3.4.x.
> I''ve been using it successfully with Xen 3.1.x, on RHEL5/CentOS5.
would be helpful to know.  thanks.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-13 20:38 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 08:39:40PM +0200, Pasi Kärkkäinen
wrote:> On Wed, Jan 13, 2010 at 10:25:45AM -0800, mail ignored wrote:
> > On Wed, Jan 13, 2010 at 10:12 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:
> > > OK.. so you might want to report a bug to the opensuse bugzilla.
> > 
> > so it *is* a bug.  ok, i''ll file @ opensuse. thanks.
> > 
> 
> I think it''s actually two different bugs :)
> 
> 1) You should be able to limit the number of dom0 vcpus by dom0_vcpus
option
> 2) It shouldn''t crash when you remove vcpus from dom0 using
vcpu-set.
> 
> > > "dom0_vcpus=1 dom0_vcpus=pin" options should do it.
> > > Verify with "xm dmesg" that there parameters are really
there.
> > 
> > seem to be,
> > 
> > xm dmesg | egrep -i "dom0|vcp"
> >  (XEN) Command line: loglvl=all loglvl_guest=all dom0_mem=1024M
> > vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1 iommu=1
> > cpufreq=xen:performance cpuidle dom0_vcpus=1 dom0_vcpus_pin
> > sched=credit
> >  (XEN)  Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x8e1000
> >  (XEN)  Dom0 alloc.:   0000000216000000->0000000218000000 (253952
> > pages to be allocated)
> >  (XEN) Dom0 has maximum 4 VCPUs
> >  (XEN) *** Serial input -> DOM0 (type ''CTRL-a''
three times to switch
> > input to Xen)
> > 
> > but it is, then.
> >
> 
> I''ll test dom0_vcpus option on my testbox at some point, with Xen
3.4.x.
> I''ve been using it successfully with Xen 3.1.x, on RHEL5/CentOS5.
> 
I just tried it on my Fedora 12 testbox with Xen 3.4.2, and I''m seeing
the same behaviour:

(XEN) Xen version 3.4.2 (root@(none)) (gcc version 4.4.2 20091027 (RedHat
4.4.2-7) (GCC) ) Mon Dec 14 06:53:18 EET 2009
(XEN) Latest ChangeSet: unavailable
(XEN) Command line: dom0_mem=1G loglvl=all guest_loglvl=all com1=38400,8n1
console=com1 dom0_vcpus=1
..
(XEN) Dom0 has maximum 4 VCPUs
..

I just sent an email to xen-devel asking about it..

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 20:41 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 12:38 PM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> I just tried it on my Fedora 12 testbox with Xen 3.4.2, and I''m
seeing
> the same behaviour:
>
> (XEN) Xen version 3.4.2 (root@(none)) (gcc version 4.4.2 20091027 (RedHat
4.4.2-7) (GCC) ) Mon Dec 14 06:53:18 EET 2009
> (XEN) Latest ChangeSet: unavailable
> (XEN) Command line: dom0_mem=1G loglvl=all guest_loglvl=all com1=38400,8n1
console=com1 dom0_vcpus=1
> ..
> (XEN) Dom0 has maximum 4 VCPUs
hrm.  bad to see.  good to know.
> I just sent an email to xen-devel asking about it..
i''ll keep an eye out there, as well.

thanks.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-13 21:55 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 10:38:18PM +0200, Pasi Kärkkäinen
wrote:> On Wed, Jan 13, 2010 at 08:39:40PM +0200, Pasi Kärkkäinen wrote:
> > On Wed, Jan 13, 2010 at 10:25:45AM -0800, mail ignored wrote:
> > > On Wed, Jan 13, 2010 at 10:12 AM, Pasi Kärkkäinen
<pasik@iki.fi> wrote:
> > > > OK.. so you might want to report a bug to the opensuse
bugzilla.
> > > 
> > > so it *is* a bug.  ok, i''ll file @ opensuse. thanks.
> > > 
> > 
> > I think it''s actually two different bugs :)
> > 
> > 1) You should be able to limit the number of dom0 vcpus by dom0_vcpus
option
> > 2) It shouldn''t crash when you remove vcpus from dom0 using
vcpu-set.
> > 
> > > > "dom0_vcpus=1 dom0_vcpus=pin" options should do
it.
> > > > Verify with "xm dmesg" that there parameters are
really there.
> > > 
> > > seem to be,
> > > 
> > > xm dmesg | egrep -i "dom0|vcp"
> > >  (XEN) Command line: loglvl=all loglvl_guest=all dom0_mem=1024M
> > > vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1 iommu=1
> > > cpufreq=xen:performance cpuidle dom0_vcpus=1 dom0_vcpus_pin
> > > sched=credit
> > >  (XEN)  Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x8e1000
> > >  (XEN)  Dom0 alloc.:   0000000216000000->0000000218000000
(253952
> > > pages to be allocated)
> > >  (XEN) Dom0 has maximum 4 VCPUs
> > >  (XEN) *** Serial input -> DOM0 (type
''CTRL-a'' three times to switch
> > > input to Xen)
> > > 
> > > but it is, then.
> > >
> > 
> > I''ll test dom0_vcpus option on my testbox at some point, with
Xen 3.4.x.
> > I''ve been using it successfully with Xen 3.1.x, on
RHEL5/CentOS5.
> > 
> 
> I just tried it on my Fedora 12 testbox with Xen 3.4.2, and I''m
seeing
> the same behaviour:
> 
> (XEN) Xen version 3.4.2 (root@(none)) (gcc version 4.4.2 20091027 (RedHat
4.4.2-7) (GCC) ) Mon Dec 14 06:53:18 EET 2009
> (XEN) Latest ChangeSet: unavailable
> (XEN) Command line: dom0_mem=1G loglvl=all guest_loglvl=all com1=38400,8n1
console=com1 dom0_vcpus=1
> ..
> (XEN) Dom0 has maximum 4 VCPUs
> ..
> 
> I just sent an email to xen-devel asking about it..
> 
like Andy pointed out on xen-devel, the xen.gz option is called
dom0_max_vcpus nowadays.

I verified dom0_max_vcpus=1 works OK with Xen 3.4.2.

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 22:10 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 1:55 PM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> like Andy pointed out on xen-devel, the xen.gz option is called
> dom0_max_vcpus nowadays.
>
> I verified dom0_max_vcpus=1 works OK with Xen 3.4.2.
and it does with my xen 3.4.1 on os11.2.  after boot,

xm dmesg | egrep -i "dom0|vcp"
(XEN) Command line: loglvl=all  ...  dom0_max_vcpus=1 dom0_vcpus_pin
sched=credit
(XEN)  Dom0 kernel: 64-bit, lsb, paddr 0x2000 -> 0x8e1000
(XEN)  Dom0 alloc.:   0000000216000000->0000000218000000 (253952 pages
to be allocated)
(XEN) Dom0 has maximum 1 VCPUs
(XEN) *** Serial input -> DOM0 (type ''CTRL-a'' three times
to switch
input to Xen)

xm vcpu-list Domain-0
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0                             0     0     0   r--      48.0 0

thanks for the help!  now back to watching ntpq ...

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-13 23:36 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Wed, Jan 13, 2010 at 2:10 PM, mail ignored <0.bugs.only.0@gmail.com>
wrote:> now back to watching ntpq ...
after applying the suggestions above, we''ve still got excessive
offset/jitter ...

boot to,

  title xen
    root (hd0,0)
    kernel /xen.gz loglvl=all loglvl_guest=all dom0_mem=1024M
vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1 iommu=1
cpufreq=xen:performance cpuidle dom0_max_vcpus=1 dom0_vcpus_pin
sched=credit
    module /vmlinuz-xen root=/dev/V0/ROOT rootfstype=ext4
rootflags=journal_checksum resume=/dev/V0/SWAP showopts splash=silent
selinux=0 SELINUX_INIT=NO elevator=cfq vga=0x31a console=tty0
console=xvc0,57600 iommu=off noirqdebug clocksource=xen
    module /initrd-xen

with,

 cat /proc/cpuinfo | grep MHz
  cpu MHz         : 2800.000
  cpu MHz         : 2800.000
  cpu MHz         : 2800.000
  cpu MHz         : 2800.000

 xenpm get-cpufreq-para | grep governor
  current_governor     : performance
  current_governor     : performance
  current_governor     : performance
  current_governor     : performance

 cat /etc/init.d/boot.local
  /usr/sbin/xenpm set-scaling-governor ondemand
  /usr/sbin/xenpm set-scaling-governor performance
  /usr/sbin/xm sched-credit -d Domain-0 -w 512

for DomU == "test_guest",

 cat /etc/xen/vm/test_guest.cfg
  ...
  cpus      = ''1-3''
  vcpus     = 3
  localtime = 0
  ...


 xm list
   Name                                        ID   Mem VCPUs
State   Time(s)
   Domain-0                                     0  1022     1
r-----     55.1
   test_guest                                   1  1024     3
-b----      7.9

 xm vcpu-list Domain-0
  Name                                ID  VCPU   CPU State   Time(s)
CPU Affinity
  Domain-0                             0     0     0   r--      56.6 0

 xm vcpu-list test_guest
  Name                                ID  VCPU   CPU State   Time(s)
CPU Affinity
  test_guest                           1     0     3   -b-       3.6 1-3
  test_guest                           1     1     2   -b-       2.2 1-3
  test_guest                           1     2     1   -b-       2.1 1-3

 xm sched-credit -d Domain-0
  Name                                ID Weight  Cap
  Domain-0                             0    256    0

??? hrm.  checking,

 cat /etc/init.d/boot.local
  /usr/sbin/xenpm set-scaling-governor ondemand
  /usr/sbin/xenpm set-scaling-governor performance
  /usr/sbin/xm sched-credit -d Domain-0 -w 512

 xm dmesg | grep -i scheduler
  (XEN) Using scheduler: SMP Credit Scheduler (credit)

 xm sched-credit -d Domain-0 -w 512
 xm sched-credit -d Domain-0
  Name                                ID Weight  Cap
  Domain-0                             0    512    0

 xm sched-credit -d test_guest
  Name                                ID Weight  Cap
  test_guest                           1    256    0

 grep dom0 /etc/xen/xend-config.sxp
  (dom0-min-mem 0)
  (enable-dom0-ballooning no)
  (dom0-cpus 0)

after ~ 1/2 hour,

 uptime
  15:29pm  up   0:34,  1 user,  load average: 0.00, 0.00, 0.02

ntp sync has reached state=2 @ stratum=2, but i''m still seeing the
excessive Xen offset/jitter -- only under Xen, e.g.

 ntpq -p -c rv
	assID=0 status=06d4 leap_none, sync_ntp, 13 events, event_peer/strat_chg,
	version="ntpd 4.2.4p8@1.1612-o Tue Dec 15 14:06:22 UTC 2009 (1)",
	processor="x86_64", system="Linux/2.6.31.8-0.1-xen",
leap=00, stratum=2,
	precision=-20, rootdelay=17.385, rootdispersion=125.916, peer=7611,
	refid=207.200.81.113,
	reftime=cef8d28b.247d36d5  Wed, Jan 13 2010 15:15:23.142, poll=6,
	clock=cef8d5e2.cd45b382  Wed, Jan 13 2010 15:29:38.801, state=2,
	offset=-90.707, frequency=-500.000, jitter=513.693, noise=25.184,
	stability=154.680, tai=0
	     remote           refid      st t when poll reach   delay   offset  jitter
	=============================================================================
+clock.fmt.he.ne .PPS.            1 u   37   64  377   16.531  -1495.9 522.188
	+otc2.psu.edu    129.6.15.29      2 u   24   64  377  108.729  -1514.8 516.844
	+clock.isc.org   .GPS.            1 u   31   64  377   20.388  -1509.1 518.560
	+clock.sjc.he.ne .CDMA.           1 u   48   64  377   20.548  -1475.4 514.975
	+zorro.sf-bay.or 216.218.254.202  2 u   48   64  377   19.124  -1476.2 520.580
	*nist1.aol-ca.tr .ACTS.           1 u    6   64  377   26.142  -1543.1 511.533

so CPU isolation is not involved ... ?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-14 16:32 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

hi,

On Wed, Jan 13, 2010 at 1:55 PM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> like Andy pointed out on xen-devel, the xen.gz option is called
> dom0_max_vcpus nowadays.
>
> I verified dom0_max_vcpus=1 works OK with Xen 3.4.2.
I''ve just been told @novell bugzilla,

 "Please be also advised that dom0_max_vcpus=1 is not recommended (and
not supported); a minimum of 2 should be set here."

Any thoughts/comments?  Should it be used @ =1, or not?  confusing, atm.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Fajar A. Nugraha

2010-Jan-15 02:19 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Thu, Jan 14, 2010 at 11:32 PM, mail ignored <0.bugs.only.0@gmail.com>
wrote:> hi,
>
> On Wed, Jan 13, 2010 at 1:55 PM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:
>> like Andy pointed out on xen-devel, the xen.gz option is called
>> dom0_max_vcpus nowadays.
>>
>> I verified dom0_max_vcpus=1 works OK with Xen 3.4.2.
>
> I''ve just been told @novell bugzilla,
>
>  "Please be also advised that dom0_max_vcpus=1 is not recommended (and
> not supported); a minimum of 2 should be set here."
>
> Any thoughts/comments?  Should it be used @ =1, or not?  confusing, atm.

Your original problem was that ntpd experience high noise/jitter. IMHO
dedicated dom0 cpu is not the fix for that, so that''s two different
issues.

First, about ntpd. Since you''re using Opensuse kernel, your best bet
is novell guys, since users on this list does not know what cause or
fix that issue. A quick workaround is to use Redhat''s kernel, which
worked fine for me with both Intel and AMD cpus. But from your posts I
gather you want to stick with Opensuse kernel.

Second, about dom0 cpu. It''s generally best to assign at least one
physical core/cpu exclusive for dom0 usage. What I did is use this on
grub.conf

	kernel /xen.gz-3.4.1 com2=115200,8n1 console=vga,com2 dom0_mem=768M
dom0_vcpus_pin

and this on xend-config.sxp

(dom0-cpus 1)

and set domUs to use cpus other than cpu0, with a config line like
"cpus=1-7"

The difference between this approach and the one you''re trying to use:
- I hard-limit dom0 memory. This is useful because dom0 won''t have to
balloon-down memory for domU usage, plus it doesn''t have to maintain
necessary memory metadata to support full physical available memory
(thus more efficient memory usage)
- I don''t limit dom0 cpu on xen.gz line, but I do it on
xend-config.sxp. What this means is that dom0 will start initially
with all available cpu, but when xend starts it changes to only 1 cpu.
The part "dom_vcpus_pin" ensures that the active cpu for dom0 will be
cpu0, thus you can assign other cpus to domUs. The benefit of this
approach is that it allows me to later add more cpus to dom0 when
necessary (for example, if I want to compile a new kernel on dom0).

If I understand correctly, dom0_max_vcpus=1 means that you hardlimit
dom0''s cpu to 1, thus you''ll be unable to add more cpu to dom0
without
reboot. This might be the reason why novell guys recommended against
it. You''ll have to ask them for the details though. Note that
"unsupported" does not allways mean it won''t work :D

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-15 03:11 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Thu, Jan 14, 2010 at 6:19 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:> Your original problem was that ntpd experience high noise/jitter. IMHO
> dedicated dom0 cpu is not the fix for that, so that''s two
different
> issues.
It certainly did not appear to make a difference.
> First, about ntpd. Since you''re using Opensuse kernel, your best
bet
> is novell guys, since users on this list does not know what cause or
> fix that issue.
As per my OP, "Already reported this downstream; everybody''s
"stumped".

I.e., it''s already been filed at novell, and discussed ad nauseum.  No
solution/progress there, simply statements that they can''t reproduce
the problem; of course, that''s with not identical environments.  I
*am* able to reproduce this issue on 5 different boxes, so I''m hard
pressed to believe that its any one box that''s at fault.,

There have been suggestions to come ''here'', though.  And,
there are
@novell types on these lists -- if more prominently on xen-devel
> Second, about dom0 cpu. It''s generally best to assign at least one
> physical core/cpu exclusive for dom0 usage. What I did is use this on
> grub.conf
>
>        kernel /xen.gz-3.4.1 com2=115200,8n1 console=vga,com2 dom0_mem=768M
> dom0_vcpus_pin
...> The difference between this approach and the one you''re trying to
use:
> - I hard-limit dom0 memory.
you''ll note from above that i already *have*limited dom0 memory, in
grub,

kernel /xen.gz loglvl=all loglvl_guest=all dom0_mem=1024M
vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1 iommu=1
cpufreq=xen:performance cpuidle dom0_max_vcpus=2 dom0_vcpus_pin
sched=credit

have already disabled ballooning in xend-config.sxp,

   (enable-dom0-ballooning no)
   (dom0-cpus 0),

and, have already isolated guests to the ''other'' cpus, e.g.,
in test_guest.cfg
 ...
 cpus      = ''2-3''
 vcpus     = 2
 ...

with this config I verify 2 cpus available to Dom0

 cat /proc/cpuinfo | grep processor
  processor       : 0
  processor       : 1

with 2 vcpus using those two cpus,

xm vcpu-list Domain-0
 Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
 Domain-0                             0     0     0   -b-      31.5 0
 Domain-0                             0     1     1   r--      24.6 1

and @ the guest, the ''other'' two cpus being used,

xm vcpu-list test_guest
 Name                              ID  VCPU   CPU State   Time(s) CPU Affinity
 test_guest                         1     0     3   -b-       5.0 2-3
 test_guest                         1     1     2   -b-       2.4 2-3
> This might be the reason why novell guys recommended against it.
> You''ll have to ask them for the details though.
already have ... waiting for clarification
> Note that "unsupported" does not allways mean it won''t
work :D
that''s certainly true!

the point remains, however, that ntpd withOUT xen''s presence, on
any/all hardware I have, runs quiet & stable.  clearly, "xen" is
involved.  whether it''s the opensuse implementation of kernel-xen or
Xen, or something inherent in Xen''s design, is unknown atm ...

thanks.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Fajar A. Nugraha

2010-Jan-15 03:23 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Fri, Jan 15, 2010 at 10:11 AM, mail ignored <0.bugs.only.0@gmail.com>
wrote:> xm vcpu-list Domain-0
>  Name                                ID  VCPU   CPU State   Time(s) CPU
Affinity
>  Domain-0                             0     0     0   -b-      31.5 0
>  Domain-0                             0     1     1   r--      24.6 1
With my setup, it''s like this
# cat /proc/cpuinfo | grep processor
processor	: 0

# xm vcpu-list 0
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0                             0     0     0   r--   93253.0 0
Domain-0                             0     1     -   --p     579.8 1
Domain-0                             0     2     -   --p      11.4 2
Domain-0                             0     3     -   --p       8.0 3
Domain-0                             0     4     -   --p       8.0 4
Domain-0                             0     5     -   --p      13.5 5
Domain-0                             0     6     -   --p      11.7 6
Domain-0                             0     7     -   --p      10.0 7

the benefit is that I can hotplug other cpus to dom0 if necessary later.
> the point remains, however, that ntpd withOUT xen''s presence, on
> any/all hardware I have, runs quiet & stable.  clearly, "xen"
is
> involved.  whether it''s the opensuse implementation of kernel-xen
or
> Xen, or something inherent in Xen''s design, is unknown atm ...
Hope you can solve this problem. When you do, please share your results.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-15 04:12 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

> Hope you can solve this problem. When you do, please share your results.
@ novell say it''s xen & ntp
@ ntp say it''s novell & xen
@ xen say it''s novell & ntp

this does not bode well :-(

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Fajar A. Nugraha

2010-Jan-15 04:20 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Fri, Jan 15, 2010 at 11:12 AM, mail ignored <0.bugs.only.0@gmail.com>
wrote:>> Hope you can solve this problem. When you do, please share your
results.
>
> @ novell say it''s xen & ntp
> @ ntp say it''s novell & xen
> @ xen say it''s novell & ntp
>
> this does not bode well :-(
That''s why I suggested you use a workaround: use 2.6.18 kernel, either
from RHEL or from xen.org :D
If this works -> novell problem
If this doesn''t work -> xen problem

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-15 04:43 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Thu, Jan 14, 2010 at 8:20 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:> That''s why I suggested you use a workaround: use 2.6.18 kernel,
either
> from RHEL or from xen.org :D
> If this works -> novell problem
> If this doesn''t work -> xen problem
i have no idea how to boot opensuse from a redhat kernel rpm -- or
even install it (e.g.
http://mirror.centos.org/centos/5.4/centosplus/x86_64/RPMS/kernel-xen-2.6.18-164.10.1.el5.centos.plus.x86_64.rpm)
without complaints about deps etc.

or do you mean build a new from src @xen or @redhat??

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Fajar A. Nugraha

2010-Jan-15 07:14 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Fri, Jan 15, 2010 at 11:43 AM, mail ignored <0.bugs.only.0@gmail.com>
wrote:> On Thu, Jan 14, 2010 at 8:20 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:
>> That''s why I suggested you use a workaround: use 2.6.18
kernel, either
>> from RHEL or from xen.org :D
>> If this works -> novell problem
>> If this doesn''t work -> xen problem
>
> i have no idea how to boot opensuse from a redhat kernel rpm -- or
> even install it (e.g.
>
http://mirror.centos.org/centos/5.4/centosplus/x86_64/RPMS/kernel-xen-2.6.18-164.10.1.el5.centos.plus.x86_64.rpm)
> without complaints about deps etc.
>
> or do you mean build a new from src @xen or @redhat??
xen.org''s 2.6.18 kernel -> build from src,
http://www.xen.org/products/xen_source.html

RHEL/Centos kernel rpm ->  quite tricky, admittedly :D
I suppose you could try something like this

wget -c
http://mirror.centos.org/centos/5.4/centosplus/x86_64/RPMS/kernel-xen-2.6.18-164.10.1.el5.centos.plus.x86_64.rpm
mkdir tmp-root
cd tmp-root
rpm2cpio ../kernel-xen-2.6.18-164.10.1.el5.centos.plus.x86_64.rpm | cpio -idmv
cp -a ./boot/* /boot
cp -a ./lib/modules/* /lib/modules

that is, extract the contents of kernel-xen rpm, and take only what''s
in /boot and /lib/modules. After this you need to build initrd. The
command should be mkinitrd (haven''t done this for a long time in
opensuse, so I forgot the exact syntax). Then you need to edit to add
the kernel and initrd entry to /boot/grub/menu.lst.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-15 07:42 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

ok, both _should_ be simple enough.

On Thu, Jan 14, 2010 at 11:14 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:> xen.org''s 2.6.18 kernel -> build from src,
> http://www.xen.org/products/xen_source.html
tried that; scads of errors @ make.

tried a build of opensuse kernel src, just to check my procedure -- works fine.
> RHEL/Centos kernel rpm ->  quite tricky, admittedly :D
> I suppose you could try something like this
>
> wget -c
http://mirror.centos.org/centos/5.4/centosplus/x86_64/RPMS/kernel-xen-2.6.18-164.10.1.el5.centos.plus.x86_64.rpm
> mkdir tmp-root
> cd tmp-root
> rpm2cpio ../kernel-xen-2.6.18-164.10.1.el5.centos.plus.x86_64.rpm | cpio
-idmv
> cp -a ./boot/* /boot
> cp -a ./lib/modules/* /lib/modules
>
> that is, extract the contents of kernel-xen rpm, and take only
what''s
> in /boot and /lib/modules. After this you need to build initrd. The
> command should be mkinitrd (haven''t done this for a long time in
> opensuse, so I forgot the exact syntax). Then you need to edit to add
> the kernel and initrd entry to /boot/grub/menu.lst.
it *is* just mkinitrd ... but, when i do,

 mkinitrd

Kernel image:   /boot/vmlinuz-2.6.18-164.10.1.el5.centos.plusxen
Initrd image:   /boot/initrd-2.6.18-164.10.1.el5.centos.plusxen
Root device:    /dev/root (mounted on / as ext4)
Resume device:  /dev/md2
setup-md.sh: md1 found multiple times
modprobe: Could not load
/lib/modules/2.6.18-164.10.1.el5.centos.plusxen/modules.dep: No such
file or directory
WARNING: no dependencies for kernel module ''raid0'' found.
modprobe: Could not load
/lib/modules/2.6.18-164.10.1.el5.centos.plusxen/modules.dep: No such
file or directory
WARNING: no dependencies for kernel module ''raid1'' found.
modprobe: Could not load
/lib/modules/2.6.18-164.10.1.el5.centos.plusxen/modules.dep: No such
file or directory
WARNING: no dependencies for kernel module ''raid456'' found.
...
...
...

where do i get the modules.dep file?  afaict, it''s not in the redhat
.rpm''s extract.

thanks.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-15 08:10 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Thu, Jan 14, 2010 at 11:42:43PM -0800, mail ignored
wrote:> ok, both _should_ be simple enough.
> 
> On Thu, Jan 14, 2010 at 11:14 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:
> > xen.org''s 2.6.18 kernel -> build from src,
> > http://www.xen.org/products/xen_source.html
> 
> tried that; scads of errors @ make.
> 
> tried a build of opensuse kernel src, just to check my procedure -- works
fine.
> 
> > RHEL/Centos kernel rpm ->  quite tricky, admittedly :D
> > I suppose you could try something like this
> >
> > wget -c
http://mirror.centos.org/centos/5.4/centosplus/x86_64/RPMS/kernel-xen-2.6.18-164.10.1.el5.centos.plus.x86_64.rpm
> > mkdir tmp-root
> > cd tmp-root
> > rpm2cpio ../kernel-xen-2.6.18-164.10.1.el5.centos.plus.x86_64.rpm |
cpio -idmv
> > cp -a ./boot/* /boot
> > cp -a ./lib/modules/* /lib/modules
> >
> > that is, extract the contents of kernel-xen rpm, and take only
what''s
> > in /boot and /lib/modules. After this you need to build initrd. The
> > command should be mkinitrd (haven''t done this for a long time
in
> > opensuse, so I forgot the exact syntax). Then you need to edit to add
> > the kernel and initrd entry to /boot/grub/menu.lst.
> 
> it *is* just mkinitrd ... but, when i do,
> 
>  mkinitrd
> 
> Kernel image:   /boot/vmlinuz-2.6.18-164.10.1.el5.centos.plusxen
> Initrd image:   /boot/initrd-2.6.18-164.10.1.el5.centos.plusxen
> Root device:    /dev/root (mounted on / as ext4)
> Resume device:  /dev/md2
> setup-md.sh: md1 found multiple times
> modprobe: Could not load
> /lib/modules/2.6.18-164.10.1.el5.centos.plusxen/modules.dep: No such
> file or directory
> WARNING: no dependencies for kernel module ''raid0'' found.
> modprobe: Could not load
> /lib/modules/2.6.18-164.10.1.el5.centos.plusxen/modules.dep: No such
> file or directory
> WARNING: no dependencies for kernel module ''raid1'' found.
> modprobe: Could not load
> /lib/modules/2.6.18-164.10.1.el5.centos.plusxen/modules.dep: No such
> file or directory
> WARNING: no dependencies for kernel module ''raid456''
found.
> ...
> ...
> ...
> 
> where do i get the modules.dep file?  afaict, it''s not in the
redhat
> .rpm''s extract.
> 
Run "depmod -a 2.6.18-164.10.1.el5.centos.plusxen".

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-15 16:30 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Fri, Jan 15, 2010 at 12:10 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> Run "depmod -a 2.6.18-164.10.1.el5.centos.plusxen".
i had run ''depmod -a'', as i always do when building opensuse
kernels,
with no effect.

i did not realize that "depmod -a 2.6.18-164.10.1.el5.centos.plusxen"
-- which works! -- was required. thanks.

sorry this is becoming a treatise on kernel-building, but i appreciate
the necessity to try to rule out xen ...

atm, i''ve extracted/installed from the centos rpm, as Fajar
instructed.  i''ve also run ''mkinitrd -f lvm2 -f md''

so now i have,

ls -al /boot/*el*
 -rw-r--r-- 1 root root   68951 2010-01-08 14:54
/boot/config-2.6.18-164.10.1.el5.centos.plusxen
 -rw-r--r-- 1 root root 8199432 2010-01-15 07:51
/boot/initrd-2.6.18-164.10.1.el5.centos.plusxen
 -rw-r--r-- 1 root root  116122 2010-01-08 14:55
/boot/symvers-2.6.18-164.10.1.el5.centos.plusxen.gz
 -rw-r--r-- 1 root root 1196044 2010-01-08 14:54
/boot/System.map-2.6.18-164.10.1.el5.centos.plusxen
 -rw-r--r-- 1 root root 2031384 2010-01-08 14:54
/boot/vmlinuz-2.6.18-164.10.1.el5.centos.plusxen
 -rw-r--r-- 1 root root  413719 2010-01-08 13:41
/boot/xen.gz-2.6.18-164.10.1.el5.centos.plus
 -rwxr-xr-x 1 root root  965728 2010-01-08 15:01
/boot/xen-syms-2.6.18-164.10.1.el5.centos.plus

i''ve setup grub, to use the
{vmlinuz,initrd}-2.6.18-164.10.1.el5.centos.plusxen.  at boot, i see
the hypervisor load, and the passoff to the centos kernel.  but it
then gets stuck in a loop, endlessly repeating "volume group VG0 not
found".

fwiw, "/boot" is on a linux native RAID-1 part, and "/" is
on a LV on
a RAID-1.  of course, works fine @ boot to opensuse kernel(s).

what else do I need to get that centos kernel to boot my box?  Or is
the 2.6.18* too old for support of LVM, RAID, etc (i doubt that ...)?

thanks.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-15 16:58 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Fri, Jan 15, 2010 at 8:30 AM, mail ignored <0.bugs.only.0@gmail.com>
wrote:> what else do I need to get that centos kernel to boot my box?  Or is
> the 2.6.18* too old for support of LVM, RAID, etc (i doubt that ...)?
fwiw, creating a "monster initrd", with ''mkinitrd
-A'', although
successful, fails to change the behavior -- still VolGrp not found.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-15 17:28 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Fri, Jan 15, 2010 at 8:58 AM, mail ignored <0.bugs.only.0@gmail.com>
wrote:> On Fri, Jan 15, 2010 at 8:30 AM, mail ignored
<0.bugs.only.0@gmail.com> wrote:
>> what else do I need to get that centos kernel to boot my box?  Or is
>> the 2.6.18* too old for support of LVM, RAID, etc (i doubt that ...)?
is the centos/redhat 2.6.18 kernel ext4-friendly?

opensuse 11.2 installs as ext4 by default, and i chose to stick with
it @ install.  my fstab includes,

 LABEL=ROOT /     ext4 relatime,acl,user_xattr,journal_checksum,barrier=1 1 1

since "/" is using extents, i can''t re-mount it as ext3 ...

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-15 17:33 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Fri, Jan 15, 2010 at 9:28 AM, mail ignored <0.bugs.only.0@gmail.com>
wrote:> is the centos/redhat 2.6.18 kernel ext4-friendly?
this

http://etbe.coker.com.au/2010/01/05/ext4-and-rhel5-centos5/

suggests that ext4 should not be a problem with this kernel ...

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Fajar A. Nugraha

2010-Jan-15 22:34 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Sat, Jan 16, 2010 at 12:33 AM, mail ignored <0.bugs.only.0@gmail.com>
wrote:> On Fri, Jan 15, 2010 at 9:28 AM, mail ignored
<0.bugs.only.0@gmail.com> wrote:
>> is the centos/redhat 2.6.18 kernel ext4-friendly?
>
> this
>
> http://etbe.coker.com.au/2010/01/05/ext4-and-rhel5-centos5/
>
> suggests that ext4 should not be a problem with this kernel ...
Last time I check it only recognize ext4dev. I''ll try this next week.
What does your partition setup looks like? Is it the default opensuse
layout, or did you customize it? last time I check grub can''t support
/boot on raid-1. Also, does the Centos kernel recognize the disks
succesfully (i.e. did you see messages about sda/sdb during boot)?

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-15 23:10 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

hi,

On Fri, Jan 15, 2010 at 2:34 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:> Last time I check it only recognize ext4dev. I''ll try this next
week.
in addition to the reference above, this,

  https://bugzilla.redhat.com/show_bug.cgi?id=485315#c13

suggests it does.  thanks.
> What does your partition setup looks like?
sda, 1TB
sdb, 1TB

/dev/md0 = 150 MB, ext3, RAID-1 (/dev/sda1 + /dev/sdb1)
/dev/md1 = *, raw LVM, RAID-1 (/dev/sda2 + /dev/sdb2)
    LVMs,
        /dev/VG0/ROOT  50GB, ext4
        /dev/VG0/SWAP  2 GB, swap
        /dev/VG0/HOME  *, ext4
> Is it the default opensuse layout, or did you customize it?
It''s certainly not the default.  But it is fully supported by
opensuse, and readily configurable in its Partitioner
> last time I check grub can''t support  /boot on raid-1.
 Works just fine on OpenSuse.  Using Centos kernel, the hypervisor &
kernel both seem to found/recognized just fine ...

cref e.g.,

http://www.linuxfoundation.org/collaborate/workgroups/linux-raid/preventing_against_a_failing_disk#Prepare_for_boot
" ... On Intel-compatibe hardware, there are two common boot loaders,
grub and lilo.
Both grub and lilo can only boot off a raid1. They cannot boot off
any other software raid device type ... "
> Also, does the Centos kernel recognize the disks
> succesfully (i.e. did you see messages about sda/sdb during boot)?
yes, both are there.  it''s the VG that''s it''s
complaining is not being found.

although the messages scroll by too quickly (i''ll find a serial cable
around here ...), i do see something about "udevadm ... Connection
Refused ...".

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-16 19:17 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Thu, Jan 14, 2010 at 8:20 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:>> @ novell say it''s xen & ntp
>> @ ntp say it''s novell & xen
>> @ xen say it''s novell & ntp
>>
> That''s why I suggested you use a workaround: use 2.6.18 kernel,
either
> from RHEL or from xen.org :D
> If this works -> novell problem
> If this doesn''t work -> xen problem
On Sat, Jan 16, 2010 at 5:05 AM, Fajar A. Nugraha <fajar@fajar.net>
wrote:> You might want to try latest pv_ops as well
> (http://wiki.xensource.com/xenwiki/XenParavirtOps), which should be
> easier to work with Opensuse since the version number is close.
it seems it''s _not_ the kernel -- unless there''s a coincident
kernel
problem across source/vendor ...

per
 http://lists.xensource.com/archives/html/xen-devel/2009-12/msg01127.html
 http://wiki.xensource.com/xenwiki/XenParavirtOps
 http://wiki.xensource.com/xenwiki/Kernel.org_Linux_on_Xen

cd /home/workspace && rm -rf linux-2.6-xen
git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
linux-2.6-xen
cd linux-2.6-xen
make clean
cp -a .config .config-old
wget -O .config
http://pasik.reaktio.net/xen/pv_ops-dom0-debug/config-2.6.31.4-pvops_dom0-x86_64
make oldconfig
  ...
  Xen PCI Frontend (XEN_PCIDEV_FRONTEND) [N/m/y/?] (NEW)           <-- m
  ...
  "IMQ" target support (NETFILTER_XT_TARGET_IMQ) [N/m/?] (NEW)    
<-- m
  ...
  IMQ (intermediate queueing device) support (IMQ) [M/y/?] (NEW)   <--
[enter]
  ...
  Block-device tap backend driver (XEN_BLKDEV_TAP) [N/m/y/?] (NEW) <-- m
  ...

make bzImage
make modules
make modules_install
strip vmlinux -o vmlinux-stripped
gzip vmlinux-stripped -c > vmlinuz-gzipped

ls -al .config \
        System.map \
        arch/x86/boot/bzImage \
        vmlinux \
        vmlinux-stripped \
        vmlinuz-gzipped

cp -a .config               /boot/config-2.6.31.6
cp -a System.map            /boot/System.map-2.6.31.6
cp -a arch/x86/boot/bzImage /boot/vmlinuz-2.6.31.6
mkinitrd
ls -al /boot/*2.6.31.6*
 -rw-r--r-- 1 root root    98510 2010-01-16 03:02 /boot/config-2.6.31.6
 -rw-r--r-- 1 root root 21304611 2010-01-16 04:14 /boot/initrd-2.6.31.6
 -rw-r--r-- 1 root root  1940879 2010-01-16 03:14 /boot/System.map-2.6.31.6
 -rw-r--r-- 1 root root  3751936 2010-01-16 03:14 /boot/vmlinuz-2.6.31.6


vi /boot/grub/menu.lst
...
title xen.git
 root   (hd0,0)
 kernel /xen.gz  loglvl=all loglvl_guest=all dom0_mem=1024M
vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1
 module /vmlinuz-2.6.31.6 root=/dev/VG0/ROOT rootfstype=ext4
rootflags=journal_checksum resume=/dev/VG0/SWAP nomodeset selinux=0
SELINUX_INIT=NO vga=0x31a console=tty0 console=xvc0,57600
 module /initrd-2.6.31.6

title xen.git DISABLED
 root (hd0,0)
 kernel /vmlinuz-2.6.31.6 root=/dev/VG_SVR/LV_SVR_DOM0_ROOT
rootfstype=ext4 rootflags=journal_checksum
resume=/dev/VG_SVR/LV_SVR_DOM0_SWAP nomodeset showopts splash=silent
selinux=0 SELINUX_INIT=NO elevator=cfq vga=0x31a console=tty0
console=ttyS0,57600n8
 module /initrd-2.6.31.6
...


boot to "xen.git"

 uname -a
  Linux test 2.6.31.6 #1 SMP Sat Jan 16 03:14:19 PST 2010 x86_64
x86_64 x86_64 GNU/Linux
 lsb_release -a
  LSB Version:
core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:graphics-4.0-noarch
  Distributor ID: SUSE LINUX
  Description:    openSUSE 11.2 (x86_64)
  Release:        11.2
  Codename:       n/a

service ntp stop
rm /etc/adjtime /var/
rm /var/lib/ntp/drift/ntp.drift

cat << EOF > /etc/sysconfig/clock
HWCLOCK="--utc"
SYSTOHC="yes"
TIMEZONE="America/Los_Angeles"
DEFAULT_TIMEZONE="US/Pacific"
EOF
hwclock --systohc --utc
service ntp start

sntp clock.isc.org && date && hwclock --utc --show
 2010 Jan 16 04:37:09.000729 + 1.435060 +/- 0.028944 secs
 Sat Jan 16 04:37:11 PST 2010
 Sat 16 Jan 2010 04:37:11 AM PST  -0.345191 seconds

... (wait ~ 6 hours) ...

date
	Sat Jan 16 11:01:42 PST 2010
ntpq -p -c rv
	assID=0 status=06c4 leap_none, sync_ntp, 12 events, event_peer/strat_chg,
	version="ntpd 4.2.4p8@1.1612 Mon Dec 28 17:26:30 UTC 2009 (1)",
	processor="x86_64", system="Linux/2.6.31.6", leap=00,
stratum=2,
	precision=-20, rootdelay=14.718, rootdispersion=135.995, peer=11390,
	refid=66.220.9.122,
	reftime=cefc89b2.a36f1342  Sat, Jan 16 2010 10:53:38.638, poll=6,
	clock=cefc8b82.7e523feb  Sat, Jan 16 2010 11:01:22.493, state=2,
	offset=-115.130, frequency=-500.000, jitter=464.242, noise=29.343,
	stability=154.680
	     remote           refid      st t when poll reach   delay   offset  jitter
	=============================================================================
*clock.fmt.he.ne .PPS.            1 u   29   64  377   21.438  -811.68 462.837
	+otc2.psu.edu    129.6.15.29      2 u    7   64  377  109.399  -850.99 470.522
	+clock.isc.org   .GPS.            1 u    7   64  377   21.440  -854.82 468.274
	+clock.sjc.he.ne .CDMA.           1 u   49   64  377   16.889  -780.55 469.481
	+zorro.sf-bay.or 216.218.254.202  2 u   59   64  377   20.366  -762.14 458.947
	+nist1.aol-ca.tr .ACTS.           1 u   38   64  377   16.260  -795.87 463.170

boot to "xen.git DISABLED"

uname -a
	Linux test 2.6.31.6 #1 SMP Sat Jan 16 09:14:19 PST 2010 x86_64 x86_64
x86_64 GNU/Linux

service ntp stop
rm /etc/adjtime /var/
rm /var/lib/ntp/drift/ntp.drift
service ntp start

... (wait ~ 10 minutes) ...
date
	Sat Jan 16 11:15:05 PST 2010
ntpq -p -c rv
	assID=0 status=0664 leap_none, sync_ntp, 6 events, event_peer/strat_chg,
	version="ntpd 4.2.4p8@1.1612 Mon Dec 28 17:26:30 UTC 2009 (1)",
	processor="x86_64", system="Linux/2.6.31.6", leap=00,
stratum=2,
	precision=-20, rootdelay=22.750, rootdispersion=975.184, peer=43091,
	refid=66.220.9.122,
	reftime=cefc8c7c.35ca466a  Sat, Jan 16 2010 11:05:32.210, poll=6,
	clock=cefc8eb9.25103a1f  Sat, Jan 16 2010 11:15:05.144, state=3,
	offset=27.534, frequency=0.000, jitter=6.740, noise=16.331,
	stability=0.000
	     remote           refid      st t when poll reach   delay   offset  jitter
	=============================================================================
*clock.fmt.he.ne .PPS.            1 u    4   64  377   18.112   10.541   5.892
	+otc2.psu.edu    147.84.59.145    2 u   64   64  377  101.103   12.106   8.621
	+clock.isc.org   .GPS.            1 u   57   64  377   14.990    4.830   9.622
	+clock.sjc.he.ne .CDMA.           1 u   59   64  377   16.503    9.299   9.440
	+zorro.sf-bay.or 216.218.254.202  2 u   56   64  377   22.212    6.323  11.031
	+nist1.aol-ca.tr .ACTS.           1 u   60   64  377   17.076   14.316   9.260

in summary,

 kernel                                OS/distro       xen     timekeeping
 ------------------------------------- --------------- ------- ------------
 opensuse kernel-default-2.6.31.8-0.1  opensuse 11.2   (n/a)   OK
 opensuse kernel-default-2.6.32-41     opensuse 11.2   (n/a)   OK
 opensuse kernel-xen-2.6.31.8-0.1      opensuse 11.2   3.4.x   NOISY/FAIL
 opensuse kernel-xen-2.6.32-41         opensuse 11.2   3.4.x   NOISY/FAIL
 opensuse kernel-xen-2.6.31.8-0.1      opensuse 11.2   4.0.x   NOISY/FAIL
 opensuse kernel-xen-2.6.32-41         opensuse 11.2   4.0.x   NOISY/FAIL
 xen.git  kernel-2.6.31.6 (pvops Dom0) opensuse 11.2   4.0.x   NOISY/FAIL
 xen.git  kernel-2.6.31.6              opensuse 11.2   (n/a)   OK

i.e.,

3 different kernel versions, from two different vendors, have
low-noise/accurate timekeeping in NON-xen configs.

boot any of those kernels with either v34x or v40x hypervisor in
place, and timekeeping suffers from unusably excessive jitter &
offset.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-17 20:03 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Fri, Jan 15, 2010 at 09:33:20AM -0800, mail ignored
wrote:> On Fri, Jan 15, 2010 at 9:28 AM, mail ignored
<0.bugs.only.0@gmail.com> wrote:
> > is the centos/redhat 2.6.18 kernel ext4-friendly?
> 
> this
> 
> http://etbe.coker.com.au/2010/01/05/ext4-and-rhel5-centos5/
> 
> suggests that ext4 should not be a problem with this kernel ...

RHEL/CentOS 5 -164 kernel does support ext4.
Please pastebin the full console log during system start up so we can
see what happens.

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-17 20:22 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Keir and others,

Please check the table in the end of the mail below.. is there
hypervisor (boot time) option to make it behave better for ntpd? 

-- Pasi

On Sat, Jan 16, 2010 at 11:17:36AM -0800, mail ignored
wrote:> On Thu, Jan 14, 2010 at 8:20 PM, Fajar A. Nugraha <fajar@fajar.net>
wrote:
> >> @ novell say it''s xen & ntp
> >> @ ntp say it''s novell & xen
> >> @ xen say it''s novell & ntp
> >>
> 
> > That''s why I suggested you use a workaround: use 2.6.18
kernel, either
> > from RHEL or from xen.org :D
> > If this works -> novell problem
> > If this doesn''t work -> xen problem
> 
> On Sat, Jan 16, 2010 at 5:05 AM, Fajar A. Nugraha <fajar@fajar.net>
wrote:
> > You might want to try latest pv_ops as well
> > (http://wiki.xensource.com/xenwiki/XenParavirtOps), which should be
> > easier to work with Opensuse since the version number is close.
> 
> it seems it''s _not_ the kernel -- unless there''s a
coincident kernel
> problem across source/vendor ...
> 
> per
>  http://lists.xensource.com/archives/html/xen-devel/2009-12/msg01127.html
>  http://wiki.xensource.com/xenwiki/XenParavirtOps
>  http://wiki.xensource.com/xenwiki/Kernel.org_Linux_on_Xen
> 
> cd /home/workspace && rm -rf linux-2.6-xen
> git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
> linux-2.6-xen
> cd linux-2.6-xen
> make clean
> cp -a .config .config-old
> wget -O .config
>
http://pasik.reaktio.net/xen/pv_ops-dom0-debug/config-2.6.31.4-pvops_dom0-x86_64
> make oldconfig
>   ...
>   Xen PCI Frontend (XEN_PCIDEV_FRONTEND) [N/m/y/?] (NEW)           <-- m
>   ...
>   "IMQ" target support (NETFILTER_XT_TARGET_IMQ) [N/m/?] (NEW)   
<-- m
>   ...
>   IMQ (intermediate queueing device) support (IMQ) [M/y/?] (NEW)   <--
[enter]
>   ...
>   Block-device tap backend driver (XEN_BLKDEV_TAP) [N/m/y/?] (NEW) <-- m
>   ...
> 
> make bzImage
> make modules
> make modules_install
> strip vmlinux -o vmlinux-stripped
> gzip vmlinux-stripped -c > vmlinuz-gzipped
> 
> ls -al .config \
>         System.map \
>         arch/x86/boot/bzImage \
>         vmlinux \
>         vmlinux-stripped \
>         vmlinuz-gzipped
> 
> cp -a .config               /boot/config-2.6.31.6
> cp -a System.map            /boot/System.map-2.6.31.6
> cp -a arch/x86/boot/bzImage /boot/vmlinuz-2.6.31.6
> mkinitrd
> ls -al /boot/*2.6.31.6*
>  -rw-r--r-- 1 root root    98510 2010-01-16 03:02 /boot/config-2.6.31.6
>  -rw-r--r-- 1 root root 21304611 2010-01-16 04:14 /boot/initrd-2.6.31.6
>  -rw-r--r-- 1 root root  1940879 2010-01-16 03:14 /boot/System.map-2.6.31.6
>  -rw-r--r-- 1 root root  3751936 2010-01-16 03:14 /boot/vmlinuz-2.6.31.6
> 
> 
> vi /boot/grub/menu.lst
> ...
> title xen.git
>  root   (hd0,0)
>  kernel /xen.gz  loglvl=all loglvl_guest=all dom0_mem=1024M
> vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1
>  module /vmlinuz-2.6.31.6 root=/dev/VG0/ROOT rootfstype=ext4
> rootflags=journal_checksum resume=/dev/VG0/SWAP nomodeset selinux=0
> SELINUX_INIT=NO vga=0x31a console=tty0 console=xvc0,57600
>  module /initrd-2.6.31.6
> 
> title xen.git DISABLED
>  root (hd0,0)
>  kernel /vmlinuz-2.6.31.6 root=/dev/VG_SVR/LV_SVR_DOM0_ROOT
> rootfstype=ext4 rootflags=journal_checksum
> resume=/dev/VG_SVR/LV_SVR_DOM0_SWAP nomodeset showopts splash=silent
> selinux=0 SELINUX_INIT=NO elevator=cfq vga=0x31a console=tty0
> console=ttyS0,57600n8
>  module /initrd-2.6.31.6
> ...
> 
> 
> boot to "xen.git"
> 
>  uname -a
>   Linux test 2.6.31.6 #1 SMP Sat Jan 16 03:14:19 PST 2010 x86_64
> x86_64 x86_64 GNU/Linux
>  lsb_release -a
>   LSB Version:
>
core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:graphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:graphics-4.0-noarch
>   Distributor ID: SUSE LINUX
>   Description:    openSUSE 11.2 (x86_64)
>   Release:        11.2
>   Codename:       n/a
> 
> service ntp stop
> rm /etc/adjtime /var/
> rm /var/lib/ntp/drift/ntp.drift
> 
> cat << EOF > /etc/sysconfig/clock
> HWCLOCK="--utc"
> SYSTOHC="yes"
> TIMEZONE="America/Los_Angeles"
> DEFAULT_TIMEZONE="US/Pacific"
> EOF
> hwclock --systohc --utc
> service ntp start
> 
> sntp clock.isc.org && date && hwclock --utc --show
>  2010 Jan 16 04:37:09.000729 + 1.435060 +/- 0.028944 secs
>  Sat Jan 16 04:37:11 PST 2010
>  Sat 16 Jan 2010 04:37:11 AM PST  -0.345191 seconds
> 
> ... (wait ~ 6 hours) ...
> 
> date
> 	Sat Jan 16 11:01:42 PST 2010
> ntpq -p -c rv
> 	assID=0 status=06c4 leap_none, sync_ntp, 12 events, event_peer/strat_chg,
> 	version="ntpd 4.2.4p8@1.1612 Mon Dec 28 17:26:30 UTC 2009 (1)",
> 	processor="x86_64", system="Linux/2.6.31.6", leap=00,
stratum=2,
> 	precision=-20, rootdelay=14.718, rootdispersion=135.995, peer=11390,
> 	refid=66.220.9.122,
> 	reftime=cefc89b2.a36f1342  Sat, Jan 16 2010 10:53:38.638, poll=6,
> 	clock=cefc8b82.7e523feb  Sat, Jan 16 2010 11:01:22.493, state=2,
> 	offset=-115.130, frequency=-500.000, jitter=464.242, noise=29.343,
> 	stability=154.680
> 	     remote           refid      st t when poll reach   delay   offset 
jitter
> 
=============================================================================>
*clock.fmt.he.ne .PPS.            1 u   29   64  377   21.438  -811.68 462.837
> 	+otc2.psu.edu    129.6.15.29      2 u    7   64  377  109.399  -850.99
470.522
> 	+clock.isc.org   .GPS.            1 u    7   64  377   21.440  -854.82
468.274
> 	+clock.sjc.he.ne .CDMA.           1 u   49   64  377   16.889  -780.55
469.481
> 	+zorro.sf-bay.or 216.218.254.202  2 u   59   64  377   20.366  -762.14
458.947
> 	+nist1.aol-ca.tr .ACTS.           1 u   38   64  377   16.260  -795.87
463.170
> 
> boot to "xen.git DISABLED"
> 
> uname -a
> 	Linux test 2.6.31.6 #1 SMP Sat Jan 16 09:14:19 PST 2010 x86_64 x86_64
> x86_64 GNU/Linux
> 
> service ntp stop
> rm /etc/adjtime /var/
> rm /var/lib/ntp/drift/ntp.drift
> service ntp start
> 
> ... (wait ~ 10 minutes) ...
> date
> 	Sat Jan 16 11:15:05 PST 2010
> ntpq -p -c rv
> 	assID=0 status=0664 leap_none, sync_ntp, 6 events, event_peer/strat_chg,
> 	version="ntpd 4.2.4p8@1.1612 Mon Dec 28 17:26:30 UTC 2009 (1)",
> 	processor="x86_64", system="Linux/2.6.31.6", leap=00,
stratum=2,
> 	precision=-20, rootdelay=22.750, rootdispersion=975.184, peer=43091,
> 	refid=66.220.9.122,
> 	reftime=cefc8c7c.35ca466a  Sat, Jan 16 2010 11:05:32.210, poll=6,
> 	clock=cefc8eb9.25103a1f  Sat, Jan 16 2010 11:15:05.144, state=3,
> 	offset=27.534, frequency=0.000, jitter=6.740, noise=16.331,
> 	stability=0.000
> 	     remote           refid      st t when poll reach   delay   offset 
jitter
> 
=============================================================================>
*clock.fmt.he.ne .PPS.            1 u    4   64  377   18.112   10.541   5.892
> 	+otc2.psu.edu    147.84.59.145    2 u   64   64  377  101.103   12.106  
8.621
> 	+clock.isc.org   .GPS.            1 u   57   64  377   14.990    4.830  
9.622
> 	+clock.sjc.he.ne .CDMA.           1 u   59   64  377   16.503    9.299  
9.440
> 	+zorro.sf-bay.or 216.218.254.202  2 u   56   64  377   22.212    6.323 
11.031
> 	+nist1.aol-ca.tr .ACTS.           1 u   60   64  377   17.076   14.316  
9.260
> 
> in summary,
> 
>  kernel                                OS/distro       xen     timekeeping
>  ------------------------------------- --------------- ------- ------------
>  opensuse kernel-default-2.6.31.8-0.1  opensuse 11.2   (n/a)   OK
>  opensuse kernel-default-2.6.32-41     opensuse 11.2   (n/a)   OK
>  opensuse kernel-xen-2.6.31.8-0.1      opensuse 11.2   3.4.x   NOISY/FAIL
>  opensuse kernel-xen-2.6.32-41         opensuse 11.2   3.4.x   NOISY/FAIL
>  opensuse kernel-xen-2.6.31.8-0.1      opensuse 11.2   4.0.x   NOISY/FAIL
>  opensuse kernel-xen-2.6.32-41         opensuse 11.2   4.0.x   NOISY/FAIL
>  xen.git  kernel-2.6.31.6 (pvops Dom0) opensuse 11.2   4.0.x   NOISY/FAIL
>  xen.git  kernel-2.6.31.6              opensuse 11.2   (n/a)   OK
> 
> i.e.,
> 
> 3 different kernel versions, from two different vendors, have
> low-noise/accurate timekeeping in NON-xen configs.
> 
> boot any of those kernels with either v34x or v40x hypervisor in
> place, and timekeeping suffers from unusably excessive jitter &
> offset.
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-17 20:23 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Sun, Jan 17, 2010 at 12:03 PM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> RHEL/CentOS 5 -164 kernel does support ext4.
Thanks for verifying.
> Please pastebin the full console log during system start up so we can see
what happens.
Is continuing to work with/on getting RHEL/CentOS 5 kernel-xen to boot
of any particular use/value to _this_ issue, given the successful
build/boot on xen.git''s pvops kernel (results posted above)?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-17 20:25 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Sun, Jan 17, 2010 at 12:22 PM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> Keir and others,
>
> Please check the table in the end of the mail below.. is there
> hypervisor (boot time) option to make it behave better for ntpd?
Thanks for forwarding this ...

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Keir Fraser

2010-Jan-17 21:33 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

What platform timer does Xen say is initialised during boot? Possibly
Xen''s
''clocksource='' boot option could be used to select a different
platform
timer exhibiting less jitter. Tbh I don''t think we''ve ever
done a detailed
analysis of ntp performance on Xen before -- this is something we might be
able to improve.

 -- Keir

On 17/01/2010 20:22, "Pasi Kärkkäinen" <pasik@iki.fi> wrote:
> 
> Keir and others,
> 
> Please check the table in the end of the mail below.. is there
> hypervisor (boot time) option to make it behave better for ntpd?
> 
> -- Pasi
> 
> On Sat, Jan 16, 2010 at 11:17:36AM -0800, mail ignored wrote:
>> On Thu, Jan 14, 2010 at 8:20 PM, Fajar A. Nugraha
<fajar@fajar.net> wrote:
>>>> @ novell say it''s xen & ntp
>>>> @ ntp say it''s novell & xen
>>>> @ xen say it''s novell & ntp
>>>> 
>> 
>>> That''s why I suggested you use a workaround: use 2.6.18
kernel, either
>>> from RHEL or from xen.org :D
>>> If this works -> novell problem
>>> If this doesn''t work -> xen problem
>> 
>> On Sat, Jan 16, 2010 at 5:05 AM, Fajar A. Nugraha
<fajar@fajar.net> wrote:
>>> You might want to try latest pv_ops as well
>>> (http://wiki.xensource.com/xenwiki/XenParavirtOps), which should be
>>> easier to work with Opensuse since the version number is close.
>> 
>> it seems it''s _not_ the kernel -- unless there''s a
coincident kernel
>> problem across source/vendor ...
>> 
>> per
>> 
http://lists.xensource.com/archives/html/xen-devel/2009-12/msg01127.html
>>  http://wiki.xensource.com/xenwiki/XenParavirtOps
>>  http://wiki.xensource.com/xenwiki/Kernel.org_Linux_on_Xen
>> 
>> cd /home/workspace && rm -rf linux-2.6-xen
>> git clone git://git.kernel.org/pub/scm/linux/kernel/git/jeremy/xen.git
>> linux-2.6-xen
>> cd linux-2.6-xen
>> make clean
>> cp -a .config .config-old
>> wget -O .config
>>
http://pasik.reaktio.net/xen/pv_ops-dom0-debug/config-2.6.31.4-pvops_dom0-x86
>> _64
>> make oldconfig
>>   ...
>>   Xen PCI Frontend (XEN_PCIDEV_FRONTEND) [N/m/y/?] (NEW)          
<-- m
>>   ...
>>   "IMQ" target support (NETFILTER_XT_TARGET_IMQ) [N/m/?]
(NEW)     <-- m
>>   ...
>>   IMQ (intermediate queueing device) support (IMQ) [M/y/?] (NEW)  
<--
>> [enter]
>>   ...
>>   Block-device tap backend driver (XEN_BLKDEV_TAP) [N/m/y/?] (NEW)
<-- m
>>   ...
>> 
>> make bzImage
>> make modules
>> make modules_install
>> strip vmlinux -o vmlinux-stripped
>> gzip vmlinux-stripped -c > vmlinuz-gzipped
>> 
>> ls -al .config \
>>         System.map \
>>         arch/x86/boot/bzImage \
>>         vmlinux \
>>         vmlinux-stripped \
>>         vmlinuz-gzipped
>> 
>> cp -a .config               /boot/config-2.6.31.6
>> cp -a System.map            /boot/System.map-2.6.31.6
>> cp -a arch/x86/boot/bzImage /boot/vmlinuz-2.6.31.6
>> mkinitrd
>> ls -al /boot/*2.6.31.6*
>>  -rw-r--r-- 1 root root    98510 2010-01-16 03:02 /boot/config-2.6.31.6
>>  -rw-r--r-- 1 root root 21304611 2010-01-16 04:14 /boot/initrd-2.6.31.6
>>  -rw-r--r-- 1 root root  1940879 2010-01-16 03:14
/boot/System.map-2.6.31.6
>>  -rw-r--r-- 1 root root  3751936 2010-01-16 03:14
/boot/vmlinuz-2.6.31.6
>> 
>> 
>> vi /boot/grub/menu.lst
>> ...
>> title xen.git
>>  root   (hd0,0)
>>  kernel /xen.gz  loglvl=all loglvl_guest=all dom0_mem=1024M
>> vga=gfx-1280x1024x32 console=vga,com1 com1=57600,8n1
>>  module /vmlinuz-2.6.31.6 root=/dev/VG0/ROOT rootfstype=ext4
>> rootflags=journal_checksum resume=/dev/VG0/SWAP nomodeset selinux=0
>> SELINUX_INIT=NO vga=0x31a console=tty0 console=xvc0,57600
>>  module /initrd-2.6.31.6
>> 
>> title xen.git DISABLED
>>  root (hd0,0)
>>  kernel /vmlinuz-2.6.31.6 root=/dev/VG_SVR/LV_SVR_DOM0_ROOT
>> rootfstype=ext4 rootflags=journal_checksum
>> resume=/dev/VG_SVR/LV_SVR_DOM0_SWAP nomodeset showopts splash=silent
>> selinux=0 SELINUX_INIT=NO elevator=cfq vga=0x31a console=tty0
>> console=ttyS0,57600n8
>>  module /initrd-2.6.31.6
>> ...
>> 
>> 
>> boot to "xen.git"
>> 
>>  uname -a
>>   Linux test 2.6.31.6 #1 SMP Sat Jan 16 03:14:19 PST 2010 x86_64
>> x86_64 x86_64 GNU/Linux
>>  lsb_release -a
>>   LSB Version:
>>
core-2.0-noarch:core-3.2-noarch:core-4.0-noarch:core-2.0-x86_64:core-3.2-x86_
>>
64:core-4.0-x86_64:desktop-4.0-amd64:desktop-4.0-noarch:graphics-2.0-amd64:gr
>>
aphics-2.0-noarch:graphics-3.2-amd64:graphics-3.2-noarch:graphics-4.0-amd64:g
>> raphics-4.0-noarch
>>   Distributor ID: SUSE LINUX
>>   Description:    openSUSE 11.2 (x86_64)
>>   Release:        11.2
>>   Codename:       n/a
>> 
>> service ntp stop
>> rm /etc/adjtime /var/
>> rm /var/lib/ntp/drift/ntp.drift
>> 
>> cat << EOF > /etc/sysconfig/clock
>> HWCLOCK="--utc"
>> SYSTOHC="yes"
>> TIMEZONE="America/Los_Angeles"
>> DEFAULT_TIMEZONE="US/Pacific"
>> EOF
>> hwclock --systohc --utc
>> service ntp start
>> 
>> sntp clock.isc.org && date && hwclock --utc --show
>>  2010 Jan 16 04:37:09.000729 + 1.435060 +/- 0.028944 secs
>>  Sat Jan 16 04:37:11 PST 2010
>>  Sat 16 Jan 2010 04:37:11 AM PST  -0.345191 seconds
>> 
>> ... (wait ~ 6 hours) ...
>> 
>> date
>> Sat Jan 16 11:01:42 PST 2010
>> ntpq -p -c rv
>> assID=0 status=06c4 leap_none, sync_ntp, 12 events,
event_peer/strat_chg,
>> version="ntpd 4.2.4p8@1.1612 Mon Dec 28 17:26:30 UTC 2009
(1)",
>> processor="x86_64", system="Linux/2.6.31.6",
leap=00, stratum=2,
>> precision=-20, rootdelay=14.718, rootdispersion=135.995, peer=11390,
>> refid=66.220.9.122,
>> reftime=cefc89b2.a36f1342  Sat, Jan 16 2010 10:53:38.638, poll=6,
>> clock=cefc8b82.7e523feb  Sat, Jan 16 2010 11:01:22.493, state=2,
>> offset=-115.130, frequency=-500.000, jitter=464.242, noise=29.343,
>> stability=154.680
>>     remote           refid      st t when poll reach   delay   offset 
jitter
>> 
=============================================================================>>>> *clock.fmt.he.ne .PPS.            1 u   29   64  377   21.438  -811.68
>> 462.837
>> +otc2.psu.edu    129.6.15.29      2 u    7   64  377  109.399  -850.99
>> 470.522
>> +clock.isc.org   .GPS.            1 u    7   64  377   21.440  -854.82
>> 468.274
>> +clock.sjc.he.ne .CDMA.           1 u   49   64  377   16.889  -780.55
>> 469.481
>> +zorro.sf-bay.or 216.218.254.202  2 u   59   64  377   20.366  -762.14
>> 458.947
>> +nist1.aol-ca.tr .ACTS.           1 u   38   64  377   16.260  -795.87
>> 463.170
>> 
>> boot to "xen.git DISABLED"
>> 
>> uname -a
>> Linux test 2.6.31.6 #1 SMP Sat Jan 16 09:14:19 PST 2010 x86_64 x86_64
>> x86_64 GNU/Linux
>> 
>> service ntp stop
>> rm /etc/adjtime /var/
>> rm /var/lib/ntp/drift/ntp.drift
>> service ntp start
>> 
>> ... (wait ~ 10 minutes) ...
>> date
>> Sat Jan 16 11:15:05 PST 2010
>> ntpq -p -c rv
>> assID=0 status=0664 leap_none, sync_ntp, 6 events,
event_peer/strat_chg,
>> version="ntpd 4.2.4p8@1.1612 Mon Dec 28 17:26:30 UTC 2009
(1)",
>> processor="x86_64", system="Linux/2.6.31.6",
leap=00, stratum=2,
>> precision=-20, rootdelay=22.750, rootdispersion=975.184, peer=43091,
>> refid=66.220.9.122,
>> reftime=cefc8c7c.35ca466a  Sat, Jan 16 2010 11:05:32.210, poll=6,
>> clock=cefc8eb9.25103a1f  Sat, Jan 16 2010 11:15:05.144, state=3,
>> offset=27.534, frequency=0.000, jitter=6.740, noise=16.331,
>> stability=0.000
>>     remote           refid      st t when poll reach   delay   offset 
jitter
>> 
=============================================================================>>>> *clock.fmt.he.ne .PPS.            1 u    4   64  377   18.112   10.541
>> 5.892
>> +otc2.psu.edu    147.84.59.145    2 u   64   64  377  101.103   12.106
>> 8.621
>> +clock.isc.org   .GPS.            1 u   57   64  377   14.990    4.830
>> 9.622
>> +clock.sjc.he.ne .CDMA.           1 u   59   64  377   16.503    9.299
>> 9.440
>> +zorro.sf-bay.or 216.218.254.202  2 u   56   64  377   22.212    6.323
>> 11.031
>> +nist1.aol-ca.tr .ACTS.           1 u   60   64  377   17.076   14.316
>> 9.260
>> 
>> in summary,
>> 
>>  kernel                                OS/distro       xen    
timekeeping
>>  ------------------------------------- --------------- -------
------------
>>  opensuse kernel-default-2.6.31.8-0.1  opensuse 11.2   (n/a)   OK
>>  opensuse kernel-default-2.6.32-41     opensuse 11.2   (n/a)   OK
>>  opensuse kernel-xen-2.6.31.8-0.1      opensuse 11.2   3.4.x  
NOISY/FAIL
>>  opensuse kernel-xen-2.6.32-41         opensuse 11.2   3.4.x  
NOISY/FAIL
>>  opensuse kernel-xen-2.6.31.8-0.1      opensuse 11.2   4.0.x  
NOISY/FAIL
>>  opensuse kernel-xen-2.6.32-41         opensuse 11.2   4.0.x  
NOISY/FAIL
>>  xen.git  kernel-2.6.31.6 (pvops Dom0) opensuse 11.2   4.0.x  
NOISY/FAIL
>>  xen.git  kernel-2.6.31.6              opensuse 11.2   (n/a)   OK
>> 
>> i.e.,
>> 
>> 3 different kernel versions, from two different vendors, have
>> low-noise/accurate timekeeping in NON-xen configs.
>> 
>> boot any of those kernels with either v34x or v40x hypervisor in
>> place, and timekeeping suffers from unusably excessive jitter &
>> offset.
>> 
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@lists.xensource.com
>> http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-17 22:21 UTC

head link

[Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

hi,

(should I keep replying to both lists now?)

On Sun, Jan 17, 2010 at 1:33 PM, Keir Fraser <keir.fraser@eu.citrix.com>
wrote:> What platform timer does Xen say is initialised during boot?
xm dmesg | egrep -i "timer|clock"
(XEN) ACPI: PM-Timer IO Port: 0x808
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) Platform timer is 14.284MHz HPET
(XEN) HPET: 4 timers in total, 3 timers will be used for broadcast
(XEN) mcheck_poll: Machine check polling timer started.

> Possibly Xen''s
> ''clocksource='' boot option could be used to select a
different platform
> timer exhibiting less jitter.
currently, i have " ... clocksource=xen ... "

on these platforms,

@ non-xen,

uname -a
  Linux test 2.6.31.8-0.1-desktop #1 SMP PREEMPT 2009-12-15 23:55:40
+0100 x86_64 x86_64 x86_64 GNU/Linux
cat /sys/devices/system/clocksource/clocksource0/available_clocksource
  tsc hpet acpi_pm
cat /sys/devices/system/clocksource/clocksource0/current_clocksource
  tsc

i''ve read "tsc" is a "good thing ..." ...

but, @ xen, i''ve only,

uname -a
  Linux test 2.6.31.8-0.1-xen #1 SMP 2009-12-15 23:55:40 +0100 x86_64
x86_64 x86_64 GNU/Linux
cat /sys/devices/system/clocksource/clocksource0/available_clocksource
  xen jiffies
cat /sys/devices/system/clocksource/clocksource0/current_clocksource
  xen

i''ve already tried the apparently only other option,
''jiffies'', as a
xen clocksource,

 echo "jiffies" >
/sys/devices/system/clocksource/clocksource0/current_clocksource
 cat /sys/devices/system/clocksource/clocksource0/current_clocksource
   jiffies
 service ntp restart
(also tried this experiemtn booting to clocksource=jiffies in grub
conf -- no difference)

but i''ve noticed no improvement in jitter/offset over time,

assID=0 status=c654 sync_alarm, sync_ntp, 5 events, event_peer/strat_chg,
version="ntpd 4.2.4p8@1.1612 Mon Dec 28 17:26:30 UTC 2009 (1)",
processor="x86_64", system="Linux/2.6.31.8-0.1-xen",
leap=11,
stratum=2, precision=-8, rootdelay=0.000, rootdispersion=10.755,
peer=34524, refid=Qq,
reftime=00000000.00000000  Wed, Feb  6 2036 22:28:16.000, poll=6,
clock=cefe0ad1.036efb16  Sun, Jan 17 2010 14:18:49.013, state=2,
offset=0.000, frequency=-362.896, jitter=568.398, noise=3.906,
stability=0.000
     remote           refid      st t when poll reach   delay   offset  jitter
=============================================================================+clock.fmt.he.ne
.PPS.            1 u   26   64  377   18.351  -1363.2 572.904
+otc2.psu.edu    128.118.2.33     2 u   58   64  377   99.412  -1298.6 575.668
+clock.isc.org   .GPS.            1 u   61   64  377   19.111  -1296.8 562.546
+clock.sjc.he.ne .CDMA.           1 u   35   64  377   24.181  -1341.8 561.600
+zorro.sf-bay.or 216.218.254.202  2 u   63   64  377   20.503  -1290.7 565.956
*nist1.aol-ca.tr .ACTS.           1 u   35   64  377   24.571  -1339.8 567.403

and, in both these results, & at initially startup,

assID=0 status=c035 sync_alarm, sync_unspec, 3 events, event_clock_reset,
version="ntpd 4.2.4p8@1.1612 Mon Dec 28 17:26:30 UTC 2009 (1)",
processor="x86_64", system="Linux/2.6.31.8-0.1-xen",
leap=11,
stratum=16, precision=-8, rootdelay=0.000, rootdispersion=0.930, peer=0,
refid=STEP, reftime=00000000.00000000  Wed, Feb  6 2036 22:28:16.000,
poll=4, clock=cefe0842.32f14db2  Sun, Jan 17 2010 13:26:54.198, state=4,
offset=0.000, frequency=-362.896, jitter=5.498, noise=3.906,
stability=0.000
     remote           refid      st t when poll reach   delay   offset  jitter
=============================================================================
clock.fmt.he.ne .PPS.            1 u   17   64    1   20.834  -88.894   3.906
 otc2.psu.edu    130.207.244.240  2 u   54   64    1  123.655  -12.796   3.906
 clock.isc.org   .GPS.            1 u   49   64    1   19.517  -34.400   3.906
 clock.sjc.he.ne .CDMA.           1 u   25   64    1   16.966  -76.911   3.906
 zorro.sf-bay.or 216.218.254.202  2 u   48   64    1   35.391  -29.823   3.906
 nist1.aol-ca.tr .ACTS.           1 u   24   64    1   24.211  -74.151   3.906

 -- i see "3.906 weirdness".  notice the 3.906''s in the
jitter
(changes over time) & noise/offset (fixed over time) fields?

looking around on the net, i''ve found no particular explanation for
this; tbh, it doesn''t exactly breed confidence.
> Tbh I don''t think we''ve ever done a detailed analysis of
ntp performance on Xen before
> -- this is something we might be able to improve.
any help would be appreciated.

thanks.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

mail ignored

2010-Jan-17 23:53 UTC

head link

[Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Sun, Jan 17, 2010 at 1:33 PM, Keir Fraser <keir.fraser@eu.citrix.com>
wrote:> Tbh I don''t think we''ve ever done a detailed
> analysis of ntp performance on Xen before -- this is something we might be
> able to improve.
just fyi -- this may be all old info for you -- this

 "Kernel Model for Precision Timekeeping"
  http://www.eecis.udel.edu/~mills/ntp/html/kern.html

and the site/info around it, seem to be the authoritative resource on
timekeeping with ntp (function, debugging, etc).

it''s been a great resource in understanding the expected behavior of
ntp.

i also found good general guidance in

 "Network Time Protocol (NTP) Overview and Configuration: An Internet2
Cookbook"
  http://e2epi.internet2.edu/npw/binder-docs/ntp-cookbook.pdf

although it does include a # of quotes/references to Mills'' bias
against Linux, e.g.,

  “I have confronted Linux many, many times with broken this and that
and concluded I will not attempt to examine, fix, or judge anything
Linux. You are on your own.” —David L. Mills, 2004-10-20

so it''s not clear, at least to me, if there are in fact
''deeper''
issues with ntp at hand.  hopefully that''s gotten better over the
years ...

hth.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Dan Magenheimer

2010-Jan-18 02:23 UTC

head link

RE: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

I''m away from the office right now so can''t easily look this
up
but the following patch was submitted by Tim Deegan in March 2009
and may be relevant.  Likely it is in Xen''s dom0 but not in
the dom0 of the distro you are using.

Apologies if I''ve missed an important point as I''m only
skimming email today and just happened to have Tim''s
March email in a folder.

Re: [Xen-devel] [PATCH] linux-2.6.18-xen.hg: fix interaction of NTP and
dom0->xen time updates
> -----Original Message-----
> From: mail ignored [mailto:0.bugs.only.0@gmail.com]
> Sent: Sunday, January 17, 2010 4:53 PM
> To: Keir Fraser
> Cc: Fajar A. Nugraha; xen-devel@lists.xensource.com; Xen User-List
> Subject: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits
> extremely high jitter/noise? runs stable/quiet under non-xen kernel.
> 
> 
> On Sun, Jan 17, 2010 at 1:33 PM, Keir Fraser 
> <keir.fraser@eu.citrix.com> wrote:
> > Tbh I don''t think we''ve ever done a detailed
> > analysis of ntp performance on Xen before -- this is 
> something we might be
> > able to improve.
> 
> just fyi -- this may be all old info for you -- this
> 
>  "Kernel Model for Precision Timekeeping"
>   http://www.eecis.udel.edu/~mills/ntp/html/kern.html
> 
> and the site/info around it, seem to be the authoritative resource on
> timekeeping with ntp (function, debugging, etc).
> 
> it''s been a great resource in understanding the expected 
> behavior of ntp.
> 
> i also found good general guidance in
> 
>  "Network Time Protocol (NTP) Overview and Configuration: An 
> Internet2 Cookbook"
>   http://e2epi.internet2.edu/npw/binder-docs/ntp-cookbook.pdf
> 
> although it does include a # of quotes/references to Mills'' bias
> against Linux, e.g.,
> 
>   “I have confronted Linux many, many times with broken this and that
> and concluded I will not attempt to examine, fix, or judge anything
> Linux. You are on your own.” —David L. Mills, 2004-10-20
> 
> so it''s not clear, at least to me, if there are in fact
''deeper''
> issues with ntp at hand.  hopefully that''s gotten better over the
> years ...
> 
> hth.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-18 03:00 UTC

head link

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Sun, Jan 17, 2010 at 6:23 PM, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:> I''m away from the office right now so can''t easily look
this up
> but the following patch was submitted by Tim Deegan in March 2009
> and may be relevant.  Likely it is in Xen''s dom0 but not in
> the dom0 of the distro you are using.
>
> Apologies if I''ve missed an important point as I''m only
> skimming email today and just happened to have Tim''s
> March email in a folder.
reading @ the patch thread, http://tinyurl.com/yjnj6wa

for reference ... checking in opensuse''s src trees,

cd /usr/src
find . | grep x86 | grep time | grep xen | grep "\.c"$
./arch/x86/kernel/time_32-xen.c
./arch/x86/xen/time.c

time_32-xen.c is posted -> http://pastebin.com/f72c6de07
time.c  -> http://pastebin.com/f3b4a7fdd

checking in xen.git''s pvops tree,
find . | grep x86 | grep time | grep xen | grep "\.c"$
  ./arch/x86/xen/time.c

-> http://pastebin.com/f3409ea7a.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Fajar A. Nugraha

2010-Jan-18 03:47 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Mon, Jan 18, 2010 at 3:23 AM, mail ignored <0.bugs.only.0@gmail.com>
wrote:> Is continuing to work with/on getting RHEL/CentOS 5 kernel-xen to boot
> of any particular use/value to _this_ issue, given the successful
> build/boot on xen.git''s pvops kernel (results posted above)?
It''s basically like this: The "stable", officially supported
kernel
version for xen as of now is still 2.6.18. When you encounter possible
kernel-related problems, it''s usually best to check if the problem is
present when using that kernel version. If it works, then you have one
quick workaorund: simply use 2.6.18 kernel. I recommended RHEL/Centos5
kernel since it''s well maintained in terms of security fixes.

pv_ops and suse''s xen kernel is based on newer kernel version. pv_ops
will be the next official xen kernel, but currently it still have some
bugs on it. For me it was the combination of suspend/resume and cpu
hotplug. There were also problems with iperf (search list archive).
Some of the bugs may take a long time to fix.

So now the question becomes how quickly you need to solve this issue.
If you can live with it, then I suggest focus all efforts on fixing
pv_ops kernel (since you said that novell guys are unable to find the
problem). If you need it to work quickly, then you should try testing
2.6.18 kernel, to the point of adding an extra disk or reinstalling
your OS.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Pasi Kärkkäinen

2010-Jan-18 06:42 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Mon, Jan 18, 2010 at 10:47:47AM +0700, Fajar A. Nugraha
wrote:> On Mon, Jan 18, 2010 at 3:23 AM, mail ignored
<0.bugs.only.0@gmail.com> wrote:
> > Is continuing to work with/on getting RHEL/CentOS 5 kernel-xen to boot
> > of any particular use/value to _this_ issue, given the successful
> > build/boot on xen.git''s pvops kernel (results posted above)?
> 
> It''s basically like this: The "stable", officially
supported kernel
> version for xen as of now is still 2.6.18. When you encounter possible
> kernel-related problems, it''s usually best to check if the problem
is
> present when using that kernel version. If it works, then you have one
> quick workaorund: simply use 2.6.18 kernel. I recommended RHEL/Centos5
> kernel since it''s well maintained in terms of security fixes.
> 
While http://xenbits.xen.org/linux-2.6.18-xen.hg (the official dom0
kernel for up to Xen 3.4.x) might have more xen-related fixes compared
to RHEL/CentOS5 kernel-xen :)

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Fajar A. Nugraha

2010-Jan-18 07:22 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Mon, Jan 18, 2010 at 1:42 PM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> On Mon, Jan 18, 2010 at 10:47:47AM +0700, Fajar A. Nugraha wrote:
>> On Mon, Jan 18, 2010 at 3:23 AM, mail ignored
<0.bugs.only.0@gmail.com> wrote:
>> > Is continuing to work with/on getting RHEL/CentOS 5 kernel-xen to
boot
>> > of any particular use/value to _this_ issue, given the successful
>> > build/boot on xen.git''s pvops kernel (results posted
above)?
>>
>> It''s basically like this: The "stable", officially
supported kernel
>> version for xen as of now is still 2.6.18. When you encounter possible
>> kernel-related problems, it''s usually best to check if the
problem is
>> present when using that kernel version. If it works, then you have one
>> quick workaorund: simply use 2.6.18 kernel. I recommended RHEL/Centos5
>> kernel since it''s well maintained in terms of security fixes.
>>
>
> While http://xenbits.xen.org/linux-2.6.18-xen.hg (the official dom0
> kernel for up to Xen 3.4.x) might have more xen-related fixes compared
> to RHEL/CentOS5 kernel-xen :)
correct :D

However, after spending over three hours, opensuse 11.2 and kernel
2.6.18 is simply a no-no combo :P
opensuse''s udev requires newer kernel, and using kernel 2.6.18 gives
"udevd: error getting signalfd". Since udevd refuses to start, md wont
start, lvm won''t start, and so on, and so on :(

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Keir Fraser

2010-Jan-18 07:27 UTC

head link

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On 17/01/2010 22:21, "mail ignored" <0.bugs.only.0@gmail.com>
wrote:
>> Possibly Xen''s
>> ''clocksource='' boot option could be used to select a
different platform
>> timer exhibiting less jitter.
> 
> currently, i have " ... clocksource=xen ... "
I meant clocksource= on Xen''s command line, not dom0''s. I
don''t think it
even has an effect when specified to a kernel running on Xen.

For example, you could try clocksource=acpi, or clocksource=pit. And then
check that the ''Platform timer is...'' line from Xen boot
output changes
correspondingly.

 -- Keir



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-18 17:21 UTC

head link

[Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

hi

On Sun, Jan 17, 2010 at 11:27 PM, Keir Fraser <keir.fraser@eu.citrix.com>
wrote:> I meant clocksource= on Xen''s command line, not dom0''s. I
don''t think it
> even has an effect when specified to a kernel running on Xen.
where is that documented?  tbh, i''ve never noticed those options at
all.  checking here,

 
http://wiki.xensource.com/xenwiki/?action=fullsearch&context=180&value=clocksource&fullsearch=Text

those references only mention clocksource= @kernel line, and the xen
user''s manual doesn''t mention the term clocksource at all :-/

there''s no mention> For example, you could try clocksource=acpi, or clocksource=pit. And then
> check that the ''Platform timer is...'' line from Xen boot
output changes
> correspondingly.
checking those now ... will report results in a bot.

thanks.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

mail ignored

2010-Jan-18 18:19 UTC

head link

[Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Mon, Jan 18, 2010 at 9:21 AM, mail ignored <0.bugs.only.0@gmail.com>
wrote:>> For example, you could try clocksource=acpi, or clocksource=pit. And
then
>> check that the ''Platform timer is...'' line from Xen
boot output changes
>> correspondingly.
>
> checking those now ... will report results in a bit.
both seem to work ...

booting to,

  /xen.gz ... clocksource=acpi ...

then,

 xm dmesg | grep -i timer
  (XEN) ACPI: PM-Timer IO Port: 0x808
  (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
  (XEN) Platform timer is 3.579MHz ACPI PM Timer
  (XEN) HPET: 4 timers in total, 3 timers will be used for broadcast
  (XEN) mcheck_poll: Machine check polling timer started.

~ 15 minutes, quiet/stable operation @ stratum=2, full reach.

	assID=0 status=0684 leap_none, sync_ntp, 8 events, event_peer/strat_chg,
	version="ntpd 4.2.4p8@1.1612 Mon Dec 28 17:26:30 UTC 2009 (1)",
	processor="x86_64", system="Linux/2.6.31.8-0.1-xen",
leap=00, stratum=2,
	precision=-20, rootdelay=13.970, rootdispersion=26.079, peer=33500,
	refid=216.218.254.202,
	reftime=ceff1d2c.ddbe13c0  Mon, Jan 18 2010  9:47:24.866, poll=6,
	clock=ceff1e07.464f2053  Mon, Jan 18 2010  9:51:03.274, state=4,
	offset=15.245, frequency=24.962, jitter=10.571, noise=17.706,
	stability=7.306
	     remote           refid      st t when poll reach   delay   offset  jitter
	=============================================================================
-clock.fmt.he.ne .PPS.            1 u   41   64  377   16.047   20.346  10.336
	-otc2.psu.edu    130.207.244.240  2 u   39   64  377  100.626   14.538   5.387
	+clock.isc.org   .GPS.            1 u   54   64  377   15.174   -0.843  10.578
	*clock.sjc.he.ne .CDMA.           1 u   26   64  377   13.970   12.255   7.051
	 zorro.sf-bay.or 216.218.254.202  2 u   31   64  377   16.489   13.730   5.781
	+nist1.aol-ca.tr .ACTS.           1 u   15   64  377   14.891    5.947  14.596

booting to,

  /xen.gz ... clocksource=pit ...

 xm dmesg | grep -i timer
  (XEN) ACPI: PM-Timer IO Port: 0x808
  (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
  (XEN) Platform timer is 1.193MHz PIT
  (XEN) mcheck_poll: Machine check polling timer started.

~ 20 minutes, quiet/stable operation @ stratum=2, full reach.

	assID=0 status=0654 leap_none, sync_ntp, 5 events, event_peer/strat_chg,
	version="ntpd 4.2.4p8@1.1612 Mon Dec 28 17:26:30 UTC 2009 (1)",
	processor="x86_64", system="Linux/2.6.31.8-0.1-xen",
leap=00, stratum=2,
	precision=-20, rootdelay=16.981, rootdispersion=31.958, peer=63536,
	refid=207.200.81.113,
	reftime=ceff2377.503b4ceb  Mon, Jan 18 2010 10:14:15.313, poll=6,
	clock=ceff240e.fa89c747  Mon, Jan 18 2010 10:16:46.978, state=4,
	offset=9.641, frequency=4.070, jitter=12.958, noise=6.244,
	stability=0.834
	     remote           refid      st t when poll reach   delay   offset  jitter
	=============================================================================
+clock.fmt.he.ne .PPS.            1 u   30   64  177   15.250    8.422   9.657
	+otc2.psu.edu    147.84.59.145    2 u   33   64  177   99.375   10.359   8.906
	+clock.isc.org   .GPS.            1 u   25   64  177   13.060    2.679   9.501
	+clock.sjc.he.ne .CDMA.           1 u   22   64  177   15.097   10.608   7.866
	+zorro.sf-bay.or 216.218.254.202  2 u   27   64  177   15.082    9.639   8.443
	*nist1.aol-ca.tr .ACTS.           1 u   23   64  177   16.981   15.780   7.743


seemingly, _nothing_ to do with the kernel, then.  rather overriding
whatever xen''s default clocksource is does the trick.

i''ll still take a look over longer term @ Dom0, and see if
there''s any
strangeness in DomU time ...

thanks!

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

mail ignored

2010-Jan-18 18:26 UTC

head link

[Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

wrt acpi vs pit,

reading @ http://the-b.org/Linux_timers, it seems (?) acpi may be, in
general, the better options, as pit " ... doesn''t gain lost time
because it doesn''t have lost tick correction code ..."

iiuc, that references those timers @kernel -- not clear if it is/isn''t
relevant to Xen ...

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Pasi Kärkkäinen

2010-Jan-18 19:29 UTC

head link

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Mon, Jan 18, 2010 at 10:26:10AM -0800, mail ignored
wrote:> wrt acpi vs pit,
> 
> reading @ http://the-b.org/Linux_timers, it seems (?) acpi may be, in
> general, the better options, as pit " ... doesn''t gain lost
time
> because it doesn''t have lost tick correction code ..."
> 
> iiuc, that references those timers @kernel -- not clear if it
is/isn''t
> relevant to Xen ...
> 
What was the default clocksource for you, showing the bad ntpd
behaviour? 

-- Pasi


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

mail ignored

2010-Jan-18 21:50 UTC

head link

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Mon, Jan 18, 2010 at 11:29 AM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:> What was the default clocksource for you, showing the bad ntpd
> behaviour?
from above,

  > On Sun, Jan 17, 2010 at 1:33 PM, Keir Fraser
<keir.fraser@eu.citrix.com> wrote:
  >> What platform timer does Xen say is initialised during boot?
  >
  > xm dmesg | egrep -i "timer|clock"
  > (XEN) ACPI: PM-Timer IO Port: 0x808
  > (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
  > (XEN) Platform timer is 14.284MHz HPET
  > (XEN) HPET: 4 timers in total, 3 timers will be used for broadcast
  > (XEN) mcheck_poll: Machine check polling timer started.

, looks like HPET,

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

mail ignored

2010-Jan-19 02:12 UTC

head link

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

this page,

  http://wiki.xensource.com/xenwiki/xenpm

seems to address hpet, pit & acpi timer sources.  and does so without
every mentioning ''clocksource'' ...
tbh, i can''t get past the broken english on the page;  it would be
really helpful if someone who actually understands that stuff cleaned
up the readability.

the recommendations there are( i think?) "by far" to use HPET/PIT.
which contraindicates the conclusion that I came to above :-/

so, questions are:

(1) *is* there, in fact, a bug here?
(2) which Timer is, in keeping with any/all issues discussed @ the
aforementioned wiki page, recommended?
(3) is the *real* answer simply intended to be "try them all, pick what
works"?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Pasi Kärkkäinen

2010-Jan-19 07:17 UTC

head link

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Mon, Jan 18, 2010 at 06:12:44PM -0800, mail ignored
wrote:> this page,
> 
>   http://wiki.xensource.com/xenwiki/xenpm
> 
> seems to address hpet, pit & acpi timer sources.  and does so without
> every mentioning ''clocksource'' ...
> tbh, i can''t get past the broken english on the page;  it would be
> really helpful if someone who actually understands that stuff cleaned
> up the readability.
> 
I added a couple of "clocksource" words and added one chapter to the
end of the page.

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2010-Jan-19 09:24 UTC

head link

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On 19/01/2010 02:12, "mail ignored" <0.bugs.only.0@gmail.com>
wrote:
> (1) *is* there, in fact, a bug here?
It''s not clear. It could alternatively be a problem with the HPET on
that
specific system. Have you reproduced your issues on a different type of
system (making sure that HPET platform timer is detected and used on that
system too, of course)? That would then obviously point more at it being a
software bug. What is the make/model of the system exhibiting the problem,
by the way?
> (2) which Timer is, in keeping with any/all issues discussed @ the
> aforementioned wiki page, recommended?
HPET or ACPI-PM timers ought to work equally well, assuming bugs
haven''t
crept into Xen''s code paths for either one, and both are implemented
correctly by the hardware Xen is running on. I have to say I think it''s
unlikely to be a software bug, because Xen''s code paths to use HPET and
ACPI-PM timers are really very simple indeed -- it''s hard to see where
a bug
could exist in there. But of course that doesn''t make it absolutely
impossible.
> (3) is the *real* answer simply intended to be "try them all, pick
what
> works"?
Well, really there oughtn''t to be enough of a difference between them
for
anyone to really notice or care which is actually being used. If this does
turn out to be a hardware issue, then yes that is just the kind of case
where you might expect to have to tweak things on the Xen command line.

One option for us is to change the priority ordering of platform timers to
prefer ACPI-PM above HPET, when both are available. That would assume that
ACPI-PM is more reliable than HPET across a wide range of systems though. We
don''t really have much evidence one way or the other.

Another would be to dynamically detect platform timers that aren''t
working
well and disable them at run time. That would require some more thought, and
isn''t going to be something for Xen 4.0 at this point.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

mail ignored

2010-Jan-19 17:58 UTC

head link

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

hi

On Tue, Jan 19, 2010 at 1:24 AM, Keir Fraser <keir.fraser@eu.citrix.com>
wrote:> On 19/01/2010 02:12, "mail ignored"
<0.bugs.only.0@gmail.com> wrote:
>
>> (1) *is* there, in fact, a bug here?
>
> It''s not clear. It could alternatively be a problem with the HPET
on that
> specific system. Have you reproduced your issues on a different type of
> system (making sure that HPET platform timer is detected and used on that
> system too, of course)? That would then obviously point more at it being a
> software bug. What is the make/model of the system exhibiting the problem,
> by the way?
the 5 boxes on which this is currently tested/reproducible are

mobo + cpu:
(1)(2) ASUS M4A78T-E           AM3 AMD 790GX ATX, 2 @ AMD Phenom II X4 945 cpu
(3)    MSI 790FX-GD70          AM3 AMD 790FX ATX, 1 @ AMD Phenom II X4 920 cpu
(4)(5) GIGABYTE GA-MA770T-UD3P AM3 AMD 770   ATX, 1 @ AMD Phenom II X4 945 cpu
                                                  1 @ AMD Phenom II X4 920 cpu


all opensuse 11.2 ...
>> (2) which Timer is, in keeping with any/all issues discussed @ the
>> aforementioned wiki page, recommended?
>
> HPET or ACPI-PM timers ought to work equally well, assuming bugs
haven''t
> crept into Xen''s code paths for either one, and both are
implemented
> correctly by the hardware Xen is running on. I have to say I think
it''s
> unlikely to be a software bug, because Xen''s code paths to use
HPET and
> ACPI-PM timers are really very simple indeed -- it''s hard to see
where a bug
> could exist in there. But of course that doesn''t make it
absolutely
> impossible.
from reading the wiki page, i got the sense that HPET is "by far"
recommended -- if only due to lack of specific mention of acpi(_pm) as
clocksource.

can''t argue with the fact that it works, though.  if "ACPI-PM
timers
ought to work equally well", then I''ve a clear option/solution --
and
knowledge enuf, now, to know where to look for related problems should
they creep up in the future, perhaps on new/different hardware.
>> (3) is the *real* answer simply intended to be "try them all, pick
what
>> works"?
>
> Well, really there oughtn''t to be enough of a difference between
them for
> anyone to really notice or care which is actually being used.
ok.  though, reproducibly across all the aforementioned systems, there is.
> If this does
> turn out to be a hardware issue, then yes that is just the kind of case
> where you might expect to have to tweak things on the Xen command line.
the ''common'' hardware is, of course, the AMD Phenom II X4 cpu
...
> One option for us is to change the priority ordering of platform timers to
> prefer ACPI-PM above HPET, when both are available. That would assume that
> ACPI-PM is more reliable than HPET across a wide range of systems though.
We
> don''t really have much evidence one way or the other.
that really only makes sense, doesn''t it, if it''s known to be
NOT
problematic on other platforms.   given my obviously limited expertise
in the xen source, this all ''smells'' like a problem ...
> Another would be to dynamically detect platform timers that aren''t
working
> well and disable them at run time. That would require some more thought,
and
> isn''t going to be something for Xen 4.0 at this point.
understood.

pls let me know if there''s additional info you need; or, if this
should issue should simply be allowed to ''fade'' now that
there''s a
clear WORKSFORSOME solution.

thanks.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2010-Jan-19 18:04 UTC

head link

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On 19/01/2010 17:58, "mail ignored" <0.bugs.only.0@gmail.com>
wrote:
>> If this does
>> turn out to be a hardware issue, then yes that is just the kind of case
>> where you might expect to have to tweak things on the Xen command line.
> 
> the ''common'' hardware is, of course, the AMD Phenom II X4
cpu ...
The particular hardware that really matters is the chipset, as that is where
the HPET resides. Is that common across all the systems?

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

mail ignored

2010-Jan-19 18:24 UTC

head link

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On Tue, Jan 19, 2010 at 10:04 AM, Keir Fraser <keir.fraser@eu.citrix.com>
wrote:> On 19/01/2010 17:58, "mail ignored"
<0.bugs.only.0@gmail.com> wrote:
>
>>> If this does
>>> turn out to be a hardware issue, then yes that is just the kind of
case
>>> where you might expect to have to tweak things on the Xen command
line.
>>
>> the ''common'' hardware is, of course, the AMD Phenom
II X4 cpu ...
>
> The particular hardware that really matters is the chipset, as that is
where
> the HPET resides. Is that common across all the systems?
>
no, not identical,

(1)(2) ASUS M4A78T-E
  North Bridge  	AMD 790GX
  South Bridge 	AMD SB750

(3)    MSI 790FX-GD70
  North Bridge 	AMD 790FX
  South Bridge 	AMD SB750

(4)(5) GIGABYTE GA-MA770T-UD3P
  North Bridge 	AMD 770
  South Bridge 	AMD SB710

but, perhaps, related enough?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser

2010-Jan-19 18:35 UTC

head link

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

On 19/01/2010 18:04, "Keir Fraser" <keir.fraser@eu.citrix.com>
wrote:
>>> If this does
>>> turn out to be a hardware issue, then yes that is just the kind of
case
>>> where you might expect to have to tweak things on the Xen command
line.
>> 
>> the ''common'' hardware is, of course, the AMD Phenom
II X4 cpu ...
> 
> The particular hardware that really matters is the chipset, as that is
where
> the HPET resides. Is that common across all the systems?
Oh, I see they do vary, although all are AMD chipsets of presumably a
similar vintage.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Apparently Analagous Threads

Search for more reasonably related threads

Xen users - Jan 2010 - ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

[Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

[Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

[Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

RE: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

[Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

[Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

[Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Re: [Xen-devel] Re: [Xen-users] ntpd under Xen Dom0 exhibits extremely high jitter/noise? runs stable/quiet under non-xen kernel.

Apparently Analagous Threads