xen discuss - Jul 2008 - Is OpenSolaris dom0 passing local time as UTC to Linux domU''s?

Hi All:

I am running the following:

dom0: SunOS lss-nfsb01 5.11 snv_94 i86pc i386 i86xpv
with the following in my /etc/inet/ntp.conf:
[i]# --- OUR TIMESERVERS ----- 
server lss-time1.ih.lucent.com
server lss-time2.ih.lucent.com
server lss-time3.ih.lucent.com
[/i]

and the following command to start up the NTP service.
svcadm enable ntp

domU: CentOS 5.2 image, with no updates yet.

With ntp running on dom0, the time is accurate (within 200 milliseconds
normally, which is very acceptable).  Both dom0 and all of the domU''s
are set up for the same time zone, which is 5 hours behind UTC (Central Daylight
Time)

However, on the domU''s (which are not running ntp), the clock are
consistently 5 hour behind the dom0 (again, within 200 milliseconds).  To me, it
looks like Dom0 is not passing UTC to domU, but rather is passing the wall clock
to domU.

So, is anyone else seeing this behavior, or did I make a mistake in my setup
somewhere?

thanks,

Paul
 
 
This message posted from opensolaris.org

> However, on the domU''s (which are not running ntp),
> the clock are consistently 5 hour behind the dom0
> (again, within 200 milliseconds).  To me, it looks
> like Dom0 is not passing UTC to domU, but rather is
> passing the wall clock to domU.
What''s the domU''s python config file? 
Are you using the localtime=0 / localtime=1 option?
 
 
This message posted from opensolaris.org

Hi Juergen:

How would I dump the python config file?

Note, I set up the domU''s with the virt-install command, and have not
modified any other config files (but I am more than willing to do so if needed,
just wanted to make sure that such changes get documented so that others can
find it).

thanks,

Paul
 
 
This message posted from opensolaris.org

Paul Jochum wrote:
> Hi All:
> 
> I am running the following:
> 
> dom0: SunOS lss-nfsb01 5.11 snv_94 i86pc i386 i86xpv
> with the following in my /etc/inet/ntp.conf:
> [i]# --- OUR TIMESERVERS ----- 
> server lss-time1.ih.lucent.com
> server lss-time2.ih.lucent.com
> server lss-time3.ih.lucent.com
> [/i]
> 
> and the following command to start up the NTP service.
> svcadm enable ntp
> 
> domU: CentOS 5.2 image, with no updates yet.
> 
> With ntp running on dom0, the time is accurate (within 200 milliseconds
normally, which is very acceptable).  Both dom0 and all of the domU''s
are set up for the same time zone, which is 5 hours behind UTC (Central Daylight
Time)
> 
> However, on the domU''s (which are not running ntp), the clock are
consistently 5 hour behind the dom0 (again, within 200 milliseconds).  To me, it
looks like Dom0 is not passing UTC to domU, but rather is passing the wall clock
to domU.
> 
> So, is anyone else seeing this behavior, or did I make a mistake in my
setup somewhere?
> 
> thanks,
> 
> Paul
Solaris programs the "hardware clock" to be kept in UTC(GMT).  When 
running on the hypervisor the hypervisor provides the simulated clock to 
the domU''s.  However, setting the time in your Solaris dom0 should set 
the time that the hypervisor provides to the domU''s.  I don''t
know how
Linux expects the time to be kept in the hardware nor how often Linux 
checks the "Hardware" time after initially reading it.  So you should
at
least be sure that your Dom0 has set the time correctly before starting 
your domU''s.
What is the output of the rtc command on dom0? (or alternately, what is 
the contents of /etc/rtc_config on your dom0?
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org

Stuart Maybee wrote:
> Paul Jochum wrote:
>   
>> Hi All:
>>
>> I am running the following:
>>
>> dom0: SunOS lss-nfsb01 5.11 snv_94 i86pc i386 i86xpv
>> with the following in my /etc/inet/ntp.conf:
>> [i]# --- OUR TIMESERVERS ----- 
>> server lss-time1.ih.lucent.com
>> server lss-time2.ih.lucent.com
>> server lss-time3.ih.lucent.com
>> [/i]
>>
>> and the following command to start up the NTP service.
>> svcadm enable ntp
>>
>> domU: CentOS 5.2 image, with no updates yet.
>>
>> With ntp running on dom0, the time is accurate (within 200 milliseconds
normally, which is very acceptable).  Both dom0 and all of the domU''s
are set up for the same time zone, which is 5 hours behind UTC (Central Daylight
Time)
>>
>> However, on the domU''s (which are not running ntp), the clock
are consistently 5 hour behind the dom0 (again, within 200 milliseconds).  To
me, it looks like Dom0 is not passing UTC to domU, but rather is passing the
wall clock to domU.
>>
>> So, is anyone else seeing this behavior, or did I make a mistake in my
setup somewhere?
>>
>> thanks,
>>
>> Paul
>>     
>
> Solaris programs the "hardware clock" to be kept in UTC(GMT). 
When
> running on the hypervisor the hypervisor provides the simulated clock to 
> the domU''s.  However, setting the time in your Solaris dom0 should
set
> the time that the hypervisor provides to the domU''s.  I
don''t know how
> Linux expects the time to be kept in the hardware nor how often Linux 
> checks the "Hardware" time after initially reading it.  So you
should at
> least be sure that your Dom0 has set the time correctly before starting 
> your domU''s.
> What is the output of the rtc command on dom0? (or alternately, what is 
> the contents of /etc/rtc_config on your dom0?
>
>   
>>  
>>  
>> This message posted from opensolaris.org
>> _______________________________________________
>> xen-discuss mailing list
>> xen-discuss@opensolaris.org
>>     
>
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org
>   
Hi Stuart:

    Yes, Dom0 is keeping time accurately (using ntpdate command, the 
time is accurate).   Other than the 5 hour offset, the DomU''s are also 
accurate (or at least seem to be for me, around 200 ms in my testing so 
far).


    The outputs from Dom0 are:
$ rtc
US/Central
$ more /etc/rtc_config
#
#       This file (/etc/rtc_config) contains information used to manage the
#       x86 real time clock hardware.  The hardware is kept in
#       the machine''s local time for compatibility with other x86
#       operating systems.  This file is read by the kernel at
#       boot time.  It is set and updated by the /usr/sbin/rtc
#       command.  The ''zone_info'' field designates the local
#       time zone.  The ''zone_lag'' field indicates the number
#       of seconds between local time and Greenwich Mean Time.
#
zone_info=US/Central
zone_lag=18000
$

thanks,

Paul

Is your time zone five hours removed from GMT?  Taking
Daylight Savings Time into account, that is.  If so, it sounds like
the Dom0 is handing local time to the DomUs, who are interpreting
it as GMT and re-applying the time zone correction.  Either the
Dom0 has to be convinced to keep time in GMT, or the DomUs have to
be convinced to keep local time.

Mike O''Brien

The dom0 does not keep time for the domU''s nor does it hand time to 
them.  The hypervisor provides the time to all domains.  It retrieves it 
from the physical hardware.  DomU''s get their time from the hardware 
emulation provided by the hypervisor.  The only interaction dom0 has is 
that it has the capability of setting the hardware time.  Solaris will 
attempt to set the hardware time to UTC based on the time zone it is 
configured in.   But if you for instance set the time in dom0 to a 
different time after you started domU''s then I doubt the
domU''s would
see the changed time. (e.g if you set dom0 to a different timezone after 
the domU''s were started) I do not know what Linux expects the hardware 
to be programmed to.  I assume either local time or UTC.  But assuming 
you set the time in your dom0 and rebooted, the hypervisor should be 
providing utc to all domains as the time base.


Mike O''Brien wrote:
> 	Is your time zone five hours removed from GMT?  Taking
> Daylight Savings Time into account, that is.  If so, it sounds like
> the Dom0 is handing local time to the DomUs, who are interpreting
> it as GMT and re-applying the time zone correction.  Either the
> Dom0 has to be convinced to keep time in GMT, or the DomUs have to
> be convinced to keep local time.
> 
> Mike O''Brien
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org

> The dom0 does not keep time for the domU''s nor does it hand time
to
> them.  The hypervisor provides the time to all domains.  It retrieves it
> from the physical hardware.  DomU''s get their time from the
hardware
> emulation provided by the hypervisor.  The only interaction dom0 has is
> that it has the capability of setting the hardware time.[...]
	That may be, but the exact 5-hour time difference just smells
like a time zone error somewhere.

Mike O''Brien wrote:
> 	Is your time zone five hours removed from GMT?  Taking
> Daylight Savings Time into account, that is.  If so, it sounds like
> the Dom0 is handing local time to the DomUs, who are interpreting
> it as GMT and re-applying the time zone correction.  Either the
> Dom0 has to be convinced to keep time in GMT, or the DomUs have to
> be convinced to keep local time.
>
> Mike O''Brien 
>   
Yes, the time zone is 5 hours removed from GMT.  According to a XEN faq 
at (http://xen.epiuse.com/xen-faq.txt), "Note that the wall-clock time 
exported by Xen is UTC --- all domains must have appropriate timezone 
handling (i.e. a correct /etc/localtime file)." (but I am wondering if 
the OpenSolaris implementation is following this, since it does not seem 
to be exporting UTC).

Paul

Stuart Maybee wrote:
> The dom0 does not keep time for the domU''s nor does it hand time
to
> them.  The hypervisor provides the time to all domains.  It retrieves it 
> from the physical hardware.
Yep.

@Paul Jochum:  what time do you have setup in the system''s BIOS 
CMOS clock setup?     UTC time?  Or local time?


For Windows compatibility, I think it''s easiest to use local time (not
UTC)
in the system''s cmos clock. 

And Solaris tries to be compatible with Windows here.

(on dual boot boxes using local time in the cmos clock has the
disadvantage that during the switch between standard time
and daylight saving time, the cmos clock gets updated multiple
times)


Problem seems to be that the xen hypervisor is not compatible
with this, and expects UTC time in the PC''s cmos clock.

> DomU''s get their time from the hardware 
> emulation provided by the hypervisor. 
That would be a HVM domU.

But PV domU and dom0 can also use the time provided by the
hypervisor.
> The only interaction dom0 has is 
> that it has the capability of setting the hardware time. 
> Solaris will  attempt to set the hardware time to UTC based on the
> time zone it is configured in.
Hmm, is this correct?

Both todpc_set() and todxen_set() try to convert UTC time to local time,
before they update the cmos hardware clock (non-xen case)
or pass the new time to the hypervisor (on xen).

http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/i86pc/io/todpc_subr.c#127

    128 static void
    129 todpc_set(tod_ops_t *top, timestruc_t ts)
    130 {
    131 	todinfo_t tod = utc_to_tod(ts.tv_sec - ggmtl());

It adjusts by ggmt() / the gmt_lag.


http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/i86xpv/io/xpvtod.c#117

    124 static void
    125 todxen_set(tod_ops_t *top, timestruc_t ts)
    126 {
    127 	xen_platform_op_t op;
    128 
    129 	if (DOMAIN_IS_INITDOMAIN(xen_info)) {
    130 		ASSERT(MUTEX_HELD(&tod_lock));
    131 		TODOP_SET(top->tod_next, ts);
    132 
    133 		op.cmd = XENPF_settime;
    134 		op.interface_version = XENPF_INTERFACE_VERSION;
    135 		op.u.settime.secs = ts.tv_sec - ggmtl();  
<<<<<<<<<<<<<<<<<
    136 		op.u.settime.nsecs = ts.tv_nsec;
    137 		op.u.settime.system_time = xpv_getsystime();
    138 		(void) HYPERVISOR_platform_op(&op);

It adjusts UTC time by gmt_lag, on line 135.


The only exception would be a setup that does use UTC in the 
pc''s cmos clock, running Solaris with an /etc/rtc_config 
setup that uses zone_info=UTC and zone_lag=0.


OTOH, the "struct shared_info" definition in /usr/include/xen/xen.h
tells us that the field wc_sec is supposed to store (the hypervisor boot)
time in UTC:

    /*
     * Wallclock time: updated only by control software. Guests should base
     * their gettimeofday() syscall on this wallclock-base value.
     */
    uint32_t wc_version;      /* Version counter: see vcpu_time_info_t. */
    uint32_t wc_sec;          /* Secs  00:00:00 UTC, Jan 1, 1970.  */
    uint32_t wc_nsec;         /* Nsecs 00:00:00 UTC, Jan 1, 1970.  */


But with a standard SXCE installation using local time in the pc cmos clock
I do have an /etc/rtc_config with

    zone_info=Europe/Berlin
    zone_lag=-7200

and when I look at the shared page field wc_sec, it do see local time
and not UTC time passed from the hypervisor to dom0 or domU.
And the Solaris dom0/domU kernel converts the local time to UTC
using the zone_lag from the /etc/rtc_config file.
 
 
This message posted from opensolaris.org

Jürgen Keil wrote:
> Stuart Maybee wrote:
> 
>> The dom0 does not keep time for the domU''s nor does it hand
time to
>> them.  The hypervisor provides the time to all domains.  It retrieves
it
>> from the physical hardware.
> 
> Yep.
> 
> @Paul Jochum:  what time do you have setup in the system''s BIOS 
> CMOS clock setup?     UTC time?  Or local time?
> 
> 
> For Windows compatibility, I think it''s easiest to use local time
(not UTC)
> in the system''s cmos clock. 
> 
> And Solaris tries to be compatible with Windows here.
> 
> (on dual boot boxes using local time in the cmos clock has the
> disadvantage that during the switch between standard time
> and daylight saving time, the cmos clock gets updated multiple
> times)
> 
> 
> Problem seems to be that the xen hypervisor is not compatible
> with this, and expects UTC time in the PC''s cmos clock.
> 
> 
>> DomU''s get their time from the hardware 
>> emulation provided by the hypervisor. 
> 
> That would be a HVM domU.
> 
> But PV domU and dom0 can also use the time provided by the
> hypervisor.
> 
>> The only interaction dom0 has is 
>> that it has the capability of setting the hardware time. 
> 
>> Solaris will  attempt to set the hardware time to UTC based on the
>> time zone it is configured in.
> 
> Hmm, is this correct?
> 
> Both todpc_set() and todxen_set() try to convert UTC time to local time,
> before they update the cmos hardware clock (non-xen case)
> or pass the new time to the hypervisor (on xen).
> 
>
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/i86pc/io/todpc_subr.c#127
> 
>     128 static void
>     129 todpc_set(tod_ops_t *top, timestruc_t ts)
>     130 {
>     131 	todinfo_t tod = utc_to_tod(ts.tv_sec - ggmtl());
> 
> It adjusts by ggmt() / the gmt_lag.
> 
> 
>
http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/i86xpv/io/xpvtod.c#117
> 
>     124 static void
>     125 todxen_set(tod_ops_t *top, timestruc_t ts)
>     126 {
>     127 	xen_platform_op_t op;
>     128 
>     129 	if (DOMAIN_IS_INITDOMAIN(xen_info)) {
>     130 		ASSERT(MUTEX_HELD(&tod_lock));
>     131 		TODOP_SET(top->tod_next, ts);
>     132 
>     133 		op.cmd = XENPF_settime;
>     134 		op.interface_version = XENPF_INTERFACE_VERSION;
>     135 		op.u.settime.secs = ts.tv_sec - ggmtl();  
<<<<<<<<<<<<<<<<<
>     136 		op.u.settime.nsecs = ts.tv_nsec;
>     137 		op.u.settime.system_time = xpv_getsystime();
>     138 		(void) HYPERVISOR_platform_op(&op);
> 
> It adjusts UTC time by gmt_lag, on line 135.
> 
> 
> The only exception would be a setup that does use UTC in the 
> pc''s cmos clock, running Solaris with an /etc/rtc_config 
> setup that uses zone_info=UTC and zone_lag=0.
> 
> 
> OTOH, the "struct shared_info" definition in
/usr/include/xen/xen.h
> tells us that the field wc_sec is supposed to store (the hypervisor boot)
> time in UTC:
> 
>     /*
>      * Wallclock time: updated only by control software. Guests should base
>      * their gettimeofday() syscall on this wallclock-base value.
>      */
>     uint32_t wc_version;      /* Version counter: see vcpu_time_info_t. */
>     uint32_t wc_sec;          /* Secs  00:00:00 UTC, Jan 1, 1970.  */
>     uint32_t wc_nsec;         /* Nsecs 00:00:00 UTC, Jan 1, 1970.  */
> 
> 
> But with a standard SXCE installation using local time in the pc cmos clock
> I do have an /etc/rtc_config with
> 
>     zone_info=Europe/Berlin
>     zone_lag=-7200
> 
> and when I look at the shared page field wc_sec, it do see local time
> and not UTC time passed from the hypervisor to dom0 or domU.
> And the Solaris dom0/domU kernel converts the local time to UTC
> using the zone_lag from the /etc/rtc_config file.
>  
>  
You are right as usual Jurgen:-)  I should have looked beyond the date 
command which does convert to utc and calls stime to have the kernel set 
the system time.
Behind the scenes the kernel reconverts back to local time.  And then 
programs the hardware. Which is the time that Xen will supply to all 
domains either via emulated hardware or via shared time structs to PV 
domains.

> This message posted from opensolaris.org
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org