Hi. I run peridocally (from cron) on all of my machines 30 * * * * root /sbin/hwclock --systohc All of those machines in question take their time via NTP from the same local server, and that server gets its time from a ntp pool. Now I had to reboot a couple of them two days ago and to my surprise all had problems with the time upon booting. Here are the important files: [root at XXXXXX ~] #>l /etc/adjtime 0.001687 1289518202 0.000000 1289518202 LOCAL [root at XXXXXXX ~] #>l /etc/sysconfig/clock ZONE="Australia/Melbourne" UTC=false ARC=false So from my understanding the hwclock should contain the local time. [root at XXXXXX ~] #>date Fri Nov 12 11:26:23 EST 2010 [root at XXXXXX ~] #>hwclock Fri 12 Nov 2010 11:26:42 EST -0.167976 seconds [root at XXXXXX ~] #> However on boot I get the following: Nov 10 19:08:37 XXXXXX syslogd 1.4.1: restart. Nov 10 19:08:37 XXXXXX kernel: klogd 1.4.1, log source = /proc/kmsg started. Nov 10 19:08:37 XXXXXX kernel: Linux version 2.6.18-164.11.1.el5 (mockbuild at builder10.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4. 1.2-46)) #1 SMP Wed Jan 20 07:32:21 EST 2010 Nov 10 19:08:37 XXXXXX kernel: Command line: ro root=/dev/sda2 vga=791 Nov 10 19:08:37 XXXXXX kernel: BIOS-provided physical RAM map: ... ... Nov 10 19:08:51 XXXXXX kernel: IPv6 over IPv4 tunneling driver Nov 10 08:08:52 XXXXXX ntpdate[2464]: step time server 192.168.1.1 offset -39599.950905 sec Nov 10 08:08:52 XXXXXX xinetd[2447]: xinetd Version 2.3.14 started with libwrap loadavg labeled-networking options compiled in. and off course dovecot falls over too "Time just moved backwards by 39599 seconds." Now, 39600s is 11 hours, which is (inc DST) *MY* offset from Greenwich. So what am I doing wrong? The idea of running hwclock is to make sure that exactly the problem with dovecot does NOT occur, and ntp does not have a coughing fit when the hardware clock is not close to the correct time upon booting. The last time I booted some of those machine was more than 200 days ago, so the hwclock will be skewed if I do not update it. Jobst -- Keyboard not found - please clean up desktop! | |0| | Jobst Schmalenbach, jobst at barrett.com.au, General Manager | | |0| Barrett Consulting Group P/L & The Meditation Room P/L |0|0|0| +61 3 9532 7677, POBox 277, Caulfield South, 3162, Australia
On 11/11/2010 06:41 PM, Jobst Schmalenbach wrote:> Hi. > > I run peridocally (from cron) on all of my machines > > 30 * * * * root /sbin/hwclock --systohc > > All of those machines in question take their time via NTP > from the same local server, and that server gets its time > from a ntp pool. > > Now I had to reboot a couple of them two days ago and to my surprise > all had problems with the time upon booting. > > Here are the important files: > > [root at XXXXXX ~] #>l /etc/adjtime > 0.001687 1289518202 0.000000 > 1289518202 > LOCAL > > [root at XXXXXXX ~] #>l /etc/sysconfig/clock > ZONE="Australia/Melbourne" > UTC=false > ARC=false > > So from my understanding the hwclock should contain the local time. > > [root at XXXXXX ~] #>date > Fri Nov 12 11:26:23 EST 2010 > [root at XXXXXX ~] #>hwclock > Fri 12 Nov 2010 11:26:42 EST -0.167976 seconds > [root at XXXXXX ~] #> > > However on boot I get the following: > > Nov 10 19:08:37 XXXXXX syslogd 1.4.1: restart. > Nov 10 19:08:37 XXXXXX kernel: klogd 1.4.1, log source = /proc/kmsg started. > Nov 10 19:08:37 XXXXXX kernel: Linux version 2.6.18-164.11.1.el5 (mockbuild at builder10.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4. > 1.2-46)) #1 SMP Wed Jan 20 07:32:21 EST 2010 > Nov 10 19:08:37 XXXXXX kernel: Command line: ro root=/dev/sda2 vga=791 > Nov 10 19:08:37 XXXXXX kernel: BIOS-provided physical RAM map: > ... > ... > Nov 10 19:08:51 XXXXXX kernel: IPv6 over IPv4 tunneling driver > Nov 10 08:08:52 XXXXXX ntpdate[2464]: step time server 192.168.1.1 offset -39599.950905 sec > Nov 10 08:08:52 XXXXXX xinetd[2447]: xinetd Version 2.3.14 started with libwrap loadavg labeled-networking options compiled in. > > and off course dovecot falls over too "Time just moved backwards by 39599 seconds." > > Now, 39600s is 11 hours, which is (inc DST) *MY* offset from Greenwich. > > > So what am I doing wrong? > The idea of running hwclock is to make sure that exactly the problem with dovecot does NOT occur, and ntp does not have a coughing fit when the hardware clock is not close to the correct time upon booting. > The last time I booted some of those machine was more than 200 days ago, so the hwclock will be skewed if I do not update it.Have you looked at altering your /etc/sysconfig/ntpd file to directly update your hardware clock?
Jobst Schmalenbach wrote:> Hi. > > I run peridocally (from cron) on all of my machines > > 30 * * * * root /sbin/hwclock --systohc > > All of those machines in question take their time via NTP > from the same local server, and that server gets its time > from a ntp pool. > > Now I had to reboot a couple of them two days ago and to my surprise > all had problems with the time upon booting. > > Here are the important files: > > [root at XXXXXX ~] #>l /etc/adjtime > 0.001687 1289518202 0.000000 > 1289518202 > LOCAL > > [root at XXXXXXX ~] #>l /etc/sysconfig/clock > ZONE="Australia/Melbourne" > UTC=false > ARC=false > > So from my understanding the hwclock should contain the local time. > > [root at XXXXXX ~] #>date > Fri Nov 12 11:26:23 EST 2010 > [root at XXXXXX ~] #>hwclock > Fri 12 Nov 2010 11:26:42 EST -0.167976 seconds > [root at XXXXXX ~] #> > > However on boot I get the following: > > Nov 10 19:08:37 XXXXXX syslogd 1.4.1: restart. > Nov 10 19:08:37 XXXXXX kernel: klogd 1.4.1, log source = /proc/kmsg started. > Nov 10 19:08:37 XXXXXX kernel: Linux version 2.6.18-164.11.1.el5 (mockbuild at builder10.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4. > 1.2-46)) #1 SMP Wed Jan 20 07:32:21 EST 2010 > Nov 10 19:08:37 XXXXXX kernel: Command line: ro root=/dev/sda2 vga=791 > Nov 10 19:08:37 XXXXXX kernel: BIOS-provided physical RAM map: > ... > ... > Nov 10 19:08:51 XXXXXX kernel: IPv6 over IPv4 tunneling driver > Nov 10 08:08:52 XXXXXX ntpdate[2464]: step time server 192.168.1.1 offset -39599.950905 sec > Nov 10 08:08:52 XXXXXX xinetd[2447]: xinetd Version 2.3.14 started with libwrap loadavg labeled-networking options compiled in. > > and off course dovecot falls over too "Time just moved backwards by 39599 seconds." > > Now, 39600s is 11 hours, which is (inc DST) *MY* offset from Greenwich. > > > So what am I doing wrong? > The idea of running hwclock is to make sure that exactly the problem with dovecot does NOT occur, and ntp does not have a coughing fit when the hardware clock is not close to the correct time upon booting. > The last time I booted some of those machine was more than 200 days ago, so the hwclock will be skewed if I do not update it. > > >Last time I checked the shut down of a machine writes the system time to the H/W prior to reboot. Thus the problem you describe should not occur if proper shutdown occurs. HTH> Jobst > > > >-------------- next part -------------- A non-text attachment was scrubbed... Name: rkampen.vcf Type: text/x-vcard Size: 278 bytes Desc: not available URL: <http://lists.centos.org/pipermail/centos/attachments/20101111/08f618a3/attachment-0001.vcf>
> and off course dovecot falls over too "Time just moved > backwards by 39599 seconds." > > Now, 39600s is 11 hours, which is (inc DST) *MY* offset from > Greenwich. > > > So what am I doing wrong?I have this problem when dead batteries on the mobo prevent the hwclock from preserving the time. Reboots don't show this (shutdown -r) but yanking the AC to fiddle with switches on the cards (which takes pulling them out) or swapping known-good with suspect-under-test gives me a boot-up time somewhere back in August of 2006. ******************************************************************* This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept for the presence of computer viruses. www.Hubbell.com - Hubbell Incorporated**
On Thursday 11 November 2010 20:41:45 Jobst Schmalenbach wrote:> Now I had to reboot a couple of them two days ago and to my surprise > all had problems with the time upon booting.Hi, Are you 100% sure that your timezone file (/etc/localtime) corresponds to the one Australia/Melbourne? Try this: diff /etc/localtime /usr/share/zoneinfo/Australia/Melbourne Besides that, try to see if there's any script within /etc that tries to set the TZ variable somewhere as it seems it is trying to set your system time to flat UTC. If I understand correctly, your hardware clock indeed is storing "localtime" as seen on the output when you are booting... but as soon as ntpd kicks in, it sets the system time to UTC (which is 11 hours behind your localtime). Right? HTH, Jorge
On Thursday 11 November 2010 20:41:45 Jobst Schmalenbach wrote:> Nov 10 08:08:52 XXXXXX ntpdate[2464]: step time server 192.168.1.1 offset > -39599.950905 secAlso, try to disable ntpdate with "chkconfig ntpdate off" and reboot the machine and see if that solves the problem. If it does, then you can concentrate on ntpdate... -- Jorge
On 11/11/2010 06:41 PM, Jobst Schmalenbach wrote:> Hi. > > I run peridocally (from cron) on all of my machines > > 30 * * * * root /sbin/hwclock --systohc > > All of those machines in question take their time via NTP > from the same local server, and that server gets its time > from a ntp pool. > > Now I had to reboot a couple of them two days ago and to my surprise > all had problems with the time upon booting. > > Here are the important files: > > [root at XXXXXX ~] #>l /etc/adjtime > 0.001687 1289518202 0.000000 > 1289518202 > LOCAL > > [root at XXXXXXX ~] #>l /etc/sysconfig/clock > ZONE="Australia/Melbourne" > UTC=false > ARC=false > > So from my understanding the hwclock should contain the local time. > > [root at XXXXXX ~] #>date > Fri Nov 12 11:26:23 EST 2010 > [root at XXXXXX ~] #>hwclock > Fri 12 Nov 2010 11:26:42 EST -0.167976 seconds > [root at XXXXXX ~] #> > > However on boot I get the following: > > Nov 10 19:08:37 XXXXXX syslogd 1.4.1: restart. > Nov 10 19:08:37 XXXXXX kernel: klogd 1.4.1, log source = /proc/kmsg started. > Nov 10 19:08:37 XXXXXX kernel: Linux version 2.6.18-164.11.1.el5 (mockbuild at builder10.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4. > 1.2-46)) #1 SMP Wed Jan 20 07:32:21 EST 2010 > Nov 10 19:08:37 XXXXXX kernel: Command line: ro root=/dev/sda2 vga=791 > Nov 10 19:08:37 XXXXXX kernel: BIOS-provided physical RAM map: > ... > ... > Nov 10 19:08:51 XXXXXX kernel: IPv6 over IPv4 tunneling driver > Nov 10 08:08:52 XXXXXX ntpdate[2464]: step time server 192.168.1.1 offset -39599.950905 sec > Nov 10 08:08:52 XXXXXX xinetd[2447]: xinetd Version 2.3.14 started with libwrap loadavg labeled-networking options compiled in. > > and off course dovecot falls over too "Time just moved backwards by 39599 seconds." > > Now, 39600s is 11 hours, which is (inc DST) *MY* offset from Greenwich. > > > So what am I doing wrong? > The idea of running hwclock is to make sure that exactly the problem with dovecot does NOT occur, and ntp does not have a coughing fit when the hardware clock is not close to the correct time upon booting. > The last time I booted some of those machine was more than 200 days ago, so the hwclock will be skewed if I do not update it.By any chance is /etc/localtime a symlink to a file system that is not mounted early in the boot process? That's why /etc/localtime is normally a _copy_ of the appropriate zoneinfo file. -- Bob Nichols "NOSPAM" is really part of my email address. Do NOT delete it.
Jobst Schmalenbach wrote, On 11/11/2010 07:41 PM:> Hi. > > I run peridocally (from cron) on all of my machines > > 30 * * * * root /sbin/hwclock --systohc >Why? AFAIK a kernel that is running ntpd and ntpd thinks has reasonably synced to the NTP server will, every _eleven_ minutes write the system time to the hardware clock, and you can't stop it without modifying the kernel or ntpd.> All of those machines in question take their time via NTP > from the same local server, and that server gets its time > from a ntp pool. >reasonable NTP setup.> Now I had to reboot a couple of them two days ago and to my surprise > all had problems with the time upon booting. > > Here are the important files: > > [root at XXXXXX ~] #>l /etc/adjtime > 0.001687 1289518202 0.000000 > 1289518202 > LOCAL > > [root at XXXXXXX ~] #>l /etc/sysconfig/clock > ZONE="Australia/Melbourne" > UTC=false > ARC=false > > So from my understanding the hwclock should contain the local time. > > [root at XXXXXX ~] #>date > Fri Nov 12 11:26:23 EST 2010 > [root at XXXXXX ~] #>hwclock > Fri 12 Nov 2010 11:26:42 EST -0.167976 seconds > [root at XXXXXX ~] #> >Is 'EST' the time zone abbreviation you expect for Melbourne? As I am based in the US, I expect 'EST' to be "Eastern Standard Time" for New York/New York, so I ask for your help in understanding. We might be able to see a different pattern if we take the TZ out of the equations. date -u ; hwclock --show --utc; date -u date ; hwclock --show ; date> However on boot I get the following: > > Nov 10 19:08:37 XXXXXX syslogd 1.4.1: restart. > Nov 10 19:08:37 XXXXXX kernel: klogd 1.4.1, log source = /proc/kmsg started. > Nov 10 19:08:37 XXXXXX kernel: Linux version 2.6.18-164.11.1.el5 (mockbuild at builder10.centos.org) (gcc version 4.1.2 20080704 (Red Hat 4. > 1.2-46)) #1 SMP Wed Jan 20 07:32:21 EST 2010 > Nov 10 19:08:37 XXXXXX kernel: Command line: ro root=/dev/sda2 vga=791 > Nov 10 19:08:37 XXXXXX kernel: BIOS-provided physical RAM map: > ... > ... > Nov 10 19:08:51 XXXXXX kernel: IPv6 over IPv4 tunneling driver > Nov 10 08:08:52 XXXXXX ntpdate[2464]: step time server 192.168.1.1 offset -39599.950905 sec > Nov 10 08:08:52 XXXXXX xinetd[2447]: xinetd Version 2.3.14 started with libwrap loadavg labeled-networking options compiled in. > > and off course dovecot falls over too "Time just moved backwards by 39599 seconds." > > Now, 39600s is 11 hours, which is (inc DST) *MY* offset from Greenwich. > > > So what am I doing wrong?Running a Linux _Server_ as if it had to dual boot with windows. i.e. the hardware clock should be kept in UTC unless you need to boot the same machine with windows.> The idea of running hwclock is to make sure that > exactly the problem with dovecot does NOT occur, > and ntp does not have a coughing fit when the hardware > clock is not close to the correct time upon booting.The standard start script (/etc/rc.d/init.d/ntpd) does a ntpdate before running (which is what you see in your log above) to keep ntp from "coughing".> The last time I booted some of those machine was more than 200 days ago, > so the hwclock will be skewed if I do not update it.I *WAS* beginning to think like the others, that the TZ file used by hwclock and by date don't match. However, I now *believe* I KNOW the source of the delta! IIRC the kernel magic (write system time to HC every eleven minutes) I was writing about earlier ... I don't think takes into account the local TZ, i.e., it ALWAYS works UTC. I would have to read the kernel source again to prove it, or suggest to you to try the following: 1) *remove* your cron job that called hwclock, because it is and will cause problems. 2) let the machine sync with the NTP server i.e., ntpdc -c kern |grep status returns something like: status: 0009 pll fll 2a) wait 12 minutes. 3) run: date -u ; hwclock --show --utc; date -u ; \ date ; hwclock --show ; date 4) run hwclock --systohc; \ date -u ; hwclock --show --utc; date -u ; \ date ; hwclock --show ; date 5) wait 23 more minutes 6) run date -u ; hwclock --show --utc; date -u ; \ date ; hwclock --show ; date if at 3 and 6 the utc versions of date and hwclock are in sync, then it is the ntpd synced kernel that is setting a utc time into the hwclock and you need to change the last line in /etc/adjtime to UTC instead of LOCAL. Otherwise a bit more thinking is in order. good luck. -- Todd Denniston Crane Division, Naval Surface Warfare Center (NSWC Crane) Harnessing the Power of Technology for the Warfighter