samba - Nov 2005 - Wintertime/summertime difference - Samba servers show wrong time ?

When the Central European Time was last switched back to standard,
at 03:00 last Sunday, the October 30th, a process died on one of my
Windows clients with a mysterious "unknown error". When it was 
restarted it just went merrily on with its task. Luckily it wasn't 
part of a life support system.

I found out that the immediate cause was how file timestamps were
interpreted/presented by samba server as opposed to the expectations
of a Windows client.

According to this M$ Article: 

     http://support.microsoft.com/kb/129574/en-us

    "When Windows NT automatically adjusts for daylight savings time,
     the times on files on Windows NT file system (NTFS) partitions 
     and the events in the event logs are retroactively shifted by 
     one hour, even though the files and event records were created 
     before the daylight savings time change."

In other words, Linux-based Samba servers keep on showing the right
time of the day when a file was created/changed/modified/accessed
whereas Windows falsifies it by an hour retroactively. But being 
right is not enough. As much as I regret having to do it, I need a 
tweak to resynch the file times representation of my Samba servers 
with the expected and well-documented behaviour of Windows clients, 
even though it means lying through one's own teeth. 

As a matter of fact, I don't understand how this discrepancy is at
all possible!
>From what I gathered in the documentations on both sides of the
fence, Unix traditionally stamps file times (create/status change, 
modify and last read access) with a long integer (32 bits) counting 
full seconds since midnight A.M. January 1, 1970 in Greenwhich, EU, 
whereas the NT File System apparently uses a larger data type to 
count decimicroseconds (or should I say hectonanoseconds) since 
the same time of night in the said British village on January 1, 
1601, when it wants to stamp one of its own set of file times, 
creation, content alteration, MFT change or last read access.

So basically both systems keep track of time in timezone-neutral
units, different in scale and offset but essentially interchangeable
within limits, and only interpret it as a certain time of day/night 
of this or that day of one or another month/year according to the 
user's locale.

I can't imagine that an M$ SMB server (a Windows server or 
workstation) marshals anything else on the wire than the raw data 
type in the file stamp (the wrong time is in the eye of the client), 
so Samba has to be doing something wrong if the time stamp is 
perceived on the client side as not retroactively an hour earlier 
than it really was for a file which was manipulated in the interval
between 03:00 a.m. on the first Sunday of April and 03:00 a.m. of
the last Sunday of October when viewed from outside of this interval.

Please correct me if I'm wrong, but also don't hesitate to let me 
know how to fix the problem, even if it was perhaps already 
discussed in the past.

Yours truely

Surprisingly few threads on this fascinating subject!
(They're all in the Cc:) Only one solution proposed by Thomas 
Honigman and Thomas Guenther. In a posting of Feb 8, this year, 
they proposed conditioning the use of "kludge GMT", which is 
what Samba marshals on the wire as GMT but which is wrong under 
most circumstances involving daylight savings time switch, 
upon the value of the boolean smb.conf option "use kludge gmt", 
even using "TRUE" as default in order not to break the 
compatibility.

Can anyone tell why this 9-months old proposal was not accepted ?

I've implemented their patches (but made FALSE the default,
of course) and it solved my problems big way.

The way it works now in the sources, Samba goes out of its way 
to force Windows clients to see the file times the way Unix and 
other more mature systems see them. If a file was modified at 
noon 12:00:00 of any day, it shows 12:00:00 always, regardless 
of the date on which it was modified or the date on which one 
is beholding it. Samba does it by subtracting from the real GMT 
in the timestamps the difference between "TimeDiff(timestamp)" 
and "get_serverzone()", which means that it fakes the timestamp 
GMT in such a way that Windows still see the right time and not 
the wrong Windows time, which is actually what everybody wants 
to see, so that pacemakers don't stop and rod injectors don't melt.

Small details like this are show stoppers.
> ... the immediate cause was how file timestamps were
> interpreted/presented by samba server as opposed to the expectations
> of a Windows client.
> 
> According to this M$ Article:
> 
>       http://support.microsoft.com/kb/129574/en-us
> 
>      "When Windows NT automatically adjusts for daylight savings time,
>       the times on files on Windows NT file system (NTFS) partitions
>       and the events in the event logs are retroactively shifted by
>       one hour, even though the files and event records were created
>       before the daylight savings time change."
> 
> In other words, Linux-based Samba servers keep on showing the right
> time of the day when a file was created/changed/modified/accessed
> whereas Windows falsifies it by an hour retroactively. But being
> right is not enough.

Dragan Krnic wrote:
>>From what I gathered in the documentations on both sides of the
> fence, Unix traditionally stamps file times (create/status change, 
> modify and last read access) with a long integer (32 bits) counting 
> full seconds since midnight A.M. January 1, 1970 in Greenwhich, EU, 
> whereas the NT File System apparently uses a larger data type to 
> count decimicroseconds (or should I say hectonanoseconds) since 
> the same time of night in the said British village on January 1, 
> 1601, when it wants to stamp one of its own set of file times, 
> creation, content alteration, MFT change or last read access.
I think the difference in timestamping is also the cause for the 
annoying bug

https://bugzilla.samba.org/show_bug.cgi?id=3124


der tom

>> Surprisingly few threads on this fascinating subject!
>> (They're all in the Cc:) Only one solution proposed by Thomas 
>> Honigman and Thomas Guenther. In a posting of Feb 8, this year, 
>> they proposed conditioning the use of "kludge GMT", which is 
>> what Samba marshals on the wire as GMT but which is wrong under 
>> most circumstances involving daylight savings time switch, 
>> upon the value of the boolean smb.conf option "use kludge
gmt",
>> even using "TRUE" as default in order not to break the
compatibility.
>> 
>> Can anyone tell why this 9-months old proposal was not accepted ?
>> 
>> I've implemented their patches (but made FALSE the default,
>> of course) and it solved my problems big way.
>> 
>> The way it works now in the sources, Samba goes out of its way to 
>> force Windows clients to see the file times the way Unix and other 
>> more mature systems see them. If a file was modified at noon 12:00:00 
>> of any day, it shows 12:00:00 always, regardless of the date on which 
>> it was modified or the date on which one is beholding it. Samba does 
>> it by subtracting from the real GMT in the timestamps the difference 
>> between "TimeDiff(timestamp)" and
"get_serverzone()", which means that
>> it fakes the timestamp GMT in such a way that Windows still see the 
>> right time and not the wrong Windows time, which is actually what 
>> everybody wants to see, so that pacemakers don't stop and rod
injectors
>> don't melt.
>> 
>> Small details like this are show stoppers.
> 
> It's funny you should comment on this - I was in the middle
> of moving to the Samba4 way of handling times and timezones
> (back porting the Samba4 code to Samba3) to fix the timezone
> problems people have. I was planning to get this into 3.0.21.
> 
> Would this fix the problem ? I'm expecting so, but would
> appreciate more feedback.
> 
> I'll look back for the patch to remove the kludge GMT, but
> I'd rather just remove it entirely and clean up the code.
Good to hear that you think the same way, Jeremy.

A clean break is what I would suggest too, but since it would
take a long time to propagate, perhaps something more eye
catching, like the mentioned weird new option in smb.conf, 
even if the default should be FALSE, would be appropriate to 
draw attention to the fact that we're M$-compliant on that count.

"Use kludge GMT" is perhaps not revealing enough, if not
downright misleading. 

How about "Ape Microsoft's Daylight Savings Time bug = TRUE"
for default and mentioning the value explicitly in checkparms?

>> It's funny you should comment on this - I was in the middle
>> of moving to the Samba4 way of handling times and timezones
>> (back porting the Samba4 code to Samba3) to fix the timezone
>> problems people have. I was planning to get this into 3.0.21.
>>
>> Would this fix the problem ? I'm expecting so, but would
>> appreciate more feedback.
>>
>> I'll look back for the patch to remove the kludge GMT, but
>> I'd rather just remove it entirely and clean up the code.
> 
> In a follow up to this - I'd like to get rid of the "time
offset"
> parameter in 3.0.21 as part of the cleanup. Here is the definition :
> 
>         time offset (G)
>                This  parameter  is a setting in minutes to add to 
>                the normal GMT to local time conversion. This is useful if
>                you are serving a lot of PCs that have incorrect daylight 
>                saving time handling.
> 
> I won't remove the parameter, just the effect it has in the
> timezone code.
> 
> Please comment if you're using this parameter.
Of course not.

>> From what I gathered in the documentations on both sides of the
>> fence, Unix traditionally stamps file times (create/status change, 
>> modify and last read access) with a long integer (32 bits) counting 
>> full seconds since midnight A.M. January 1, 1970 in Greenwhich, EU, 
>> whereas the NT File System apparently uses a larger data type to 
>> count decimicroseconds (or should I say hectonanoseconds) since the 
>> same time of night in the said British village on January 1, 1601, 
>> when it wants to stamp one of its own set of file times, creation, 
>> content alteration, MFT change or last read access.
> 
> I think the difference in timestamping is also the cause for the annoying
bug
> 
> https://bugzilla.samba.org/show_bug.cgi?id=3124
I concur.

>>> From what I gathered in the documentations on both sides of the
>>> fence, Unix traditionally stamps file times (create/status change, 
>>> modify and last read access) with a long integer (32 bits) counting
>>> full seconds since midnight A.M. January 1, 1970 in Greenwhich, EU,
>>> whereas the NT File System apparently uses a larger data type to 
>>> count decimicroseconds (or should I say hectonanoseconds) since 
>>> the same time of night in the said British village on January 1, 
>>> 1601, when it wants to stamp one of its own set of file times, 
>>> creation, content alteration, MFT change or last read access.
>>
>> I think the difference in timestamping is also the cause for the
annoying bug
>>
>> https://bugzilla.samba.org/show_bug.cgi?id=3124
> 
> I don't think this is the same bug - this looks like a difference
> in the timstamps hold on the POSIX filesystem vs the NTFS one.
It's not about daylight savings time switch but it is about the
granularity of time keeping, is what Tom tried to say.

>> The way it works now in the sources, Samba goes out of its way
>> to force Windows clients to see the file times the way Unix and
>> other more mature systems see them. If a file was modified at
>> noon 12:00:00 of any day, it shows 12:00:00 always, regardless
>> of the date on which it was modified or the date on which one
>> is beholding it. Samba does it by subtracting from the real GMT
>> in the timestamps the difference between
"TimeDiff(timestamp)"
>> and "get_serverzone()", which means that it fakes the
timestamp
>> GMT in such a way that Windows still see the right time and not
>> the wrong Windows time, which is actually what everybody wants
>> to see, so that pacemakers don't stop and rod injectors don't
melt.
> 
> If anyone is relying on date & timestamps under Windows (or
> even using any general-purpose OS!) for such safety-critical
> things as pacemakers and nuclear reactors, then we have a
> much bigger problem ;-)
We do. You'd be surprised to learn that a GPOS is BEING used in one 
of those environments I facetiously mentioned and is certified for 
that by the same organisation that certifies roll-worthiness of 
road vehicles among other things. But the point is that we are oft
in a "2b || !2b" dilemma when 2 different worldviews interface in
public. Safety of the public should trump up esthetic objections,
even sense of right and wrong.

>>> It's funny you should comment on this - I was in the middle
>>> of moving to the Samba4 way of handling times and timezones
>>> (back porting the Samba4 code to Samba3) to fix the timezone
>>> problems people have. I was planning to get this into 3.0.21.
>>>
>>> Would this fix the problem ? I'm expecting so, but would
>>> appreciate more feedback.
>>>
>>> I'll look back for the patch to remove the kludge GMT, but
>>> I'd rather just remove it entirely and clean up the code.
>>
>> In a follow up to this - I'd like to get rid of the "time
offset"
>> parameter in 3.0.21 as part of the cleanup. Here is the definition :
>>
>>         time offset (G)
>>                This  parameter  is a setting in minutes to add to
>>                the normal GMT to local time conversion. This is useful
if
>>                you are serving a lot of PCs that have incorrect
daylight
>>                saving time handling.
>>
>> I won't remove the parameter, just the effect it has in the
>> timezone code.
>>
>> Please comment if you're using this parameter.
> 
> Of course not.
Things are not so clear-cut as I thought. The patch works, but 
there seems to be another interaction with Linux setting for 
the hardware clock: UTC vs. local time.

I will test it more thorougly on a test machine and post again.