I think that there is a bug in the as.POSIXct function on Windows. Here is what I get on Win2000, Pentium III machine in R 1.8.1.> dd1 <- ISOdatetime(2003, 10, 26, 0, 59, 59) > dd2 <- ISOdatetime(2003, 10, 26, 1, 0, 0) > dd2 - dd1Time difference of 1.000278 hours Now, the 26th of October was the day that change to the standard time occurred, so I suspect that this has something to do with that. In fact> dd1[1] "2003-10-26 00:59:59 Central Daylight Time"> dd2[1] "2003-10-26 01:00:00 Central Standard Time" so it looks like the switch from CDT to CST happens at 1:00 (instead of 2:00 ?). Since the only thing the difftime function does is unclass the as.POSIXct values of its two arguments, the error seems to be in the as.POSIXct code. I looked at the C code of the as.POSIXct function but I do not know enough about R code and C handling of the time structures to find an error there. Perhaps, the error is actually in the C library handling of time structures on Windows machines. All this seems to work fine in version 1.8.1 patched 11-24 on a Linux machine, i.e. the above difference is 1second. However, the swicth between CDT and CSD also accurs at 1:00. Andy __________________________________ Andy Jaworski 518-1-01 Process Laboratory 3M Corporate Research Laboratory ----- E-mail: apjaworski at mmm.com Tel: (651) 733-6092 Fax: (651) 736-3122
apjaworski at mmm.com wrote:> I think that there is a bug in the as.POSIXct function on Windows. > > Here is what I get on Win2000, Pentium III machine in R 1.8.1. > > >>dd1 <- ISOdatetime(2003, 10, 26, 0, 59, 59) >>dd2 <- ISOdatetime(2003, 10, 26, 1, 0, 0) >>dd2 - dd1 > > Time difference of 1.000278 hours > > Now, the 26th of October was the day that change to the standard time > occurred, so I suspect that this has something to do with that. In fact > > >>dd1 > > [1] "2003-10-26 00:59:59 Central Daylight Time" > >>dd2 > > [1] "2003-10-26 01:00:00 Central Standard Time" > > so it looks like the switch from CDT to CST happens at 1:00 (instead of > 2:00 ?). >Or, it did happen at 2:00 CDT, when the time fell back one hour to 1:00 CST. 1:00 am occured twice on that day, once as CDT and once as CST. R picked the last one. A bit pathological at first glance, but date-handling often is. As for the dd2 - dd1 value, the "correct" value depends which 1:00 am was chosen. On Windows, this should be 1 hour, 1 second, no? I'm thinking 1:00 am CST == 2:00 am CDT, so in CDT entirely, your expression is basicly 02:00:00 CDT - 00:59:59 CDT. This makes me suspect that Linux picked the former 1:00 am, from your report. Since R gets its date intricacies from the OS, there really isn't much that can be done about this, until someone builds a full POSIX time implementation that takes all the world's locales and time zones into account, and welds it into R. Volunteers? It's things like this that make me convert everything to UCT (GMT, or Zulu, if you prefer). Not R's fault; stupid calendar tricks are to blame here. Cheers Jason -- Indigo Industrial Controls Ltd. http://www.indigoindustrial.co.nz 64-21-343-545 jasont at indigoindustrial.co.nz
Thanks very much for your response and the earlier one by Professor Ripley.
The general problem is indeed a tricky one.
My particular problem was much simpler. I had a bunch of data from a data
acquisition system with sampling interval of 1 minute. The system used a
simple compression scheme, where a data point was reported only when the
change in response was sufficiently large. For example, a fragment like
this
Oct. 26 0:01:00 y1
Oct. 26 0:05:00 y2
means that the values for 0:02, 0:03 0:04 where essentially y1.
I needed to "decompress" the data set, i.e., fill in the gaps, so I
was
checking for differences of 1 minute and that is when I discovered the
"error".
I am not sure what the difference between Oct. 26 0:59:00 and Oct. 26 1:00
should really be, but in this particular application it had to be 1 minute.
Otherwise I generated 60 spurious gaps between these two times.
Andy
__________________________________
Andy Jaworski
518-1-01
Process Laboratory
3M Corporate Research Laboratory
-----
E-mail: apjaworski at mmm.com
Tel: (651) 733-6092
Fax: (651) 736-3122
|---------+------------------------------->
| | Jason Turner |
| | <jasont at indigoindust|
| | rial.co.nz> |
| | |
| | 12/05/2003 02:14 |
| | |
|---------+------------------------------->
>-----------------------------------------------------------------------------------------------------------------------------|
|
|
| To: apjaworski at mmm.com
|
| cc: r-help at stat.math.ethz.ch
|
| Subject: Re: [R] bug in as.POSIXct ?
|
>-----------------------------------------------------------------------------------------------------------------------------|
apjaworski at mmm.com wrote:
> I think that there is a bug in the as.POSIXct function on Windows.
>
> Here is what I get on Win2000, Pentium III machine in R 1.8.1.
>
>
>>dd1 <- ISOdatetime(2003, 10, 26, 0, 59, 59)
>>dd2 <- ISOdatetime(2003, 10, 26, 1, 0, 0)
>>dd2 - dd1
>
> Time difference of 1.000278 hours
>
> Now, the 26th of October was the day that change to the standard time
> occurred, so I suspect that this has something to do with that. In fact
>
>
>>dd1
>
> [1] "2003-10-26 00:59:59 Central Daylight Time"
>
>>dd2
>
> [1] "2003-10-26 01:00:00 Central Standard Time"
>
> so it looks like the switch from CDT to CST happens at 1:00 (instead of
> 2:00 ?).
>
Or, it did happen at 2:00 CDT, when the time fell back one hour to 1:00
CST. 1:00 am occured twice on that day, once as CDT and once as CST. R
picked the last one. A bit pathological at first glance, but
date-handling often is.
As for the dd2 - dd1 value, the "correct" value depends which 1:00 am
was chosen. On Windows, this should be 1 hour, 1 second, no? I'm
thinking 1:00 am CST == 2:00 am CDT, so in CDT entirely, your expression
is basicly 02:00:00 CDT - 00:59:59 CDT.
This makes me suspect that Linux picked the former 1:00 am, from your
report. Since R gets its date intricacies from the OS, there really
isn't much that can be done about this, until someone builds a full
POSIX time implementation that takes all the world's locales and time
zones into account, and welds it into R. Volunteers?
It's things like this that make me convert everything to UCT (GMT, or
Zulu, if you prefer). Not R's fault; stupid calendar tricks are to
blame here.
Cheers
Jason
--
Indigo Industrial Controls Ltd.
http://www.indigoindustrial.co.nz
64-21-343-545
jasont at indigoindustrial.co.nz
Not sure but perhaps this is related to bug PR#3646 reported in: http://maths.newcastle.edu.au/~rking/R/devel/03b/0195.html --- Date: Thu, 4 Dec 2003 11:20:15 -0600 From: <apjaworski at mmm.com> To: <r-help at stat.math.ethz.ch> Subject: [R] bug in as.POSIXct ? I think that there is a bug in the as.POSIXct function on Windows. Here is what I get on Win2000, Pentium III machine in R 1.8.1.> dd1 <- ISOdatetime(2003, 10, 26, 0, 59, 59) > dd2 <- ISOdatetime(2003, 10, 26, 1, 0, 0) > dd2 - dd1Time difference of 1.000278 hours Now, the 26th of October was the day that change to the standard time occurred, so I suspect that this has something to do with that. In fact> dd1[1] "2003-10-26 00:59:59 Central Daylight Time"> dd2[1] "2003-10-26 01:00:00 Central Standard Time" so it looks like the switch from CDT to CST happens at 1:00 (instead of 2:00 ?). Since the only thing the difftime function does is unclass the as.POSIXct values of its two arguments, the error seems to be in the as.POSIXct code. I looked at the C code of the as.POSIXct function but I do not know enough about R code and C handling of the time structures to find an error there. Perhaps, the error is actually in the C library handling of time structures on Windows machines. All this seems to work fine in version 1.8.1 patched 11-24 on a Linux machine, i.e. the above difference is 1second. However, the swicth between CDT and CSD also accurs at 1:00. Andy __________________________________ Andy Jaworski 518-1-01 Process Laboratory 3M Corporate Research Laboratory ----- E-mail: apjaworski at mmm.com Tel: (651) 733-6092 Fax: (651) 736-3122
The other strategy is to use the chron package for representing times as it does not use timezones. --- Date: Thu, 4 Dec 2003 13:42:24 -0600 From: <apjaworski at mmm.com> To: Jason Turner <jasont at indigoindustrial.co.nz> Cc: <r-help at stat.math.ethz.ch> Subject: Re: [R] bug in as.POSIXct ? Thanks very much for your response and the earlier one by Professor Ripley. The general problem is indeed a tricky one. My particular problem was much simpler. I had a bunch of data from a data acquisition system with sampling interval of 1 minute. The system used a simple compression scheme, where a data point was reported only when the change in response was sufficiently large. For example, a fragment like this Oct. 26 0:01:00 y1 Oct. 26 0:05:00 y2 means that the values for 0:02, 0:03 0:04 where essentially y1. I needed to "decompress" the data set, i.e., fill in the gaps, so I was checking for differences of 1 minute and that is when I discovered the "error". I am not sure what the difference between Oct. 26 0:59:00 and Oct. 26 1:00 should really be, but in this particular application it had to be 1 minute. Otherwise I generated 60 spurious gaps between these two times. Andy __________________________________ Andy Jaworski 518-1-01 Process Laboratory 3M Corporate Research Laboratory ----- E-mail: apjaworski at mmm.com Tel: (651) 733-6092 Fax: (651) 736-3122 |---------+-------------------------------> | | Jason Turner | | | <jasont at indigoindust| | | rial.co.nz> | | | | | | 12/05/2003 02:14 | | | | |---------+------------------------------->>-----------------------------------------------------------------------------------------------------------------------------|| | | To: apjaworski at mmm.com | | cc: r-help at stat.math.ethz.ch | | Subject: Re: [R] bug in as.POSIXct ? |>-----------------------------------------------------------------------------------------------------------------------------|apjaworski at mmm.com wrote:> I think that there is a bug in the as.POSIXct function on Windows. > > Here is what I get on Win2000, Pentium III machine in R 1.8.1. > > >>dd1 <- ISOdatetime(2003, 10, 26, 0, 59, 59) >>dd2 <- ISOdatetime(2003, 10, 26, 1, 0, 0) >>dd2 - dd1 > > Time difference of 1.000278 hours > > Now, the 26th of October was the day that change to the standard time > occurred, so I suspect that this has something to do with that. In fact > > >>dd1 > > [1] "2003-10-26 00:59:59 Central Daylight Time" > >>dd2 > > [1] "2003-10-26 01:00:00 Central Standard Time" > > so it looks like the switch from CDT to CST happens at 1:00 (instead of > 2:00 ?). >Or, it did happen at 2:00 CDT, when the time fell back one hour to 1:00 CST. 1:00 am occured twice on that day, once as CDT and once as CST. R picked the last one. A bit pathological at first glance, but date-handling often is. As for the dd2 - dd1 value, the "correct" value depends which 1:00 am was chosen. On Windows, this should be 1 hour, 1 second, no? I'm thinking 1:00 am CST == 2:00 am CDT, so in CDT entirely, your expression is basicly 02:00:00 CDT - 00:59:59 CDT. This makes me suspect that Linux picked the former 1:00 am, from your report. Since R gets its date intricacies from the OS, there really isn't much that can be done about this, until someone builds a full POSIX time implementation that takes all the world's locales and time zones into account, and welds it into R. Volunteers? It's things like this that make me convert everything to UCT (GMT, or Zulu, if you prefer). Not R's fault; stupid calendar tricks are to blame here. Cheers Jason -- Indigo Industrial Controls Ltd. http://www.indigoindustrial.co.nz 64-21-343-545 jasont at indigoindustrial.co.nz