There is incompatibility between R strptime and musl libc. I posted about it on their mailing list, but they need more information I can't provide, so I'm forwarding the message here in hope R developers can help. Thanks. ---------- Forwarded message ---------- From: Rich Felker <dalias at libc.org> Date: Thu, Feb 4, 2016 at 2:07 PM Subject: Re: [musl] strptime() question To: Alba Pompeo <albapompeo at gmail.com> Cc: musl at lists.openwall.com, Martin Maechler <maechler at stat.math.ethz.ch> On Thu, Feb 04, 2016 at 01:27:40PM -0200, Alba Pompeo wrote:> Hello musl. > > I've built R on a musl system and "make check-all" fails because R > does not get correct timezone information from the system. > > Here's the error - > http://pastebin.com/raw/32D3ngNZ > > This would be correct, > > > difftime( > + as.POSIXct(c("1970-01-01 00:00:00", "1970-01-01 12:00:00"), tz="EST5EDT"), > + as.POSIXct(c("1970-01-01 00:00:00", "1970-01-01 00:00:00"), tz="UTC")) > Time differences in hours > [1] 5 17 > > but my system gives 4 16 which would suggest that it thinks that > there is only 4 hours time difference between UTC and EST5EDT and > that is just wrong. > > The above can be translated to this (slightly more low-level) R code which > is already somewhat close to the internal C code of R : > > > t0 <- strptime("1970-01-01 00:00:00", "%Y-%m-%d %H:%M:%OS", tz="UTC") > > as.numeric(t0) > [1] 0 > > > t1 <- strptime("1970-01-01 00:00:00", "%Y-%m-%d %H:%M:%OS", tz="EST5EDT") > > as.numeric(t1) > [1] 18000 > > 18000 / 3600 > [1] 5 > > > > That's why its 5 hours. > R's strptime() is close to the C library strptime(), so think musl > specialists should have it easy to do the above from C and find out > why you get 4 hour instead of 5 hour difference. > > I'm certain this isn't a problem with R and I think it's up to me to > ensure that your system's C library strptime() function returns the > correct numbers for the above example, but I can't understand why it > doesn't. > > Can anyone help here? > Thanks a lot.I think we need to understand how R's strptime function implements its tz argument, since this is not part of the POSIX strptime API. It's possible that there's a bug in musl here, but it seems equally possible that R is doing some dubious hacks with switching timezones. I'm also wondering what you're expecting from the timezone name EST5EDT, which is not generally usable; it's in POSIX TZ form rather than zoneinfo, but it lacks any information about when the switch to daylight time happens, and the defaults are not correct with respect to modern policy. Rich
On Feb 4, 2016, at 11:20 AM, Alba Pompeo <albapompeo at gmail.com> wrote:> There is incompatibility between R strptime and musl libc. I posted > about it on their mailing list, but they need more information I can't > provide, so I'm forwarding the message here in hope R developers can > help. Thanks. >Generally, it's using the standard tzset() call to set the time zone http://pubs.opengroup.org/onlinepubs/009695399/functions/tzset.html But you may want to post the full config.log and config.h somewhere since parts of the code depend on system capabilities which we don't know for musl. Cheers, Simon> > ---------- Forwarded message ---------- > From: Rich Felker <dalias at libc.org> > Date: Thu, Feb 4, 2016 at 2:07 PM > Subject: Re: [musl] strptime() question > To: Alba Pompeo <albapompeo at gmail.com> > Cc: musl at lists.openwall.com, Martin Maechler <maechler at stat.math.ethz.ch> > > > On Thu, Feb 04, 2016 at 01:27:40PM -0200, Alba Pompeo wrote: >> Hello musl. >> >> I've built R on a musl system and "make check-all" fails because R >> does not get correct timezone information from the system. >> >> Here's the error - >> http://pastebin.com/raw/32D3ngNZ >> >> This would be correct, >> >>> difftime( >> + as.POSIXct(c("1970-01-01 00:00:00", "1970-01-01 12:00:00"), tz="EST5EDT"), >> + as.POSIXct(c("1970-01-01 00:00:00", "1970-01-01 00:00:00"), tz="UTC")) >> Time differences in hours >> [1] 5 17 >> >> but my system gives 4 16 which would suggest that it thinks that >> there is only 4 hours time difference between UTC and EST5EDT and >> that is just wrong. >> >> The above can be translated to this (slightly more low-level) R code which >> is already somewhat close to the internal C code of R : >> >>> t0 <- strptime("1970-01-01 00:00:00", "%Y-%m-%d %H:%M:%OS", tz="UTC") >>> as.numeric(t0) >> [1] 0 >> >>> t1 <- strptime("1970-01-01 00:00:00", "%Y-%m-%d %H:%M:%OS", tz="EST5EDT") >>> as.numeric(t1) >> [1] 18000 >>> 18000 / 3600 >> [1] 5 >>> >> >> That's why its 5 hours. >> R's strptime() is close to the C library strptime(), so think musl >> specialists should have it easy to do the above from C and find out >> why you get 4 hour instead of 5 hour difference. >> >> I'm certain this isn't a problem with R and I think it's up to me to >> ensure that your system's C library strptime() function returns the >> correct numbers for the above example, but I can't understand why it >> doesn't. >> >> Can anyone help here? >> Thanks a lot. > > I think we need to understand how R's strptime function implements its > tz argument, since this is not part of the POSIX strptime API. It's > possible that there's a bug in musl here, but it seems equally > possible that R is doing some dubious hacks with switching timezones. > > I'm also wondering what you're expecting from the timezone name > EST5EDT, which is not generally usable; it's in POSIX TZ form rather > than zoneinfo, but it lacks any information about when the switch to > daylight time happens, and the defaults are not correct with respect > to modern policy. > > Rich > > ______________________________________________ > R-devel at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel >
On Thu, Feb 04, 2016 at 04:35:22PM -0200, Alba Pompeo wrote:> I forwarded our talk on musl mailing list to R mailing list. > I got this response. > Does it help? > Also, what do you think about making a new email that's sent to both > musl and R mailing list, that way devs from both projects can > communicate? It's a tad inefficient for me to keep forwarding messages > like this back and forth.I'm cc'ing both lists now.> ---------- Forwarded message ---------- > From: Simon Urbanek <simon.urbanek at r-project.org> > Date: Thu, Feb 4, 2016 at 3:01 PM > Subject: Re: [Rd] [musl] strptime() question > To: Alba Pompeo <albapompeo at gmail.com> > Cc: r-devel <r-devel at r-project.org> > > > > On Feb 4, 2016, at 11:20 AM, Alba Pompeo <albapompeo at gmail.com> wrote: > > > There is incompatibility between R strptime and musl libc. I posted > > about it on their mailing list, but they need more information I can't > > provide, so I'm forwarding the message here in hope R developers can > > help. Thanks. > > Generally, it's using the standard tzset() call to set the time zone > http://pubs.opengroup.org/onlinepubs/009695399/functions/tzset.htmlThe tzset function doesn't really do much interesting; it just updates some global state to match the current value of the TZ environment variable, but most time.h functions do this already anyway. Anyway this doesn't really answer the question of what R's strptime function is doing to add timezone functionality onto the underlying system's strptime. Is R changing the environment (this is highly unsafe in any potentially-multithreaded process, BTW)? Is it using the globals set by tzset to compute adjustments to the result? Rich
It is setting TZ and using tzset(). R is not multi-threaded so it is safe. Simon figure out the important settings from the config.log and config.h files on a musl system: /* #undef USE_INTERNAL_MKTIME */ #define HAVE_TM_GMTOFF 1 #define HAVE_TM_ZONE 1 Does this help anyone debug the issue? Simon just went on vacation. On Thu, Feb 4, 2016 at 7:32 PM, Rich Felker <dalias at libc.org> wrote:> On Thu, Feb 04, 2016 at 04:35:22PM -0200, Alba Pompeo wrote: >> I forwarded our talk on musl mailing list to R mailing list. >> I got this response. >> Does it help? >> Also, what do you think about making a new email that's sent to both >> musl and R mailing list, that way devs from both projects can >> communicate? It's a tad inefficient for me to keep forwarding messages >> like this back and forth. > > I'm cc'ing both lists now. > >> ---------- Forwarded message ---------- >> From: Simon Urbanek <simon.urbanek at r-project.org> >> Date: Thu, Feb 4, 2016 at 3:01 PM >> Subject: Re: [Rd] [musl] strptime() question >> To: Alba Pompeo <albapompeo at gmail.com> >> Cc: r-devel <r-devel at r-project.org> >> >> >> >> On Feb 4, 2016, at 11:20 AM, Alba Pompeo <albapompeo at gmail.com> wrote: >> >> > There is incompatibility between R strptime and musl libc. I posted >> > about it on their mailing list, but they need more information I can't >> > provide, so I'm forwarding the message here in hope R developers can >> > help. Thanks. >> >> Generally, it's using the standard tzset() call to set the time zone >> http://pubs.opengroup.org/onlinepubs/009695399/functions/tzset.html > > The tzset function doesn't really do much interesting; it just updates > some global state to match the current value of the TZ environment > variable, but most time.h functions do this already anyway. Anyway > this doesn't really answer the question of what R's strptime function > is doing to add timezone functionality onto the underlying system's > strptime. Is R changing the environment (this is highly unsafe in any > potentially-multithreaded process, BTW)? Is it using the globals set > by tzset to compute adjustments to the result? > > Rich