R help - Jan 2020 - Incorrect Conversion of Datetime

Dear Friends,
A sample of my data is:
98 05 01 02    8541
98 05 01 03    8548
98 05 01 04    8512
98 05 01 05    8541
98 05 01 06    8509
98 05 01 07    8472
98 05 01 08    8454
98 05 01 09    8461
98 05 01 10    8462
98 05 01 11    8475
98 05 01 12    8433
98 05 01 13    8479
98 05 01 14    8417
98 05 01 15    8463
98 05 01 16    8473
98 05 01 17    8450
98 05 01 18    8433
98 05 01 19    8437
98 05 01 20    8437
98 05 01 21    8438
98 05 01 22    8421
98 05 01 23    8420
98 05 02 00    8371
98 05 02 01    8338
98 05 02 02    8251
98 05 02 03    8204
98 05 02 04    8183
98 05 02 05    8231
98 05 02 06    8242
Columns 1, 2, 3, 4 and 5 stands for year, month, day , hour and count.

Using:
Sys.setenv( TZ="GMT" )


dta <- read.table("Ohr1may98", col.names = c("year",
"month", "day",
"hour", "counts"))
dta$year <- with( dta, ifelse(year < 50, year + 2000, year + 1900))
dta$datetime <- with( dta, as.POSIXct(ISOdatetime(year, month,day,hour,0,0)))
a =  dta$datetime
I converted the datetime and plotted the graph of count vs a. The plot
was great but I have issues with the date.

The raw data is for some hours for Ist and second day of may 1998 as
is evident from the sample data. But the result of date stored in "a"
above shows:
> a
 [1] "1998-01-05 02:00:00 GMT" "1998-01-05 03:00:00 GMT"
 [3] "1998-01-05 04:00:00 GMT" "1998-01-05 05:00:00 GMT"
 [5] "1998-01-05 06:00:00 GMT" "1998-01-05 07:00:00 GMT"
 [7] "1998-01-05 08:00:00 GMT" "1998-01-05 09:00:00 GMT"
 [9] "1998-01-05 10:00:00 GMT" "1998-01-05 11:00:00 GMT"
[11] "1998-01-05 12:00:00 GMT" "1998-01-05 13:00:00 GMT"
[13] "1998-01-05 14:00:00 GMT" "1998-01-05 15:00:00 GMT"
[15] "1998-01-05 16:00:00 GMT" "1998-01-05 17:00:00 GMT"
[17] "1998-01-05 18:00:00 GMT" "1998-01-05 19:00:00 GMT"
[19] "1998-01-05 20:00:00 GMT" "1998-01-05 21:00:00 GMT"
[21] "1998-01-05 22:00:00 GMT" "1998-01-05 23:00:00 GMT"
[23] "1998-01-06 00:00:00 GMT" "1998-01-06 01:00:00 GMT"
[25] "1998-01-06 02:00:00 GMT" "1998-01-06 03:00:00 GMT"
[27] "1998-01-06 04:00:00 GMT" "1998-01-06 05:00:00 GMT"
[29] "1998-01-06 06:00:00 GMT"
This seems to suggest day 5 and 6 in January 1998 instead of day 1 and
2 in May of 1998.

I have spent some time trying to resolve this but I have not been successful.

I would be thankful if you could help me to check where I went astray.

Thank you.
Best wishes
Ogbos

Hi Ogbos,
I get the correct result using strptime:

dta$date<-strptime(paste(dta$year,dta$month,dta$day,dta$hour),
 "%y %m %d %H"
)
> dta$date
[1] "1998-05-01 02:00:00 AEST" "1998-05-01 03:00:00 AEST"
[3] "1998-05-01 04:00:00 AEST" "1998-05-01 05:00:00 AEST"
[5] "1998-05-01 06:00:00 AEST" "1998-05-01 07:00:00 AEST"
[7] "1998-05-01 08:00:00 AEST" "1998-05-01 09:00:00 AEST"
[9] "1998-05-01 10:00:00 AEST" "1998-05-01 11:00:00 AEST"
[11] "1998-05-01 12:00:00 AEST" "1998-05-01 13:00:00 AEST"
[13] "1998-05-01 14:00:00 AEST" "1998-05-01 15:00:00 AEST"
[15] "1998-05-01 16:00:00 AEST" "1998-05-01 17:00:00 AEST"
[17] "1998-05-01 18:00:00 AEST" "1998-05-01 19:00:00 AEST"
[19] "1998-05-01 20:00:00 AEST" "1998-05-01 21:00:00 AEST"
[21] "1998-05-01 22:00:00 AEST" "1998-05-01 23:00:00 AEST"
[23] "1998-05-02 00:00:00 AEST" "1998-05-02 01:00:00 AEST"
[25] "1998-05-02 02:00:00 AEST" "1998-05-02 03:00:00 AEST"
[27] "1998-05-02 04:00:00 AEST" "1998-05-02 05:00:00 AEST"
[29] "1998-05-02 06:00:00 AEST"

Same result with

ISOdatetime(dta$year,dta$month,dta$day,dta$hour,0,0,tz="GMT")
as.POSIXct(ISOdatetime(dta$year,dta$month,dta$day,dta$hour,0,0,tz="GMT"))

As it should be as the lower two call strptime. I can't see from your
code why the month and day are transcribed.

Jim

Hi again,
Small typo, should be

dta$date<-strptime(paste(dta$year,dta$month,dta$day,dta$hour),
 "%Y %m %d %H"

as I tried it both with and without the century just to check and
copied the wrong line.

On Wed, Jan 8, 2020 at 9:03 PM Jim Lemon <drjimlemon at gmail.com>
wrote:
>
> Hi Ogbos,
> I get the correct result using strptime:
>
> dta$date<-strptime(paste(dta$year,dta$month,dta$day,dta$hour),
>  "%y %m %d %H"
> )
> > dta$date
> [1] "1998-05-01 02:00:00 AEST" "1998-05-01 03:00:00
AEST"
> [3] "1998-05-01 04:00:00 AEST" "1998-05-01 05:00:00
AEST"
> [5] "1998-05-01 06:00:00 AEST" "1998-05-01 07:00:00
AEST"
> [7] "1998-05-01 08:00:00 AEST" "1998-05-01 09:00:00
AEST"
> [9] "1998-05-01 10:00:00 AEST" "1998-05-01 11:00:00
AEST"
> [11] "1998-05-01 12:00:00 AEST" "1998-05-01 13:00:00
AEST"
> [13] "1998-05-01 14:00:00 AEST" "1998-05-01 15:00:00
AEST"
> [15] "1998-05-01 16:00:00 AEST" "1998-05-01 17:00:00
AEST"
> [17] "1998-05-01 18:00:00 AEST" "1998-05-01 19:00:00
AEST"
> [19] "1998-05-01 20:00:00 AEST" "1998-05-01 21:00:00
AEST"
> [21] "1998-05-01 22:00:00 AEST" "1998-05-01 23:00:00
AEST"
> [23] "1998-05-02 00:00:00 AEST" "1998-05-02 01:00:00
AEST"
> [25] "1998-05-02 02:00:00 AEST" "1998-05-02 03:00:00
AEST"
> [27] "1998-05-02 04:00:00 AEST" "1998-05-02 05:00:00
AEST"
> [29] "1998-05-02 06:00:00 AEST"
>
> Same result with
>
> ISOdatetime(dta$year,dta$month,dta$day,dta$hour,0,0,tz="GMT")
>
as.POSIXct(ISOdatetime(dta$year,dta$month,dta$day,dta$hour,0,0,tz="GMT"))
>
> As it should be as the lower two call strptime. I can't see from your
> code why the month and day are transcribed.
>
> Jim

Quoting Ogbos Okike <giftedlife2014 at gmail.com>:
> Dear Friends,
> A sample of my data is:
> 98 05 01 02    8541
> 98 05 01 03    8548
> 98 05 01 04    8512
> 98 05 01 05    8541
> 98 05 01 06    8509
> 98 05 01 07    8472
> 98 05 01 08    8454
> 98 05 01 09    8461
> 98 05 01 10    8462
> 98 05 01 11    8475
> 98 05 01 12    8433
> 98 05 01 13    8479
> 98 05 01 14    8417
> 98 05 01 15    8463
> 98 05 01 16    8473
> 98 05 01 17    8450
> 98 05 01 18    8433
> 98 05 01 19    8437
> 98 05 01 20    8437
> 98 05 01 21    8438
> 98 05 01 22    8421
> 98 05 01 23    8420
> 98 05 02 00    8371
> 98 05 02 01    8338
> 98 05 02 02    8251
> 98 05 02 03    8204
> 98 05 02 04    8183
> 98 05 02 05    8231
> 98 05 02 06    8242
> Columns 1, 2, 3, 4 and 5 stands for year, month, day , hour and count.
>
> Using:
> Sys.setenv( TZ="GMT" )
>
>
> dta <- read.table("Ohr1may98", col.names = c("year",
"month", "day",
> "hour", "counts"))
> dta$year <- with( dta, ifelse(year < 50, year + 2000, year + 1900))
> dta$datetime <- with( dta, as.POSIXct(ISOdatetime(year,
month,day,hour,0,0)))
> a =  dta$datetime
> I converted the datetime and plotted the graph of count vs a. The plot
> was great but I have issues with the date.
>
> The raw data is for some hours for Ist and second day of may 1998 as
> is evident from the sample data. But the result of date stored in
"a"
> above shows:
>> a
>  [1] "1998-01-05 02:00:00 GMT" "1998-01-05 03:00:00
GMT"
>  [3] "1998-01-05 04:00:00 GMT" "1998-01-05 05:00:00
GMT"
>  [5] "1998-01-05 06:00:00 GMT" "1998-01-05 07:00:00
GMT"
>  [7] "1998-01-05 08:00:00 GMT" "1998-01-05 09:00:00
GMT"
>  [9] "1998-01-05 10:00:00 GMT" "1998-01-05 11:00:00
GMT"
> [11] "1998-01-05 12:00:00 GMT" "1998-01-05 13:00:00
GMT"
> [13] "1998-01-05 14:00:00 GMT" "1998-01-05 15:00:00
GMT"
> [15] "1998-01-05 16:00:00 GMT" "1998-01-05 17:00:00
GMT"
> [17] "1998-01-05 18:00:00 GMT" "1998-01-05 19:00:00
GMT"
> [19] "1998-01-05 20:00:00 GMT" "1998-01-05 21:00:00
GMT"
> [21] "1998-01-05 22:00:00 GMT" "1998-01-05 23:00:00
GMT"
> [23] "1998-01-06 00:00:00 GMT" "1998-01-06 01:00:00
GMT"
> [25] "1998-01-06 02:00:00 GMT" "1998-01-06 03:00:00
GMT"
> [27] "1998-01-06 04:00:00 GMT" "1998-01-06 05:00:00
GMT"
> [29] "1998-01-06 06:00:00 GMT"
> This seems to suggest day 5 and 6 in January 1998 instead of day 1 and
> 2 in May of 1998.
>
> I have spent some time trying to resolve this but I have not been
successful.
>
> I would be thankful if you could help me to check where I went astray.
>
> Thank you.
> Best wishes
> Ogbos
>
I cannot reproduce these results. Could you please provide a fully
reproducible example, by providing a small example dataset via
'dput(dta)'?


-- 
Enrico Schumann
Lucerne, Switzerland
http://enricoschumann.net

Dear Enrico,
Thanks for your time.
I have tried to learn how to use dput in R. I have not yet made much progress.

I have succeeded in using dput to store my data frame. I first
converted my data into a data frame and then used:
dput(dd,file="Ogbos2",control = c("keepNA",
"keepInteger",
"showAttributes")) to output the dput file. dd is my data frame.

When I opened the file, I didn't like its content as it differs very
much from my data frame. But I don't know whether that makes sense to
you. I am attaching the file.
I am thanking you in advance for additional suggestions.
Best wishes
Ogbos

On Wed, Jan 8, 2020 at 1:07 PM Enrico Schumann <es at enricoschumann.net>
wrote:
>
>
> Quoting Ogbos Okike <giftedlife2014 at gmail.com>:
>
> > Dear Friends,
> > A sample of my data is:
> > 98 05 01 02    8541
> > 98 05 01 03    8548
> > 98 05 01 04    8512
> > 98 05 01 05    8541
> > 98 05 01 06    8509
> > 98 05 01 07    8472
> > 98 05 01 08    8454
> > 98 05 01 09    8461
> > 98 05 01 10    8462
> > 98 05 01 11    8475
> > 98 05 01 12    8433
> > 98 05 01 13    8479
> > 98 05 01 14    8417
> > 98 05 01 15    8463
> > 98 05 01 16    8473
> > 98 05 01 17    8450
> > 98 05 01 18    8433
> > 98 05 01 19    8437
> > 98 05 01 20    8437
> > 98 05 01 21    8438
> > 98 05 01 22    8421
> > 98 05 01 23    8420
> > 98 05 02 00    8371
> > 98 05 02 01    8338
> > 98 05 02 02    8251
> > 98 05 02 03    8204
> > 98 05 02 04    8183
> > 98 05 02 05    8231
> > 98 05 02 06    8242
> > Columns 1, 2, 3, 4 and 5 stands for year, month, day , hour and count.
> >
> > Using:
> > Sys.setenv( TZ="GMT" )
> >
> >
> > dta <- read.table("Ohr1may98", col.names =
c("year", "month", "day",
> > "hour", "counts"))
> > dta$year <- with( dta, ifelse(year < 50, year + 2000, year +
1900))
> > dta$datetime <- with( dta, as.POSIXct(ISOdatetime(year,
month,day,hour,0,0)))
> > a =  dta$datetime
> > I converted the datetime and plotted the graph of count vs a. The plot
> > was great but I have issues with the date.
> >
> > The raw data is for some hours for Ist and second day of may 1998 as
> > is evident from the sample data. But the result of date stored in
"a"
> > above shows:
> >> a
> >  [1] "1998-01-05 02:00:00 GMT" "1998-01-05 03:00:00
GMT"
> >  [3] "1998-01-05 04:00:00 GMT" "1998-01-05 05:00:00
GMT"
> >  [5] "1998-01-05 06:00:00 GMT" "1998-01-05 07:00:00
GMT"
> >  [7] "1998-01-05 08:00:00 GMT" "1998-01-05 09:00:00
GMT"
> >  [9] "1998-01-05 10:00:00 GMT" "1998-01-05 11:00:00
GMT"
> > [11] "1998-01-05 12:00:00 GMT" "1998-01-05 13:00:00
GMT"
> > [13] "1998-01-05 14:00:00 GMT" "1998-01-05 15:00:00
GMT"
> > [15] "1998-01-05 16:00:00 GMT" "1998-01-05 17:00:00
GMT"
> > [17] "1998-01-05 18:00:00 GMT" "1998-01-05 19:00:00
GMT"
> > [19] "1998-01-05 20:00:00 GMT" "1998-01-05 21:00:00
GMT"
> > [21] "1998-01-05 22:00:00 GMT" "1998-01-05 23:00:00
GMT"
> > [23] "1998-01-06 00:00:00 GMT" "1998-01-06 01:00:00
GMT"
> > [25] "1998-01-06 02:00:00 GMT" "1998-01-06 03:00:00
GMT"
> > [27] "1998-01-06 04:00:00 GMT" "1998-01-06 05:00:00
GMT"
> > [29] "1998-01-06 06:00:00 GMT"
> > This seems to suggest day 5 and 6 in January 1998 instead of day 1 and
> > 2 in May of 1998.
> >
> > I have spent some time trying to resolve this but I have not been
successful.
> >
> > I would be thankful if you could help me to check where I went astray.
> >
> > Thank you.
> > Best wishes
> > Ogbos
> >
>
> I cannot reproduce these results. Could you please provide a fully
> reproducible example, by providing a small example dataset via
'dput(dta)'?
>
>
> --
> Enrico Schumann
> Lucerne, Switzerland
> http://enricoschumann.net
>