Very succinct, Rui! One warning to Diego.... automatic data recorders tend to use the local standard timezone year-round. R by default assumes that timestamps converted from character to POSIXct using the current timezone on your computer... which may not be in the same zone that the logger was in but even more commonly the computer follows daylight savings time. This leads to NAs showing up in your converted timestamps in spring and duplicated values in autumn as the data are misinterpreted. The easiest solution can be to use Sys.setenv( TZ="GMT" ) though if you need the actual timezone you can use a zone name of the form "Etc/GMT+5" (5 hrs west of GMT). Note that Rui's solution will only work correctly near the month transition if you pretend the data timezone is GMT or UTC. (Technically these are different so your mileage may vary but most implementations treat them as identical and I have not encountered any cases where they differ.) On January 27, 2019 10:03:44 AM PST, Rui Barradas <ruipbarradas at sapo.pt> wrote:>Hello, > >See if the following can get you started. >It uses package CRAN zoo, function as.yearmon. > >dati$MES <- zoo::as.yearmon(dati$DATAORA) >PMES <- ave(dati$PREC, dati$MES, FUN = cumsum) > >plot(dati$DATAORA, PMES) > > >Hope this helps, > >Rui Barradas > >?s 15:25 de 27/01/2019, Diego Avesani escreveu: >> Dear all, >> >> I have a set of data with has hourly value: >> >> # ID >> # Lo >> # L >> # Q >> Time, T, RH,PSFC,DIR,VEL10, PREC, RAD, CC,FOG >> yyyy-mm-dd hh:mm, ?C, %, hPa, ?N, m/s, mm/h,W/m?, %,- >> 2012-01-01 06:00, -0.1,100, 815,313, 2.6, 0.0, 0, 0,0 >> 2012-01-01 07:00, -1.2, 93, 814,314, 4.8, 0.0, 0, 0,0 >> 2012-01-01 08:00, 1.7, 68, 815,308, 7.5, 0.0, 41, 11,0 >> 2012-01-01 09:00, 2.4, 65, 815,308, 7.4, 0.0, 150, 33,0 >> ..... >> ..... >> >> I was able to read it, create my-own data frame and to plot the >total >> cumulative function. >> This is basically what I have done: >> >> dati <- read.csv(file="116.txt", header=FALSE, sep="," , >> na.strings="-999",skip = 6) >> colnames(dati)=c("DATAORA","T", "RH","PSFC","DIR","VEL10", "PREC", >"RAD", >> "CC","FOG") >> >> dati$DATAORA<-as.POSIXct(strptime(dati$DATAORA,format="%Y-%m-%d >%H:%M")) >> >> >> P <- cumsum(dati$PREC) >> plot(dati$DATAORA, P) >> >> I would like to select the data according to an starting and ending >date. >> In addition, I would like to plot the monthly and not the total one. >> I mean, I would like to have a cumulative plot for each month of the >> selected year. >> >> I am struggling with "ddply" but probably it is the wrong way. >> >> Could someone help me? Really Really thanks, >> >> >> Diego >> >> [[alternative HTML version deleted]] >> >> ______________________________________________ >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide >http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. >> > >______________________________________________ >R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see >https://stat.ethz.ch/mailman/listinfo/r-help >PLEASE do read the posting guide >http://www.R-project.org/posting-guide.html >and provide commented, minimal, self-contained, reproducible code.-- Sent from my phone. Please excuse my brevity.
Dear Jeff, Dear Rui, Dear all, I will try Rui's solution as soon as possible. If I could ask: As a first step, I would like to follow Jeff's suggestion. I will represent the precipitation data with a cumulative distribution, one for each year. This follow that I would like to select the starting date and the ending date properly form dati$DATA in order to perform the cumulative function. Could you help me on that. Again, really really thanks Diego On Sun, 27 Jan 2019 at 21:37, Jeff Newmiller <jdnewmil at dcn.davis.ca.us> wrote:> Very succinct, Rui! > > One warning to Diego.... automatic data recorders tend to use the local > standard timezone year-round. R by default assumes that timestamps > converted from character to POSIXct using the current timezone on your > computer... which may not be in the same zone that the logger was in but > even more commonly the computer follows daylight savings time. This leads > to NAs showing up in your converted timestamps in spring and duplicated > values in autumn as the data are misinterpreted. The easiest solution can > be to use > > Sys.setenv( TZ="GMT" ) > > though if you need the actual timezone you can use a zone name of the form > "Etc/GMT+5" (5 hrs west of GMT). > > Note that Rui's solution will only work correctly near the month > transition if you pretend the data timezone is GMT or UTC. (Technically > these are different so your mileage may vary but most implementations treat > them as identical and I have not encountered any cases where they differ.) > > On January 27, 2019 10:03:44 AM PST, Rui Barradas <ruipbarradas at sapo.pt> > wrote: > >Hello, > > > >See if the following can get you started. > >It uses package CRAN zoo, function as.yearmon. > > > >dati$MES <- zoo::as.yearmon(dati$DATAORA) > >PMES <- ave(dati$PREC, dati$MES, FUN = cumsum) > > > >plot(dati$DATAORA, PMES) > > > > > >Hope this helps, > > > >Rui Barradas > > > >?s 15:25 de 27/01/2019, Diego Avesani escreveu: > >> Dear all, > >> > >> I have a set of data with has hourly value: > >> > >> # ID > >> # Lo > >> # L > >> # Q > >> Time, T, RH,PSFC,DIR,VEL10, PREC, RAD, CC,FOG > >> yyyy-mm-dd hh:mm, ?C, %, hPa, ?N, m/s, mm/h,W/m?, %,- > >> 2012-01-01 06:00, -0.1,100, 815,313, 2.6, 0.0, 0, 0,0 > >> 2012-01-01 07:00, -1.2, 93, 814,314, 4.8, 0.0, 0, 0,0 > >> 2012-01-01 08:00, 1.7, 68, 815,308, 7.5, 0.0, 41, 11,0 > >> 2012-01-01 09:00, 2.4, 65, 815,308, 7.4, 0.0, 150, 33,0 > >> ..... > >> ..... > >> > >> I was able to read it, create my-own data frame and to plot the > >total > >> cumulative function. > >> This is basically what I have done: > >> > >> dati <- read.csv(file="116.txt", header=FALSE, sep="," , > >> na.strings="-999",skip = 6) > >> colnames(dati)=c("DATAORA","T", "RH","PSFC","DIR","VEL10", "PREC", > >"RAD", > >> "CC","FOG") > >> > >> dati$DATAORA<-as.POSIXct(strptime(dati$DATAORA,format="%Y-%m-%d > >%H:%M")) > >> > >> > >> P <- cumsum(dati$PREC) > >> plot(dati$DATAORA, P) > >> > >> I would like to select the data according to an starting and ending > >date. > >> In addition, I would like to plot the monthly and not the total one. > >> I mean, I would like to have a cumulative plot for each month of the > >> selected year. > >> > >> I am struggling with "ddply" but probably it is the wrong way. > >> > >> Could someone help me? Really Really thanks, > >> > >> > >> Diego > >> > >> [[alternative HTML version deleted]] > >> > >> ______________________________________________ > >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > >> https://stat.ethz.ch/mailman/listinfo/r-help > >> PLEASE do read the posting guide > >http://www.R-project.org/posting-guide.html > >> and provide commented, minimal, self-contained, reproducible code. > >> > > > >______________________________________________ > >R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see > >https://stat.ethz.ch/mailman/listinfo/r-help > >PLEASE do read the posting guide > >http://www.R-project.org/posting-guide.html > >and provide commented, minimal, self-contained, reproducible code. > > -- > Sent from my phone. Please excuse my brevity. >[[alternative HTML version deleted]]
I have no idea what you mean when you say "select starting date and ending
date properly form [sic] datai$DATA". For one thing there is no column
called DATA, and for another I don't know what starting dates and ending
dates you might be interested in. If you need help to subset by time,
perhaps you should ask a question about that instead.
Here is a reproducible example of making monthly data and manipulating it
using artificial data:
###############
library(zoo)
Sys.setenv( TZ = "GMT" )
set.seed(42)
dati <- data.frame( DATAORA = as.POSIXct( "2012-01-01" )
+ as.difftime( seq( 0, 365*3*24
), units="hours" )
)
# terrible simulation of precipitation
dati$PREC <- 0.1 * trunc( 50 * rbeta( nrow( dati ), 1, 80 ) )
dati$ym <- as.yearmon( dati$DATAORA )
# aggregate usually reduces the number of rows given to it
datim <- aggregate( list( PREC = dati$PREC ) # data to summarize
, dati[ , "ym", drop=FALSE ] # columns to group on
, FUN = sum # calculation on data
)
plot(PREC ~ ym, data=datim) # This is how I would usually look at it
as.year <- function(x) floor( as.numeric( x ) ) # from help file on
as.yearmon
datim$y <- as.year( datim$ym )
# ave typically does not change the number of rows given to it
datim$PMES <- ave( datim$PREC, datim$y, FUN = cumsum)
plot(PMES ~ ym, data=datim) # My guess as to what you asked for?
###############
On Sun, 27 Jan 2019, Diego Avesani wrote:
> Dear? Jeff, Dear Rui, Dear all,
>
> I will try Rui's solution as soon as possible.
> If I could ask:
> As a first step, I would like to follow Jeff's suggestion. I will
represent the precipitation data with a cumulative
> distribution, one for each year.
> This follow that I would like to select the starting date and the ending
date properly form dati$DATA in order to
> perform the cumulative function.
>
> Could you help me on that.
>
> Again, really really thanks
>
> Diego
>
>
>
> On Sun, 27 Jan 2019 at 21:37, Jeff Newmiller <jdnewmil at
dcn.davis.ca.us> wrote:
> Very succinct, Rui!
>
> One warning to Diego.... automatic data recorders tend to use the
local standard timezone year-round. R by
> default assumes that timestamps converted from character to POSIXct
using the current timezone on your
> computer... which may not be in the same zone that the logger was in
but even more commonly the computer
> follows daylight savings time. This leads to NAs showing up in your
converted timestamps in spring and
> duplicated values in autumn as the data are misinterpreted. The
easiest solution can be to use
>
> Sys.setenv( TZ="GMT" )
>
> though if you need the actual timezone you can use a zone name of the
form "Etc/GMT+5" (5 hrs west of GMT).
>
> Note that Rui's solution will only work correctly near the month
transition if you pretend the data timezone
> is GMT or UTC. (Technically these are different so your mileage may
vary but most implementations treat them
> as identical and I have not encountered any cases where they differ.)
>
> On January 27, 2019 10:03:44 AM PST, Rui Barradas <ruipbarradas at
sapo.pt> wrote:
> >Hello,
> >
> >See if the following can get you started.
> >It uses package CRAN zoo, function as.yearmon.
> >
> >dati$MES <- zoo::as.yearmon(dati$DATAORA)
> >PMES <- ave(dati$PREC, dati$MES, FUN = cumsum)
> >
> >plot(dati$DATAORA, PMES)
> >
> >
> >Hope this helps,
> >
> >Rui Barradas
> >
> >?s 15:25 de 27/01/2019, Diego Avesani escreveu:
> >> Dear all,
> >>
> >> I have a set of data with has hourly value:
> >>
> >> # ID
> >> # Lo
> >> # L
> >> # Q
> >> Time,? ? T, RH,PSFC,DIR,VEL10, PREC, RAD, CC,FOG
> >> yyyy-mm-dd hh:mm,? ??C,? %, hPa, ?N,? m/s, mm/h,W/m?,? %,-
> >> 2012-01-01 06:00, -0.1,100, 815,313,? 2.6,? 0.0,? ?0,? 0,0
> >> 2012-01-01 07:00, -1.2, 93, 814,314,? 4.8,? 0.0,? ?0,? 0,0
> >> 2012-01-01 08:00,? 1.7, 68, 815,308,? 7.5,? 0.0,? 41, 11,0
> >> 2012-01-01 09:00,? 2.4, 65, 815,308,? 7.4,? 0.0, 150, 33,0
> >> .....
> >> .....
> >>
> >> I was able to read it,? create my-own data frame and to plot
the
> >total
> >> cumulative function.
> >> This is basically what I have done:
> >>
> >> dati <- read.csv(file="116.txt", header=FALSE,
sep="," ,
> >> na.strings="-999",skip = 6)
> >> colnames(dati)=c("DATAORA","T",
"RH","PSFC","DIR","VEL10",
"PREC",
> >"RAD",
> >> "CC","FOG")
> >>
> >>
dati$DATAORA<-as.POSIXct(strptime(dati$DATAORA,format="%Y-%m-%d
> >%H:%M"))
> >>
> >>
> >> P <- cumsum(dati$PREC)
> >> plot(dati$DATAORA, P)
> >>
> >> I would like to select the data according to an starting and
ending
> >date.
> >> In addition, I would like to plot the monthly and not the
total one.
> >> I mean, I would like to have a cumulative plot for each
month of the
> >> selected year.
> >>
> >> I am struggling with "ddply" but probably it is
the wrong way.
> >>
> >> Could someone help me?? Really Really thanks,
> >>
> >>
> >> Diego
> >>
> >>? ? ? [[alternative HTML version deleted]]
> >>
> >> ______________________________________________
> >> R-help at r-project.org mailing list -- To UNSUBSCRIBE and
more, see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible
code.
> >>
> >
> >______________________________________________
> >R-help at r-project.org mailing list -- To UNSUBSCRIBE and more,
see
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible
code.
>
> --
> Sent from my phone. Please excuse my brevity.
>
>
>
---------------------------------------------------------------------------
Jeff Newmiller The ..... ..... Go Live...
DCN:<jdnewmil at dcn.davis.ca.us> Basics: ##.#. ##.#. Live
Go...
Live: OO#.. Dead: OO#.. Playing
Research Engineer (Solar/Batteries O.O#. #.O#. with
/Software/Embedded Controllers) .OO#. .OO#. rocks...1k
---------------------------------------------------------------------------