Timothy W. Hilton
2008-Mar-13 22:47 UTC
[R] zoo object: replace NAs from another zoo object
Hello, I have two zoo objects, new and old, indexed by chron objects. Their structure is like this: (05/25/06 00:00:00) NA NA NA (05/25/06 00:02:00) 948.20 24.198 0 (05/25/06 00:04:00) 948.26 20.640 0 (05/25/06 00:06:00) 948.37 19.653 0 (05/25/06 00:08:00) 948.48 19.135 0 (05/25/06 00:10:00) 948.54 22.510 0 (05/25/06 00:12:00) 948.55 27.175 0 (05/25/06 00:14:00) 948.46 30.075 0 (05/25/06 00:16:00) 948.59 28.815 0 (05/25/06 00:18:00) 948.96 27.938 0 old spans a larger range of dates than new. I would like to take all the values in new that are not NA and place them in old, matching the time indices. I have tried a number of variations on the themes of merge and replace, but can't get them to work, and the brute force for loop solution is unnacceptably slow. Could someone suggest a solution? Many thanks, Tim
Gabor Grothendieck
2008-Mar-14 00:55 UTC
[R] zoo object: replace NAs from another zoo object
An example that included input and output would help. I'll take a guess but if that's not it please clarify and provide that. Also its helpful if you can provide the data in easily copied form using dput(z). I will assume that time(New) is a superset of time(Old) and "put them in" means replace the values in Old with those from New so that Old should be the same shape and have the same times before and after.> library(zoo) > Old <- zoo(matrix(1:10, 5)) > New <- zoo(matrix(c(NA, 12:16), 3)) > New.omit <- na.omit(New) > window(Old, time(New.omit)) <- New.omit > Old1 1 6 2 12 15 3 13 16 4 4 9 5 5 10 On Thu, Mar 13, 2008 at 6:47 PM, Timothy W. Hilton <twh142 at psu.edu> wrote:> Hello, > > I have two zoo objects, new and old, indexed by chron objects. Their structure is like this: > > (05/25/06 00:00:00) NA NA NA > (05/25/06 00:02:00) 948.20 24.198 0 > (05/25/06 00:04:00) 948.26 20.640 0 > (05/25/06 00:06:00) 948.37 19.653 0 > (05/25/06 00:08:00) 948.48 19.135 0 > (05/25/06 00:10:00) 948.54 22.510 0 > (05/25/06 00:12:00) 948.55 27.175 0 > (05/25/06 00:14:00) 948.46 30.075 0 > (05/25/06 00:16:00) 948.59 28.815 0 > (05/25/06 00:18:00) 948.96 27.938 0 > > old spans a larger range of dates than new. I would like to take all the values in new that are not NA and place them in old, matching the time indices. I have tried a number of variations on the themes of merge and replace, but can't get them to work, and the brute force for loop solution is unnacceptably slow. Could someone suggest a solution? > > Many thanks, > Tim > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >
Timothy W. Hilton
2008-Mar-14 10:16 UTC
[R] zoo object: replace NAs from another zoo object
Gabor, Many thanks for your suggestion. This solution is very close, but not exactly what I had in mind, because na.omit removes the entire row in which an NA appears. I want to replace values in old with non-NA values from corresponding times in new. If a timestamp from old does not appear in new, or appears but the new value is NA, then I want to keep the old value. That is, I want to write mergeOldNew(old, new) to get the following result:> newP PAR pcp (05/20/06 00:00:00) 956.23 252.30 NA (05/20/06 00:02:00) NA 244.85 NA (05/20/06 00:04:00) NA 236.45 NA> oldP PAR pcp (05/19/06 23:58:00) NA NA NA (05/20/06 00:00:00) NA NA NA (05/20/06 00:02:00) 955.68 311.54 0 (05/20/06 00:04:00) 955.70 320.74 0 (05/20/06 00:06:00) 955.68 309.16 0 (05/20/06 00:08:00) 955.69 301.72 0> mergeOldNew(old, new)(05/19/06 23:58:00) NA NA NA (05/20/06 00:00:00) 956.23 252.30 NA (05/20/06 00:02:00) 955.68 244.85 0 (05/20/06 00:04:00) 955.70 236.45 0 (05/20/06 00:06:00) 955.68 309.16 0 (05/20/06 00:08:00) 955.69 301.72 0 Looping your suggestion over the individual columns in old and new works, but I wonder if there is a more elegant solution to avoid the loop? Many thanks for your help. dput output for old and new are below. -Tim =======> dput(old)structure(c(NA, NA, 955.68, 955.7, 955.68, 955.69, NA, NA, 311.54, 320.74, 309.16, 301.72, NA, NA, 0, 0, 0, 0), .Dim = c(6L, 3L), .Dimnames = list( c("13679", "13680", "13681", "13682", "13683", "13684"), c("P", "PAR", "pcp")), index = structure(c(13287.9986111111, 13288, 13288.0013888889, 13288.0027777778, 13288.0041666667, 13288.0055555556), format = structure(c("m/d/y", "h:m:s"), .Names = c("dates", "times")), origin = c(1, 1, 1970), class = c("chron", "dates", "times")), class = "zoo")> dput(new)structure(c(956.23, NA, NA, 252.3, 244.85, 236.45, NA, NA, NA ), .Dim = c(3L, 3L), .Dimnames = list(c("1", "2", "3"), c("P", "PAR", "pcp")), index = structure(c(13288, 13288.0013888889, 13288.0027777778), format = structure(c("m/d/y", "h:m:s"), .Names = c("dates", "times")), origin = c(1, 1, 1970), class = c("chron", "dates", "times")), class = "zoo") ========
Gabor Grothendieck
2008-Mar-14 11:30 UTC
[R] zoo object: replace NAs from another zoo object
Try this:> window(old, time(new)) <- ifelse(is.na(new), old[time(new)], new) > oldP PAR pcp (05/19/06 23:58:00) NA NA NA (05/20/06 00:00:00) 956.23 252.30 NA (05/20/06 00:02:00) 955.68 244.85 0 (05/20/06 00:04:00) 955.70 236.45 0 (05/20/06 00:06:00) 955.68 309.16 0 (05/20/06 00:08:00) 955.69 301.72 0 On Fri, Mar 14, 2008 at 6:16 AM, Timothy W. Hilton <hilton at meteo.psu.edu> wrote:> Gabor, > > Many thanks for your suggestion. This solution is very close, but not exactly what I had in mind, because na.omit removes the entire row in which an NA appears. I want to replace values in old with non-NA values from corresponding times in new. If a timestamp from old does not appear in new, or appears but the new value is NA, then I want to keep the old value. That is, I want to write mergeOldNew(old, new) to get the following result: > > > new > P PAR pcp > (05/20/06 00:00:00) 956.23 252.30 NA > (05/20/06 00:02:00) NA 244.85 NA > (05/20/06 00:04:00) NA 236.45 NA > > > old > P PAR pcp > (05/19/06 23:58:00) NA NA NA > (05/20/06 00:00:00) NA NA NA > (05/20/06 00:02:00) 955.68 311.54 0 > (05/20/06 00:04:00) 955.70 320.74 0 > (05/20/06 00:06:00) 955.68 309.16 0 > (05/20/06 00:08:00) 955.69 301.72 0 > > > mergeOldNew(old, new) > > (05/19/06 23:58:00) NA NA NA > (05/20/06 00:00:00) 956.23 252.30 NA > (05/20/06 00:02:00) 955.68 244.85 0 > (05/20/06 00:04:00) 955.70 236.45 0 > (05/20/06 00:06:00) 955.68 309.16 0 > (05/20/06 00:08:00) 955.69 301.72 0 > > Looping your suggestion over the individual columns in old and new works, but I wonder if there is a more elegant solution to avoid the loop? > > Many thanks for your help. dput output for old and new are below. > > -Tim > > =======> > > dput(old) > structure(c(NA, NA, 955.68, 955.7, 955.68, 955.69, NA, NA, 311.54, > 320.74, 309.16, 301.72, NA, NA, 0, 0, 0, 0), .Dim = c(6L, 3L), .Dimnames = list( > c("13679", "13680", "13681", "13682", "13683", "13684"), > c("P", "PAR", "pcp")), index = structure(c(13287.9986111111, > 13288, 13288.0013888889, 13288.0027777778, 13288.0041666667, > 13288.0055555556), format = structure(c("m/d/y", "h:m:s"), .Names = c("dates", > "times")), origin = c(1, 1, 1970), class = c("chron", "dates", > "times")), class = "zoo") > > > dput(new) > structure(c(956.23, NA, NA, 252.3, 244.85, 236.45, NA, NA, NA > ), .Dim = c(3L, 3L), .Dimnames = list(c("1", "2", "3"), c("P", > "PAR", "pcp")), index = structure(c(13288, 13288.0013888889, > 13288.0027777778), format = structure(c("m/d/y", "h:m:s"), .Names = c("dates", > "times")), origin = c(1, 1, 1970), class = c("chron", "dates", > "times")), class = "zoo") > > =======> >