Hello: I have an issue with filtering in a time series of animal growth data
that seems conceptually simple but I have not come up with effective code to
implement this. I have temporal sequences of lengths by individuals and I want
to retain only those data that are >10 days apart sequentially within an
individuals records. I can readily compute intervals between successive dates
by individual using data.table() and its by = INDIVIDUAL functionality. See
example data for one individual below. But what currently eludes me in
processing this is how to recognize for example that deleting the 2nd and 3rd
rows is required because the totality of their time interval is 9 days, deleting
8th record with 4 days is required, deleting 17th record with 1 day is
required, deleting 22nd and 23rd records is required because their sum is 2
days, but we do not delete 24th record of 10 days because the sum of previous 2
records deleted and this one is now 12 days. Each individual can have very
different patterns of these sort of sequences. These sequences are easy to look
at and determine what needs to be done but writing effective code to accomplish
this filtering seems to require some functionality that I am currently missing.
Any suggestions would be greatly appreciated.
Date INDIVIDUAL DATENUMBER LENGTH length.prev interval
228 12-May-04 57084544 133 682.4 NA NA
229 28-Sep-04 57084544 272 724.8 682.4 139
230 30-Sep-04 57084544 274 740.8 724.8 2
231 7-Oct-04 57084544 281 745.4 740.8 7
232 22-Nov-04 57084544 327 780.2 745.4 46
233 27-Jan-05 57084544 393 817.2 780.2 66
234 8-Mar-05 57084544 433 834.1 817.2 40
235 2-Jul-05 57084544 549 876.3 834.1 116
236 6-Jul-05 57084544 553 871.5 876.3 4
237 4-Aug-05 57084544 582 887.5 871.5 29
238 28-Dec-05 57084544 728 921.8 887.5 146
239 31-Jan-06 57084544 762 936.8 921.8 34
240 27-Feb-06 57084544 789 962.4 936.8 27
241 21-Nov-06 57084544 1056 972.3 962.4 267
242 30-Mar-07 57084544 1185 1007.2 972.3 129
243 23-Apr-07 57084544 1209 1009.1 1007.2 24
244 22-May-07 57084544 1238 991.6 1009.1 29
245 23-May-07 57084544 1239 1015.9 991.6 1
246 16-Jul-07 57084544 1293 1006.5 1015.9 54
247 9-Aug-07 57084544 1317 1013.0 1006.5 24
248 27-Aug-07 57084544 1335 1013.0 1013.0 18
249 29-Jul-08 57084544 1672 1021.5 1013.0 337
250 30-Jul-08 57084544 1673 984.3 1021.5 1
251 31-Jul-08 57084544 1674 1008.5 984.3 1
252 10-Aug-08 57084544 1684 1002.8 1008.5 10
253 22-Oct-08 57084544 1757 977.6 1002.8 73
254 2-Dec-08 57084544 1798 1000.6 977.6 41
Brian
Brian S. Cade, PhD
U. S. Geological Survey
Fort Collins Science Center
2150 Centre Ave., Bldg. C
Fort Collins, CO 80526-8818
email: cadeb at usgs.gov<mailto:brian_cade at usgs.gov>
tel: 970 226-9326
[[alternative HTML version deleted]]
Like this?
winnow <- function(x, int=5){
keep <- x[1]
remaining <- x[-1]
while (length(remaining))
{
nxt <- tail(keep,1) + int
if(length(remaining) ==1 ||
all(remaining < nxt))break
remaining <- remaining[remaining >tail(keep,1) + int]
keep <- c(keep,remaining[1])
}
keep
}
> x
[1] 1 2 5 7 8 9 15 16 17 19 20 21 28 35 37 41 43 45 46
50> winnow(x,7)
[1] 1 9 17 28 37 45> winnow(x,5)
[1] 1 7 15 21 28 35 41 50
Cheers,
Bert
"The trouble with having an open mind is that people keep coming along
and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )
On Mon, Apr 4, 2022 at 12:56 PM Cade, Brian S via R-help
<r-help at r-project.org> wrote:>
> Hello: I have an issue with filtering in a time series of animal growth
data that seems conceptually simple but I have not come up with effective code
to implement this. I have temporal sequences of lengths by individuals and I
want to retain only those data that are >10 days apart sequentially within an
individuals records. I can readily compute intervals between successive dates
by individual using data.table() and its by = INDIVIDUAL functionality. See
example data for one individual below. But what currently eludes me in
processing this is how to recognize for example that deleting the 2nd and 3rd
rows is required because the totality of their time interval is 9 days, deleting
8th record with 4 days is required, deleting 17th record with 1 day is
required, deleting 22nd and 23rd records is required because their sum is 2
days, but we do not delete 24th record of 10 days because the sum of previous 2
records deleted and this one is now 12 days. Each individual can have very
> different patterns of these sort of sequences. These sequences are easy
to look at and determine what needs to be done but writing effective code to
accomplish this filtering seems to require some functionality that I am
currently missing.
>
> Any suggestions would be greatly appreciated.
>
> Date INDIVIDUAL DATENUMBER LENGTH length.prev interval
> 228 12-May-04 57084544 133 682.4 NA NA
> 229 28-Sep-04 57084544 272 724.8 682.4 139
> 230 30-Sep-04 57084544 274 740.8 724.8 2
> 231 7-Oct-04 57084544 281 745.4 740.8 7
> 232 22-Nov-04 57084544 327 780.2 745.4 46
> 233 27-Jan-05 57084544 393 817.2 780.2 66
> 234 8-Mar-05 57084544 433 834.1 817.2 40
> 235 2-Jul-05 57084544 549 876.3 834.1 116
> 236 6-Jul-05 57084544 553 871.5 876.3 4
> 237 4-Aug-05 57084544 582 887.5 871.5 29
> 238 28-Dec-05 57084544 728 921.8 887.5 146
> 239 31-Jan-06 57084544 762 936.8 921.8 34
> 240 27-Feb-06 57084544 789 962.4 936.8 27
> 241 21-Nov-06 57084544 1056 972.3 962.4 267
> 242 30-Mar-07 57084544 1185 1007.2 972.3 129
> 243 23-Apr-07 57084544 1209 1009.1 1007.2 24
> 244 22-May-07 57084544 1238 991.6 1009.1 29
> 245 23-May-07 57084544 1239 1015.9 991.6 1
> 246 16-Jul-07 57084544 1293 1006.5 1015.9 54
> 247 9-Aug-07 57084544 1317 1013.0 1006.5 24
> 248 27-Aug-07 57084544 1335 1013.0 1013.0 18
> 249 29-Jul-08 57084544 1672 1021.5 1013.0 337
> 250 30-Jul-08 57084544 1673 984.3 1021.5 1
> 251 31-Jul-08 57084544 1674 1008.5 984.3 1
> 252 10-Aug-08 57084544 1684 1002.8 1008.5 10
> 253 22-Oct-08 57084544 1757 977.6 1002.8 73
> 254 2-Dec-08 57084544 1798 1000.6 977.6 41
>
>
>
> Brian
>
>
>
> Brian S. Cade, PhD
>
> U. S. Geological Survey
> Fort Collins Science Center
> 2150 Centre Ave., Bldg. C
> Fort Collins, CO 80526-8818
>
> email: cadeb at usgs.gov<mailto:brian_cade at usgs.gov>
> tel: 970 226-9326
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.