R help - May 2011 - Averaging uneven measurements by time with uneven numbers of measurements

I have a new device that takes measurements anywhere from every second, to
every 15 minutes (depending on changes). The matrix has a date, time and Y
column (Y is the measurement). For three days it is 25,000 rows. How do I
average the measurements by every 30 minutes so my matrix is 48 rows per
day? I have been working on this and cannot figure out a simple method. Any
ideas? Thank you.

-----
In theory, practice and theory are the same. In practice, they are not - Albert
Einstein
--
View this message in context:
http://r.789695.n4.nabble.com/Averaging-uneven-measurements-by-time-with-uneven-numbers-of-measurements-tp3499337p3499337.html
Sent from the R help mailing list archive at Nabble.com.

I'd be tempted to do a robust fit (loess?) to the data with a 
relatively small span (I'm assuming that there are errors in the 
measurements and some degree of smoothing is acceptable) then 
predict the fit at a regular interval (e.g., every 30 minutes).

-- 
Clint Bowman			INTERNET:	clint at ecy.wa.gov
Air Quality Modeler		INTERNET:	clint at math.utah.edu
Department of Ecology		VOICE:		(360) 407-6815
PO Box 47600			FAX:		(360) 407-7534
Olympia, WA 98504-7600


         USPS:           PO Box 47600, Olympia, WA 98504-7600
         Parcels:        300 Desmond Drive, Lacey, WA 98503-1274


On Thu, 5 May 2011, Schatzi wrote:
> I have a new device that takes measurements anywhere from every second, to
> every 15 minutes (depending on changes). The matrix has a date, time and Y
> column (Y is the measurement). For three days it is 25,000 rows. How do I
> average the measurements by every 30 minutes so my matrix is 48 rows per
> day? I have been working on this and cannot figure out a simple method. Any
> ideas? Thank you.
>
> -----
> In theory, practice and theory are the same. In practice, they are not -
Albert Einstein
> --
> View this message in context:
http://r.789695.n4.nabble.com/Averaging-uneven-measurements-by-time-with-uneven-numbers-of-measurements-tp3499337p3499337.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

I do not want smoothing as the data should have jumps (it is weight left in
feeding bunker). I was thinking of maybe using a histogram-like function and
then averaging that. Not sure if this is possible.

-----
In theory, practice and theory are the same. In practice, they are not - Albert
Einstein
--
View this message in context:
http://r.789695.n4.nabble.com/Averaging-uneven-measurements-by-time-with-uneven-numbers-of-measurements-tp3499337p3499386.html
Sent from the R help mailing list archive at Nabble.com.

On 2011-05-05 14:20, Schatzi wrote:
> I do not want smoothing as the data should have jumps (it is weight left in
> feeding bunker). I was thinking of maybe using a histogram-like function
and
> then averaging that. Not sure if this is possible.
(It would be useful to include your original request - not everyone
uses Nabble.)

Actually, averaging *is* smoothing, but I suppose your intent is, for
some reason, not to smooth across 30-minute boundaries.

Perhaps you could use findInterval() to identify which measurements to
average.

Peter Ehlers
>
> -----
> In theory, practice and theory are the same. In practice, they are not -
Albert Einstein
> --
> View this message in context:
http://r.789695.n4.nabble.com/Averaging-uneven-measurements-by-time-with-uneven-numbers-of-measurements-tp3499337p3499386.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

Taking the final value for each 30-minute interval seems like it would get what
I want. The problem is that sometimes this value would be 15 minutes before the
end of the 30-minute interval. What would I use to pick up this value?

-----Original Message-----
From: ehlers at ucalgary.ca [mailto:ehlers at ucalgary.ca] 
Sent: Thursday, May 05, 2011 03:58 PM
To: Thompson, Adele - Adele_Thompson at cargill.com
Cc: r-help at r-project.org
Subject: Re: [R] Averaging uneven measurements by time with uneven numbers of
measurements

On 2011-05-05 14:20, Schatzi wrote:
> I do not want smoothing as the data should have jumps (it is weight left in
> feeding bunker). I was thinking of maybe using a histogram-like function
and
> then averaging that. Not sure if this is possible.
(It would be useful to include your original request - not everyone
uses Nabble.)

Actually, averaging *is* smoothing, but I suppose your intent is, for
some reason, not to smooth across 30-minute boundaries.

Perhaps you could use findInterval() to identify which measurements to
average.

Peter Ehlers
>
> -----
> In theory, practice and theory are the same. In practice, they are not -
Albert Einstein
> --
> View this message in context:
http://r.789695.n4.nabble.com/Averaging-uneven-measurements-by-time-with-uneven-numbers-of-measurements-tp3499337p3499386.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

Here is an example of what I would like to do:
meas = measurements 
times = time of measurement
measf = measurements in final, reduced matrix
timesf = time of measurement in final matrix

meas<-runif(30)
times<-sort(runif(30))
inputmat<-cbind(times,meas)
names(inputmat)<-c("timef","measf")

I would then like to break the times up into 0.2 increments so the final matrix
would look like this:
timef measf
0.2  <mean of meas where (time >=0) and (time<0.2)>
0.4
.
.
.
1.0

Instead of measf being the mean, it could be the last measurement taken.


-----Original Message-----
From: ehlers at ucalgary.ca [mailto:ehlers at ucalgary.ca] 
Sent: Thursday, May 05, 2011 03:58 PM
To: Thompson, Adele - Adele_Thompson at cargill.com
Cc: r-help at r-project.org
Subject: Re: [R] Averaging uneven measurements by time with uneven numbers of
measurements

On 2011-05-05 14:20, Schatzi wrote:
> I do not want smoothing as the data should have jumps (it is weight left in
> feeding bunker). I was thinking of maybe using a histogram-like function
and
> then averaging that. Not sure if this is possible.
(It would be useful to include your original request - not everyone
uses Nabble.)

Actually, averaging *is* smoothing, but I suppose your intent is, for
some reason, not to smooth across 30-minute boundaries.

Perhaps you could use findInterval() to identify which measurements to
average.

Peter Ehlers
>
> -----
> In theory, practice and theory are the same. In practice, they are not -
Albert Einstein
> --
> View this message in context:
http://r.789695.n4.nabble.com/Averaging-uneven-measurements-by-time-with-uneven-numbers-of-measurements-tp3499337p3499386.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

I figured out a poor way to do what I want.
meas<-runif(30)
times<-sort(runif(30))

timesdec<-seq(0,1,0.2)
ltim<-length(timesdec)
storing<-rep(0,ltim)

for (i in 1:ltim) {
if (i=1) {rowstart=1} else {rowstart<-findInterval(timesdec[i-1],times)+1}
rowfinal<-findInterval(timesdec[i],times)
storing[i]<-mean(meas[rowstart:rowfinal])
} #end i-for loop


-----Original Message-----
From: ehlers at ucalgary.ca [mailto:ehlers at ucalgary.ca] 
Sent: Thursday, May 05, 2011 03:58 PM
To: Thompson, Adele - Adele_Thompson at cargill.com
Cc: r-help at r-project.org
Subject: Re: [R] Averaging uneven measurements by time with uneven numbers of
measurements

On 2011-05-05 14:20, Schatzi wrote:
> I do not want smoothing as the data should have jumps (it is weight left in
> feeding bunker). I was thinking of maybe using a histogram-like function
and
> then averaging that. Not sure if this is possible.
(It would be useful to include your original request - not everyone
uses Nabble.)

Actually, averaging *is* smoothing, but I suppose your intent is, for
some reason, not to smooth across 30-minute boundaries.

Perhaps you could use findInterval() to identify which measurements to
average.

Peter Ehlers
>
> -----
> In theory, practice and theory are the same. In practice, they are not -
Albert Einstein
> --
> View this message in context:
http://r.789695.n4.nabble.com/Averaging-uneven-measurements-by-time-with-uneven-numbers-of-measurements-tp3499337p3499386.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.