Nathan Miller
2011-Oct-20 03:38 UTC
[R] Applying function with separate dataframe (calibration file) supplying some inputs
Hello,
I am not entirely sure the subject line captures what I am trying to do, but
hopefully this description of the problem will help folks to see my
challenge and hopefully offer constructive assistance.
I have an experimental setup where I measure the decrease in oxygen in small
vials as an organism, such as an oyster, consumes the oxygen. Each vial is
calibrated before the experiment and these calibrations are used to convert
the raw data after the experiment into oxygen values. I end up with two
dataframes. One has the calibration data and for example could look like
this
vial cal0 T0 cal100 T100
1 61 18 28 18
2 60.8 18 27.1 18
3 60.2 18 28.3 18
4 59.8 18 27.2 18
The second is a data file which could look like this
vial phase temp time
1 31 17.5 10
1 31.5 17.4 20
1 32.8 17.5 30
2 29.0 17.5 10
2 29.7 17.5 20
2 30.9 17.5 30
3 27.1 17.4 10
3 27.6 17.4 20
3 28.1 17.5 30
4 31.0 17.6 10
4 33.3 17.6 20
4 35.6 17.6 30
I have a complicated function (included at the bottom) that uses the
calibration values and the raw data to calculate actual oxygen levels. It
works great, but as its currently written it requires that each calibration
be entered individually. I would rather apply the function based upon the
vial number (applying the calibration for vial 1 to all vial 1 data,
calibration for vial 2 to all vial 2 data).
I have managed to do this by combining the two dataframes into one that
looks like this
data
vial phase temp time cal0 T0 cal100 T100
1 31 17.5 10 61 18 28 18
1 31.5 17.4 20 61 18 28 18
1 32.8 17.5 30 61 18 28 18
1 33.6 17.5 40 61 18 28 18
2 29.0 17.5 10 60.8 18 27.1 18
2 29.7 17.5 20 60.8 18 27.1 18
2 30.9 17.5 30 60.8 18 27.1 18
3 27.1 17.4 10 60.2 18 28.3 18
3 27.6 17.4 20 60.2 18 28.3 18
3 28.1 17.5 30 60.2 18 28.3 18
4 31.0 17.6 10 59.8 18 27.2 18
4 33.3 17.6 20 59.8 18 27.2 18
4 35.6 17.6 30 59.8 18 27.2 18
I have then used ddply to apply my function grouped by "vial"
oxygen<-ddply(data,.(vial), function(d) optode(d$cal0, d$T0, d$cal100,
d$T100, d$phase, d$temp))
This works, but I do not like having to put the calibrations into the same
dataframe as the data. Can anyone show me an example of how I could have a
function reference a dataframe (like the calibration data) based upon a
grouping variable (like "vial") and use that data as partial inputs to
the
function when it is applied to another dataframe (the actual data)? I don't
necessarily need an example using my sample data or the function I have
included here (as it is rather unwieldly). A simplified example of how to
achieve this type of cross referencing would suffice and I can apply the
principle to my current problem.
Thanks so much,
Nate
optode<-function(cal0,T0,cal100,T100,phase,temp) {
f1=0.801
deltaPsiK=-0.08
deltaKsvK=0.000383
m=22.9
tan0T100=tan(((cal0+deltaPsiK*(T100-T0)))*pi/180)
tan0Tm=tan((cal0+(deltaPsiK*(temp-T0)))*pi/180)
tan100T100=tan(cal100*pi/180)
tanmTm=tan(phase*pi/180)
A=tan100T100/tan0T100*1/m*100^2
B=tan100T100/tan0T100*100+tan100T100/tan0T100*1/m*100-f1*1/m*100-100+f1*100
C=tan100T100/tan0T100-1
KsvT100=(-B+(sqrt(B^2-4*A*C)))/(2*A)
KsvTm=KsvT100+(deltaKsvK*(temp-T100))
a=tanmTm/tan0Tm*1/m*KsvTm^2
b=tanmTm/tan0Tm*KsvTm+tanmTm/tan0Tm*1/m*KsvTm-f1*1/m*KsvTm-KsvTm+f1*KsvTm
c=tanmTm/tan0Tm-1
saturation=(-((tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*(KsvT100+(deltaKsvK*(temp-T100)))+(tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*1/m*(KsvT100+(deltaKsvK*(temp-T100)))-f1*1/m*(KsvT100+(deltaKsvK*(temp-T100)))-(KsvT100+(deltaKsvK*(temp-T100)))+f1*(KsvT100+(deltaKsvK*(temp-T100))))+(sqrt((((tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*(KsvT100+(deltaKsvK*(temp-T100)))+(tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*1/m*(KsvT100+(deltaKsvK*(temp-T100)))-f1*1/m*(KsvT100+(deltaKsvK*(temp-T100)))-(KsvT100+(deltaKsvK*(temp-T100)))+f1*(KsvT100+(deltaKsvK*(temp-T100)))))^2-4*((tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*1/m*(KsvT100+(deltaKsvK*(temp-T100)))^2)*((tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))-1))))/(2*((tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*1/m*(KsvT100+(deltaKsvK*(temp-T100)))^2))
print(saturation)
}
[[alternative HTML version deleted]]
Joshua Wiley
2011-Oct-20 05:33 UTC
[R] Applying function with separate dataframe (calibration file) supplying some inputs
Hi Nathan,
I honestly do not think that anything else will be much better than
merging the two datasets. If the datasets are not merged, you
essentially have to apply your optode function to each vial, store the
results, then combine them all together. This is innefficient.
Merging the datasets may be innefficient in a way, but once it is
done, your function can be applied to the entire dataset in one step.
If you have big data and the merge is slow, take a look at the
data.table package. I have recently (not that it was bad before, I
just never really knew how much it could do) been quite impressed with
it. As a whole other note, your optode function was quite difficult
to read, and I highly doubt you can confidently look at the code and
ensure there is not a typo, missed operator, etc. somewhere in that
block of formula code. I attempted to clean it up some, though
perhaps not with 100% success.
#######################################
optode2 <- function(cal0, T0, cal100, T100, phase, temp) {
dr <- pi/180
f1 <- 0.801
deltaPsiK <- (-0.08)
deltaKsvK <- 0.000383
m <- 22.9
tan0T100 <- tan(((cal0 + deltaPsiK * (T100 - T0))) * dr)
tan0Tm <- tan((cal0 + (deltaPsiK * (temp - T0))) * dr)
tan100T100 <- tan(cal100 * dr)
tanmTm <- tan(phase * dr)
A <- tan100T100 / tan0T100 / m * 100^2
B <- tan100T100 / tan0T100 * 100 + tan100T100 / tan0T100 / m *100 -
f1 / m * 100 - 100 + f1 * 100
C <- tan100T100 / tan0T100 - 1
KsvT100 <- (- B + (sqrt(B^2 - 4 * A * C))) / (2 * A)
KsvTm <- KsvT100 + (deltaKsvK * (temp - T100))
a <- tanmTm / tan0Tm / m * KsvTm^2
b <- tanmTm / tan0Tm * KsvTm + tanmTm / tan0Tm / m * KsvTm - f1 / m
* KsvTm - KsvTm + f1 * KsvTm
c <- tanmTm / tan0Tm - 1
tot <- tanmTm / tan0T100
big <- tot * KsvTm + tanmTm / tan0T100 / m * KsvTm - f1 / m * KsvTm
- KsvTm + f1 * KsvTm
saturation <- (-big + (sqrt((big)^2-4 * (tanmTm / tan0T100 / m *
KsvTm^2) * (tot - 1)))) / (2 * (tot / m * KsvTm^2))
return(saturation)
}
## Read in your example data
d1 <- read.table(textConnection("
vial cal0 T0 cal100 T100
1 61 18 28 18
2 60.8 18 27.1 18
3 60.2 18 28.3 18
4 59.8 18 27.2 18"), header = TRUE, stringsAsFactors =
FALSE)
d2 <- read.table(textConnection("
vial phase temp time
1 31 17.5 10
1 31.5 17.4 20
1 32.8 17.5 30
2 29.0 17.5 10
2 29.7 17.5 20
2 30.9 17.5 30
3 27.1 17.4 10
3 27.6 17.4 20
3 28.1 17.5 30
4 31.0 17.6 10
4 33.3 17.6 20
4 35.6 17.6 30"), header = TRUE, stringsAsFactors =
FALSE)
closeAllConnections()
dat <- merge(d1, d2, by = "vial")
## optode wrapper
f <- function(d) optode2(d$cal0, d$T0, d$cal100, d$T100, d$phase, d$temp)
dat$oxygen <- f(dat)
dat
#######################################
Cheers,
Josh
On Wed, Oct 19, 2011 at 8:38 PM, Nathan Miller <natemiller77 at gmail.com>
wrote:> Hello,
>
> I am not entirely sure the subject line captures what I am trying to do,
but
> hopefully this description of the problem will help folks to see my
> challenge and hopefully offer constructive assistance.
>
> I have an experimental setup where I measure the decrease in oxygen in
small
> vials as an organism, such as an oyster, consumes the oxygen. Each vial is
> calibrated before the experiment and these calibrations are used to convert
> the raw data after the experiment into oxygen values. I end up with two
> dataframes. One has the calibration data and for example could look like
> this
>
> vial cal0 ? ?T0 ?cal100 ?T100
> 1 ? ? ?61 ? ? ?18 ? ?28 ? ? ? 18
> 2 ? ? ?60.8 ? ?18 ? ?27.1 ? ?18
> 3 ? ? 60.2 ? ?18 ? ?28.3 ? ? 18
> 4 ? ? 59.8 ? ?18 ? ? 27.2 ? ? 18
>
> The second is a data file which could look like this
>
>
> vial ? phase ? ?temp ? time
> 1 ? ? ? 31 ? ? ? ? ? ?17.5 ? ?10
> 1 ? ? ? 31.5 ? ? ? ? ?17.4 ? ?20
> 1 ? ? ? 32.8 ? ? ? ? ?17.5 ? ?30
> 2 ? ? ?29.0 ? ? ? ? ? 17.5 ? ? 10
> 2 ? ? ?29.7 ? ? ? ? ? 17.5 ? ? 20
> 2 ? ? ?30.9 ? ? ? ? ? 17.5 ? ? 30
> 3 ? ? ?27.1 ? ? ? ? ? 17.4 ? ? 10
> 3 ? ? ?27.6 ? ? ? ? ? 17.4 ? ? 20
> 3 ? ? ?28.1 ? ? ? ? ? 17.5 ? ? 30
> 4 ? ? ?31.0 ? ? ? ? ? 17.6 ? ? 10
> 4 ? ? ?33.3 ? ? ? ? ? 17.6 ? ? 20
> 4 ? ? 35.6 ? ? ? ? ? ?17.6 ? ? 30
>
> I have a complicated function (included at the bottom) that uses the
> calibration values and the raw data to calculate actual oxygen levels. It
> works great, but as its currently written it requires that each calibration
> be entered individually. I would rather apply the function based upon the
> vial number (applying the calibration for vial 1 to all vial 1 data,
> calibration for vial 2 to all vial 2 data).
>
> I have managed to do this by combining the two dataframes into one that
> looks like this
>
> data
> vial ? phase ? ?temp ? time ? cal0 ? ? T0 ? ?cal100 ?T100
> 1 ? ? ? 31 ? ? ? ? ? ?17.5 ? ?10 ? ? 61 ? ? ? 18 ? ? 28 ? ? ? ?18
> 1 ? ? ? 31.5 ? ? ? ? ?17.4 ? ?20 ? ?61 ? ? ? 18 ? ? 28 ? ? ? ?18
> 1 ? ? ? 32.8 ? ? ? ? ?17.5 ? ?30 ? ?61 ? ? ? 18 ? ? 28 ? ? ? ?18
> 1 ? ? ? 33.6 ? ? ? ? ?17.5 ? ? 40 ? 61 ? ? ? 18 ? ? 28 ? ? ? ?18
> 2 ? ? ?29.0 ? ? ? ? ? 17.5 ? ? 10 ? 60.8 ? ?18 ? ?27.1 ? ? ?18
> 2 ? ? ?29.7 ? ? ? ? ? 17.5 ? ? 20 ? 60.8 ? ?18 ? ?27.1 ? ? ?18
> 2 ? ? ?30.9 ? ? ? ? ? 17.5 ? ? 30 ? 60.8 ? ?18 ? ?27.1 ? ? ?18
> 3 ? ? ?27.1 ? ? ? ? ? 17.4 ? ? 10 ? 60.2 ? ?18 ? ?28.3 ? ? ?18
> 3 ? ? ?27.6 ? ? ? ? ? 17.4 ? ? 20 ? 60.2 ? ?18 ? ?28.3 ? ? ?18
> 3 ? ? ?28.1 ? ? ? ? ? 17.5 ? ? 30 ? 60.2 ? ?18 ? ?28.3 ? ? ?18
> 4 ? ? ?31.0 ? ? ? ? ? 17.6 ? ? 10 ? 59.8 ? ?18 ? ? 27.2 ? ? 18
> 4 ? ? ?33.3 ? ? ? ? ? 17.6 ? ? 20 ? 59.8 ? ?18 ? ? 27.2 ? ? 18
> 4 ? ? 35.6 ? ? ? ? ? ?17.6 ? ? 30 ? 59.8 ? ?18 ? ? 27.2 ? ? 18
>
> I have then used ddply to apply my function grouped by "vial"
>
> oxygen<-ddply(data,.(vial), function(d) optode(d$cal0, d$T0, d$cal100,
> d$T100, d$phase, d$temp))
>
> This works, but I do not like having to put the calibrations into the same
> dataframe as the data. Can anyone show me an example of how I could have a
> function reference a dataframe (like the calibration data) based upon a
> grouping variable (like "vial") and use that data as partial
inputs to the
> function when it is applied to another dataframe (the actual data)? I
don't
> necessarily need an example using my sample data or the function I have
> included here (as it is rather unwieldly). A simplified example of how to
> achieve this type of cross referencing would suffice and I can apply the
> principle to my current problem.
>
> Thanks so much,
> Nate
>
>
> optode<-function(cal0,T0,cal100,T100,phase,temp) {
>
> ? ?f1=0.801
> ? ?deltaPsiK=-0.08
> ? ?deltaKsvK=0.000383
> ? ?m=22.9
> ? ?tan0T100=tan(((cal0+deltaPsiK*(T100-T0)))*pi/180)
> ? ?tan0Tm=tan((cal0+(deltaPsiK*(temp-T0)))*pi/180)
> ? ?tan100T100=tan(cal100*pi/180)
> ? ?tanmTm=tan(phase*pi/180)
> ? ?A=tan100T100/tan0T100*1/m*100^2
>
> B=tan100T100/tan0T100*100+tan100T100/tan0T100*1/m*100-f1*1/m*100-100+f1*100
> ? ?C=tan100T100/tan0T100-1
> ? ?KsvT100=(-B+(sqrt(B^2-4*A*C)))/(2*A)
> ? ?KsvTm=KsvT100+(deltaKsvK*(temp-T100))
> ? ?a=tanmTm/tan0Tm*1/m*KsvTm^2
>
> b=tanmTm/tan0Tm*KsvTm+tanmTm/tan0Tm*1/m*KsvTm-f1*1/m*KsvTm-KsvTm+f1*KsvTm
> ? ?c=tanmTm/tan0Tm-1
>
>
saturation=(-((tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*(KsvT100+(deltaKsvK*(temp-T100)))+(tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*1/m*(KsvT100+(deltaKsvK*(temp-T100)))-f1*1/m*(KsvT100+(deltaKsvK*(temp-T100)))-(KsvT100+(deltaKsvK*(temp-T100)))+f1*(KsvT100+(deltaKsvK*(temp-T100))))+(sqrt((((tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*(KsvT100+(deltaKsvK*(temp-T100)))+(tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*1/m*(KsvT100+(deltaKsvK*(temp-T100)))-f1*1/m*(KsvT100+(deltaKsvK*(temp-T100)))-(KsvT100+(deltaKsvK*(temp-T100)))+f1*(KsvT100+(deltaKsvK*(temp-T100)))))^2-4*((tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*1/m*(KsvT100+(deltaKsvK*(temp-T100)))^2)*((tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))-1))))/(2*((tan(phase*pi/180))/(tan((cal0+(deltaPsiK*(temp-T0)))*pi/180))*1/m*(KsvT100+(deltaKsvK*(temp-T100)))^2))
>
> ? ?print(saturation)
>
> }
>
> ? ? ? ?[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
--
Joshua Wiley
Ph.D. Student, Health Psychology
Programmer Analyst II, ATS Statistical Consulting Group
University of California, Los Angeles
https://joshuawiley.com/