Hi there,
I ran the following code:
vols=read.csv(file="C:/Documents and Settings/Hugh/My
Documents/PhD/Swaption
vols.csv"
, header=TRUE, sep=",")
X<-ts(vols[,2])
#X
dcOU<-function(x,t,x0,theta,log=FALSE){
Ex<-theta[1]/theta[2]+(x0-theta[1]/theta[2])*exp(-theta[2]*t)
Vx<-theta[3]^2*(1-exp(-2*theta[2]*t))/(2*theta[2])
dnorm(x,mean=Ex,sd=sqrt(Vx),log=log)
}
OU.lik<-function(theta1,theta2,theta3){
n<-length(X)
dt<-deltat(X)
-sum(dcOU(X[2:n],dt,X[1:(n-1)],c(theta1,theta2,theta3),log=TRUE))
}
require(stats4)
require(sde)
set.seed(1)
#X<-sde.sim(model="OU",theta=c(3,1,2),N=10000,delta=1)
mle(OU.lik,start=list(theta1=1,theta2=1,theta3=1),
method="L-BFGS-B",lower=c(-Inf,-Inf,-Inf),upper=c(Inf,Inf,Inf))->fit
summary(fit)
#ex3.01 R
prof<-profile(fit)
par(mfrow=c(1,3))
plot(prof)
par(mfrow=c(1,1))
vcov(fit)
I run the code above and I get:
> summary(fit)
Maximum likelihood estimation
Call:
mle(minuslogl = OU.lik, start = list(theta1 = 1, theta2 = 1,
theta3 = 1), method = "L-BFGS-B", lower = c(-Inf, -Inf, -Inf),
upper = c(Inf, Inf, Inf))
Coefficients:
Estimate Std. Error
theta1 0.03595581 0.013929892
theta2 4.30910365 1.663781710
theta3 0.02120220 0.004067477
-2 log L: -5136.327
I need to run the same analysis for 40 different time series.
I want to be able to collate all the estimates of theta and the associated
stadard errors and then transfer them into excel?
Can someone please point me to some R code that will allow me to do this?
Thanks
--
View this message in context:
http://r.789695.n4.nabble.com/Reading-Data-from-mle-into-excel-tp3545569p3545569.html
Sent from the R help mailing list archive at Nabble.com.
I would read the datasets into a list first, something like this which will make a list of dataframes: filenames <- dir() # where only filenames you want to read in are in this directory dataframelist <- lapply(filenames, read.csv, header = TRUE, sep = ",") You should be able to put the whole procedure, after reading in dataframes, into one lapply perhaps, e.g., lapply(dataframelist, yourfunction) where dataframelist is a list of dataframes, and yourfunction is a function that does all the procedures for one dataset. The function 'yourfunction' will be applied to each dataset in the list separately, then the results output into a list. Then, if the results from each dataset will have the same dimensions, you can do something like ldply using package plyr ldply(output, 'identity') # where 'output' is the output list of results from the lapply call above This will give you a data frame of all the results. Scott Chamberlain Rice University, EEB Dept. On Monday, May 23, 2011 at 4:32 PM, Bazman76 wrote: Hi there,> > I ran the following code: > > vols=read.csv(file="C:/Documents and Settings/Hugh/My Documents/PhD/Swaption > vols.csv" > , header=TRUE, sep=",") > X<-ts(vols[,2]) > #X > > > dcOU<-function(x,t,x0,theta,log=FALSE){ > Ex<-theta[1]/theta[2]+(x0-theta[1]/theta[2])*exp(-theta[2]*t) > Vx<-theta[3]^2*(1-exp(-2*theta[2]*t))/(2*theta[2]) > dnorm(x,mean=Ex,sd=sqrt(Vx),log=log) > } > OU.lik<-function(theta1,theta2,theta3){ > n<-length(X) > dt<-deltat(X) > -sum(dcOU(X[2:n],dt,X[1:(n-1)],c(theta1,theta2,theta3),log=TRUE)) > } > > require(stats4) > require(sde) > set.seed(1) > #X<-sde.sim(model="OU",theta=c(3,1,2),N=10000,delta=1) > mle(OU.lik,start=list(theta1=1,theta2=1,theta3=1), > method="L-BFGS-B",lower=c(-Inf,-Inf,-Inf),upper=c(Inf,Inf,Inf))->fit > summary(fit) > > #ex3.01 R > prof<-profile(fit) > par(mfrow=c(1,3)) > plot(prof) > par(mfrow=c(1,1)) > vcov(fit) > > I run the code above and I get: > > > summary(fit) > Maximum likelihood estimation > > Call: > mle(minuslogl = OU.lik, start = list(theta1 = 1, theta2 = 1, > theta3 = 1), method = "L-BFGS-B", lower = c(-Inf, -Inf, -Inf), > upper = c(Inf, Inf, Inf)) > > Coefficients: > Estimate Std. Error > theta1 0.03595581 0.013929892 > theta2 4.30910365 1.663781710 > theta3 0.02120220 0.004067477 > > -2 log L: -5136.327 > > I need to run the same analysis for 40 different time series. > > I want to be able to collate all the estimates of theta and the associated > stadard errors and then transfer them into excel? > > Can someone please point me to some R code that will allow me to do this? > > Thanks > > > > -- > View this message in context: http://r.789695.n4.nabble.com/Reading-Data-from-mle-into-excel-tp3545569p3545569.html > Sent from the R help mailing list archive at Nabble.com. > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >[[alternative HTML version deleted]]
Hi Scott,
Thanks for this.
Got some questions below:
Thanks
Hugh
Date: Mon, 23 May 2011 17:32:52 -0500
From: scttchamberlain4@gmail.com
To: h_a_patience@hotmail.com
CC: r-help@r-project.org
Subject: Re: [R] Reading Data from mle into excel?
I would read the datasets into a list first, something like this which will make
a list of dataframes:
filenames <- dir() # where only filenames you want to read in are in this
directory
dataframelist <- lapply(filenames, read.csv, header = TRUE, sep =
",")
OK I tried the code you suggest and I get:
filenames<-dir("C:/Documents and Settings/Hugh/My
Documents/PhD/Swaption vols.csv")> dataframelist<-lapply(filenames, read.csv, header = TRUE, sep =
",")
>
> dataframelist
list()> list
function (...) .Primitive("list")
Is this correct?
I only actually need one file, all the time series are stored in seperate
columns.
vols=read.csv(file="C:/Documents and Settings/Hugh/My
Documents/PhD/Swaption vols.csv"
, header=TRUE, sep=",")
X<-ts(vols[,2])
Can I still use this format?
You should be able to put the whole procedure, after reading in dataframes, into
one lapply perhaps, e.g.,
lapply(dataframelist, yourfunction)
as for my function the function mle involves calls to other functions?
dcOU<-function(x,t,x0,theta,log=FALSE){
Ex<-theta[1]/theta[2]+(x0-theta[1]/theta[2])*exp(-theta[2]*t)
Vx<-theta[3]^2*(1-exp(-2*theta[2]*t))/(2*theta[2])
dnorm(x,mean=Ex,sd=sqrt(Vx),log=log)
}
OU.lik<-function(theta1,theta2,theta3){
n<-length(X)
dt<-deltat(X)
-sum(dcOU(X[2:n],dt,X[1:(n-1)],c(theta1,theta2,theta3),log=TRUE))
}
require(stats4)
require(sde)
set.seed(1)
#X<-sde.sim(model="OU",theta=c(3,1,2),N=10000,delta=1)
mle(OU.lik,start=list(theta1=1,theta2=1,theta3=1),
method="L-BFGS-B",lower=c(-Inf,-Inf,-Inf),upper=c(Inf,Inf,Inf))->fit
summary(fit)
Should I store each function is a seperate script?
If so how do I then make sure that they are available in the
"workspace"?
Assuming that vols contains the dataframelist who would I call the mle function
using lapply like you showed?
where dataframelist is a list of dataframes, and yourfunction is a function that
does all the procedures for one dataset. The function 'yourfunction'
will be applied to each dataset in the list separately, then the results output
into a list.
Then, if the results from each dataset will have the same dimensions, you can do
something like ldply using package plyr
ldply(output, 'identity') # where 'output' is the output list of
results from the lapply call above
This will give you a data frame of all the results.
Scott Chamberlain
Rice University, EEB Dept.
On Monday, May 23, 2011 at 4:32 PM, Bazman76 wrote:
Hi there,
I ran the following code:
vols=read.csv(file="C:/Documents and Settings/Hugh/My
Documents/PhD/Swaption
vols.csv"
, header=TRUE, sep=",")
X<-ts(vols[,2])
#X
dcOU<-function(x,t,x0,theta,log=FALSE){
Ex<-theta[1]/theta[2]+(x0-theta[1]/theta[2])*exp(-theta[2]*t)
Vx<-theta[3]^2*(1-exp(-2*theta[2]*t))/(2*theta[2])
dnorm(x,mean=Ex,sd=sqrt(Vx),log=log)
}
OU.lik<-function(theta1,theta2,theta3){
n<-length(X)
dt<-deltat(X)
-sum(dcOU(X[2:n],dt,X[1:(n-1)],c(theta1,theta2,theta3),log=TRUE))
}
require(stats4)
require(sde)
set.seed(1)
#X<-sde.sim(model="OU",theta=c(3,1,2),N=10000,delta=1)
mle(OU.lik,start=list(theta1=1,theta2=1,theta3=1),
method="L-BFGS-B",lower=c(-Inf,-Inf,-Inf),upper=c(Inf,Inf,Inf))->fit
summary(fit)
#ex3.01 R
prof<-profile(fit)
par(mfrow=c(1,3))
plot(prof)
par(mfrow=c(1,1))
vcov(fit)
I run the code above and I get:
summary(fit)
Maximum likelihood estimation
Call:
mle(minuslogl = OU.lik, start = list(theta1 = 1, theta2 = 1,
theta3 = 1), method = "L-BFGS-B", lower = c(-Inf, -Inf, -Inf),
upper = c(Inf, Inf, Inf))
Coefficients:
Estimate Std. Error
theta1 0.03595581 0.013929892
theta2 4.30910365 1.663781710
theta3 0.02120220 0.004067477
-2 log L: -5136.327
I need to run the same analysis for 40 different time series.
I want to be able to collate all the estimates of theta and the associated
stadard errors and then transfer them into excel?
Can someone please point me to some R code that will allow me to do this?
Thanks
--
View this message in context:
http://r.789695.n4.nabble.com/Reading-Data-from-mle-into-excel-tp3545569p3545569.html
Sent from the R help mailing list archive at Nabble.com.
______________________________________________
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]
Hi:
This isn't too hard to do. The strategy is basically this:
(1) Create a list of file names. (See ?list.files for some ideas)
(2) Read the data files from (1) into a list.
(3) Create a function to apply to each data frame in the list.
(4) Apply the function to each data frame.
(5) Extract the coefficients/output and put them into a new data frame or list.
Here's a really simple example, but it exemplifies the process. I
created five data frames and exported them to .csv files in my current
directory. The game is to create a vector of file names, use lapply()
to pass them into a list, create a function to extract the
coefficients from a linear regression model, and then use ldply() from
the plyr package to combine steps (4) and (5) above.
files <- paste('dat', 1:5, '.csv', sep = '')
# Code to create the output files in case you want to try out the example
# wf <- function(f)
# write.csv(data.frame(x1 = rnorm(10), x2 = rnorm(10), y = rnorm(10)),
# file = f, quote = FALSE, row.names = FALSE)
# lapply(files, wf)
library(plyr)
datalist <- lapply(files, read.csv, header = TRUE)
lfun <- function(d) coef(lm(y ~ x1 + x2, data = d))
ldply(datalist, lfun)
You should look at ?list.files to help create the file list you need.
You should then be able to use something similar to the code that
generates datalist to get the data frames into a single list.
Obviously, my function is a lot simpler than yours, but the principle
is that the function should work for any generic data object in your
list. I like the ldply() function for this simple example because it's
'one-stop shopping', but the base package alternative would be
something like
do.call(rbind, lapply(datalist, lfun))
The challenge in your problem is that you want to return a coefficient
matrix, which raises a new set of issues. If you use the do.call()
approach, it won't keep track of the corresponding data frame to which
the results pertain. ldply() won't work because the coefficient matrix
is not a data frame and if you coerce it to one, the row names
(variables) will disappear. One approach is to use llply() instead;
consider the following function and its application:
# This function returns a matrix of coefficients, standard errors,
# t-tests of significance and p-values rather than a vector
lfun2 <- function(d) summary(lm(y ~ ., data = d))['coefficients']
# returns a list rather than a data frame
llply(datalist, lfun2)
It's not hard to figure out how to write the results to one or more
files from there. You may want to adapt your function to get the
output you need.
HTH,
Dennis
On Mon, May 23, 2011 at 2:32 PM, Bazman76 <h_a_patience at hotmail.com>
wrote:> Hi there,
>
> I ran the following code:
>
> vols=read.csv(file="C:/Documents and Settings/Hugh/My
Documents/PhD/Swaption
> vols.csv"
> , header=TRUE, sep=",")
> X<-ts(vols[,2])
> #X
>
>
> dcOU<-function(x,t,x0,theta,log=FALSE){
> Ex<-theta[1]/theta[2]+(x0-theta[1]/theta[2])*exp(-theta[2]*t)
> Vx<-theta[3]^2*(1-exp(-2*theta[2]*t))/(2*theta[2])
> dnorm(x,mean=Ex,sd=sqrt(Vx),log=log)
> }
> OU.lik<-function(theta1,theta2,theta3){
> n<-length(X)
> dt<-deltat(X)
> -sum(dcOU(X[2:n],dt,X[1:(n-1)],c(theta1,theta2,theta3),log=TRUE))
> }
>
> require(stats4)
> require(sde)
> set.seed(1)
> #X<-sde.sim(model="OU",theta=c(3,1,2),N=10000,delta=1)
> mle(OU.lik,start=list(theta1=1,theta2=1,theta3=1),
>
method="L-BFGS-B",lower=c(-Inf,-Inf,-Inf),upper=c(Inf,Inf,Inf))->fit
> summary(fit)
>
> #ex3.01 R
> prof<-profile(fit)
> par(mfrow=c(1,3))
> plot(prof)
> par(mfrow=c(1,1))
> vcov(fit)
>
> I run the code above and I get:
>
>> summary(fit)
> Maximum likelihood estimation
>
> Call:
> mle(minuslogl = OU.lik, start = list(theta1 = 1, theta2 = 1,
> ? ?theta3 = 1), method = "L-BFGS-B", lower = c(-Inf, -Inf, -Inf),
> ? ?upper = c(Inf, Inf, Inf))
>
> Coefficients:
> ? ? ? ? Estimate ?Std. Error
> theta1 0.03595581 0.013929892
> theta2 4.30910365 1.663781710
> theta3 0.02120220 0.004067477
>
> -2 log L: -5136.327
>
> I need to run the same analysis for 40 different time series.
>
> I want to be able to collate all the estimates of theta and the associated
> stadard errors and then transfer them into excel?
>
> Can someone please point me to some R code that will allow me to do this?
>
> Thanks
>
>
>
> --
> View this message in context:
http://r.789695.n4.nabble.com/Reading-Data-from-mle-into-excel-tp3545569p3545569.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
I think cognizance should be taken of fortune("very uneasy").
cheers,
Rolf Turner
thanks for all your help
I have taken a slightly different route but I think I am getting there
library(plyr)
#setwd("C:/Documents and Settings/Hugh/My Documents/PhD")
#files<-list.files("C:/Documents and Settings/Hugh/My
Documents/PhD/",pattern="Swaption Vols.csv")
#vols <- lapply(files, read.csv, header = TRUE)
vols=read.csv(file="C:/Documents and Settings/Hugh/My
Documents/PhD/Swaption
vols.csv"
+ , header=TRUE, sep=",")
dcOU<-function(x,t,x0,theta,log=FALSE){
Ex<-theta[1]/theta[2]+(x0-theta[1]/theta[2])*exp(-theta[2]*t)
Vx<-theta[3]^2*(1-exp(-2*theta[2]*t))/(2*theta[2])
dnorm(x,mean=Ex,sd=sqrt(Vx),log=log)
}
OU.lik<-function(theta1,theta2,theta3){
n<-length(X)
dt<-deltat(X)
-sum(dcOU(X[2:n],dt,X[1:(n-1)],c(theta1,theta2,theta3),log=TRUE))
}
require(stats4)
require(sde)
nc=ncol(vols)
for(i in 2:nc){
X<-ts(vols[,i])
mle(OU.lik,start=list(theta1=1,theta2=1,theta3=1),
method="L-BFGS-B",lower=c(-Inf,-Inf,-Inf),upper=c(Inf,Inf,Inf))->fit
summary(fit)
}
right now the summary(fit) gives the summary results for the 53rd column so
the code is working correctly.
How can I save these summary results in an array or datatable on each loop?
--
View this message in context:
http://r.789695.n4.nabble.com/Reading-Data-from-mle-into-excel-tp3545569p3547236.html
Sent from the R help mailing list archive at Nabble.com.
Can I use sink() to transfer the MLE results which are a S4 type object to a text file? Can someone show me how to do this? -- View this message in context: http://r.789695.n4.nabble.com/Reading-Data-from-mle-into-excel-tp3545569p3563385.html Sent from the R help mailing list archive at Nabble.com.