Hi there,
I need to do the same thing as cumsum but with the variance and skewness. I
have tried to do a loop for like this:
var.value <- vector(mode = "numeric", length =
length(daily))
for (i in (1:length(daily))) {
var.value[i] <- var(daily[1:i])
}
But because my dataset is so huge, I run out of memory.....
Any ideas?!?!
Much appreciate it!
--
View this message in context:
http://r.789695.n4.nabble.com/cumVar-and-cumSkew-tp3816899p3816899.html
Sent from the R help mailing list archive at Nabble.com.
Wow, this takes me back a ways :-)
As I recall, BMDP used updating algorithms to compute sample means,
variances, and covariances in one pass through the dataset.
You need the updating algorithm for means:
\bar{X}_{n} = \frac{n-1}{n}\bar{X}_{n-1}+\frac{X_n}{n}
You can work out the algorithm for variances using the same idea,
or consult:
Algorithms for Computing the Sample Variance:
Analysis and Recommendations Algorithms for Computing the Sample Variance:
Tony F. Chan, Gene H. Golub, Randall J. LeVeque
The American Statistician, Vol. 37, No. 3 (Aug., 1983), pp. 242-247
I think the updating and other algorithms are described therein.
albyn
On Thu, Sep 15, 2011 at 02:57:40PM -0700, D_Tomas wrote:> Hi there,
>
> I need to do the same thing as cumsum but with the variance and skewness. I
> have tried to do a loop for like this:
> var.value <- vector(mode = "numeric", length =
length(daily))
> for (i in (1:length(daily))) {
> var.value[i] <- var(daily[1:i])
> }
>
> But because my dataset is so huge, I run out of memory.....
>
>
> Any ideas?!?!
>
> Much appreciate it!
>
> --
> View this message in context:
http://r.789695.n4.nabble.com/cumVar-and-cumSkew-tp3816899p3816899.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
--
Albyn Jones
Reed College
jones at reed.edu
On Sep 15, 2011, at 5:57 PM, D_Tomas wrote:> Hi there, > > I need to do the same thing as cumsum but with the variance and > skewness. I > have tried to do a loop for like this: > var.value <- vector(mode = "numeric", length = length(daily)) > for (i in (1:length(daily))) { > var.value[i] <- var(daily[1:i]) > } >daily <- rnorm(1000000) mbar <- mean(daily) cumvar <- cumsum( (daily-cumsum(daily)/1:length(daily) )^2) cumskew <- cumsum( (daily-cumsum(daily)/1:length(daily))^3)/ cumvar^(3/2) Hmm. Not sure it should be defined at the first couple of indices. First version scrapped but this one seems to be working: > str(cumvar) num [1:1000000] 0 0.265 0.334 1.324 1.845 ... > str(cumskew) num [1:1000000] NaN -1 -0.8 0.546 0.182 ... > plot(cumskew[1:1000]) # settles out as it should for > plot(cumvar[1:1000]) # appears to be linear with index> But because my dataset is so huge, I run out of memory..... >Clean up your workspace and don't run lots of apps in teh background. That operation should not have been memory intensive. It would have been slow if you had not preallocated var.value, though.> > Any ideas?!?! > > Much appreciate it! > > -- > View this message in context: http://r.789695.n4.nabble.com/cumVar-and-cumSkew-tp3816899p3816899.html > Sent from the R help mailing list archive at Nabble.com. > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.David Winsemius, MD West Hartford, CT