R help - Jul 2012 - do I need plyr, apply or something else?

Dear all,

This is what I'd like to do (I have an implementation using for loops, which
I designed before I realised just how slow R is at executing them - this process
currently takes days to run).

I have a large dataframe containing corporate bond data, columns are:
BondID
Date (goes back 5years)
Var1
Var2
Term2Maturity

What I want to do is this:

1)      For each bond, at each given date, look back over 1 year and append some
statistics to each row ( sd(Var1), cor(Var1,Var2) over that year etc)

a.       It seems I might be able to use ddply for this, but I can't work
out how to code the stats function to only look back over one year, rather than
the full data range

b.      For example: dfBondsWithCorr<-ddply(dfBonds, .(BondID),
transform,corr=cor(Var1,Var2),.progress="text")
returns a dataframe where for each bond it has same corr for each date

2)      On each date, subset dfBondsWithCorr by certain qualification criteria,
then to the qualifiers fit a regression through a Var1 and Term2Maturity, output
the regression as a df of curves (say for each date, a curve represented by
points every 0.5 years)

a.       I can do this pretty efficiently for a single date (and I suppose I
could wrap that in a function) , but can't quite see how to do the filtering
and spitting out of curves over multiple dates without using for loops

Would appreciate any thoughts, many thanks in advance


Russ



This email and any attachments are confidential and inte...{{dropped:30}}

On Wed, Jul 11, 2012 at 10:05 AM, Russell Bowdrey
<Russell.Bowdrey at justretirement.com> wrote:
>
> Dear all,
>
> This is what I'd like to do (I have an implementation using for loops,
which I designed before I realised just how slow R is at executing them - this
process currently takes days to run).
>
> I have a large dataframe containing corporate bond data, columns are:
> BondID
> Date (goes back 5years)
> Var1
> Var2
> Term2Maturity
>
> What I want to do is this:
>
> 1)      For each bond, at each given date, look back over 1 year and append
some statistics to each row ( sd(Var1), cor(Var1,Var2) over that year etc)
>
Look at the TTR package and the various run** functions. Much faster.
> a.       It seems I might be able to use ddply for this, but I can't
work out how to code the stats function to only look back over one year, rather
than the full data range
>
> b.      For example: dfBondsWithCorr<-ddply(dfBonds, .(BondID),
transform,corr=cor(Var1,Var2),.progress="text")
> returns a dataframe where for each bond it has same corr for each date
>
> 2)      On each date, subset dfBondsWithCorr by certain qualification
criteria, then to the qualifiers fit a regression through a Var1 and
Term2Maturity, output the regression as a df of curves (say for each date, a
curve represented by points every 0.5 years)
>
> a.       I can do this pretty efficiently for a single date (and I suppose
I could wrap that in a function) , but can't quite see how to do the
filtering and spitting out of curves over multiple dates without using for loops
>
This ones harder. For simple linear regressions, you can solve the
regression analytically (e.g., slope = runCov / runVar and mean
similarly) but doing it for more complicated regressions will pretty
much require a for loop of one sort or another. Can you say what sort
of model you are looking to use?

Best,
Michael
> Would appreciate any thoughts, many thanks in advance
>
>
> Russ
>
>
>
> This email and any attachments are confidential and inte...{{dropped:30}}
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.