R help - Mar 2006 - (newbie) Accessing the pieces of a 'by' object

Folks,
I know that I can do the following using a loop. That's been a lot
easier for me to write and understand. But I am trying to force myself
to use more vectorized / matrixed code so that eventually I will
become a better R programmer.

I have a dataframe that has some values by Year, Quarter and Ranking.
The variable of interest is the return (F3MRet), to be weighted
averaged within the year, quarter and ranking. At the end, we want to
end up with a table like this:
year  quarter ranking1 ranking2 ... ranking10
1987 1 1.33 1.45 ... 1.99
1987 2 6.45 3.22 ... 8.33
.
.
2005 4 2.22 3.33 ... 1.22

The dataset is too large to post and I can't come up with a small
working example very easily.

I tried the Reshape() package and also the aggregate and reshape
functions. Those don't work too well becuase of the need to pass
weighted.mean a weights vector. I tried the by() function, but now I
don't know how to coerce the returned object into a matrix so that I
can reshape it.
> fvs_weighted.mean <- function(y) weighted.mean(y$F3MRet, y$IndexWeight,
na.rm=T);
> tmp_byRet <- by(dfReturns,
     list(dfReturns$Quarter,dfReturns$Year,dfReturns$Ranking),
     fvs_weighted.mean);

And various other ways to get the tmp_byRet object into a matrix were
tried, eg. unlist(), a loop like this:
dfRet <- data.frame(tmp_byRet);
for(i in 1:dim(dfRet)[2]){
       dfRet[ ,i] <- as.vector(dfRet[ ,i]);
}
In each case, I got some error or the other.

So, please help me get unstuck. How can I get the tmp_byRet() object
into a matrix or a dataframe?

--
-- Vivek Satsangi
Rochester, NY USA
"No amount of sophistication is going to allay the fact that all your
knowledge is about the past and all your decisions are about the
future." -- Ian Wilson

Try this:

iris.by <- do.call("rbind", by(iris[,-5],
iris[,5,drop=FALSE],colSums))
do.call("rbind", iris.by)

On 3/7/06, Vivek Satsangi <vivek.satsangi at gmail.com>
wrote:
> Folks,
> I know that I can do the following using a loop. That's been a lot
> easier for me to write and understand. But I am trying to force myself
> to use more vectorized / matrixed code so that eventually I will
> become a better R programmer.
>
> I have a dataframe that has some values by Year, Quarter and Ranking.
> The variable of interest is the return (F3MRet), to be weighted
> averaged within the year, quarter and ranking. At the end, we want to
> end up with a table like this:
> year  quarter ranking1 ranking2 ... ranking10
> 1987 1 1.33 1.45 ... 1.99
> 1987 2 6.45 3.22 ... 8.33
> .
> .
> 2005 4 2.22 3.33 ... 1.22
>
> The dataset is too large to post and I can't come up with a small
> working example very easily.
>
> I tried the Reshape() package and also the aggregate and reshape
> functions. Those don't work too well becuase of the need to pass
> weighted.mean a weights vector. I tried the by() function, but now I
> don't know how to coerce the returned object into a matrix so that I
> can reshape it.
>
> > fvs_weighted.mean <- function(y) weighted.mean(y$F3MRet,
y$IndexWeight, na.rm=T);
> > tmp_byRet <- by(dfReturns,
>     list(dfReturns$Quarter,dfReturns$Year,dfReturns$Ranking),
>     fvs_weighted.mean);
>
> And various other ways to get the tmp_byRet object into a matrix were
> tried, eg. unlist(), a loop like this:
> dfRet <- data.frame(tmp_byRet);
> for(i in 1:dim(dfRet)[2]){
>       dfRet[ ,i] <- as.vector(dfRet[ ,i]);
> }
> In each case, I got some error or the other.
>
> So, please help me get unstuck. How can I get the tmp_byRet() object
> into a matrix or a dataframe?
>
> --
> -- Vivek Satsangi
> Rochester, NY USA
> "No amount of sophistication is going to allay the fact that all your
> knowledge is about the past and all your decisions are about the
> future." -- Ian Wilson
>
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide!
http://www.R-project.org/posting-guide.html
>

I am writing to document the answer for the next poor sod who comes along.

To get tmp_byRet() into a multi-dimentional matrix, copy the object
using as.vector(), then copy the dim and dimnames from tmp_byRet into
the new object. However, this may not be what you want, since you
probably want the values of the factors within the object (i.e. it
should be a dataframe, not a matrix).

To get tmp_byRet into a dataframe, use unique() to create a dataframe
with just the unique values of your factors. Add a new column to the
dataframe, where you will store the summary stats. Use a loop to
populate this vector. Then use reshape() on the dataframe to get it to
the shape you want it in. It is difficult at best to vectorize this
and avoid the loop  -- and trying to do so will lead to probably less
transparent code.

Vivek


On 3/7/06, Vivek Satsangi <vivek.satsangi at gmail.com>
wrote:
> Folks,
> I know that I can do the following using a loop. That's been a lot
> easier for me to write and understand. But I am trying to force myself
> to use more vectorized / matrixed code so that eventually I will
> become a better R programmer.
>
> I have a dataframe that has some values by Year, Quarter and Ranking.
> The variable of interest is the return (F3MRet), to be weighted
> averaged within the year, quarter and ranking. At the end, we want to
> end up with a table like this:
> year  quarter ranking1 ranking2 ... ranking10
> 1987 1 1.33 1.45 ... 1.99
> 1987 2 6.45 3.22 ... 8.33
> .
> .
> 2005 4 2.22 3.33 ... 1.22
>
> The dataset is too large to post and I can't come up with a small
> working example very easily.
>
> I tried the Reshape() package and also the aggregate and reshape
> functions. Those don't work too well becuase of the need to pass
> weighted.mean a weights vector. I tried the by() function, but now I
> don't know how to coerce the returned object into a matrix so that I
> can reshape it.
>
> > fvs_weighted.mean <- function(y) weighted.mean(y$F3MRet,
y$IndexWeight, na.rm=T);
> > tmp_byRet <- by(dfReturns,
>      list(dfReturns$Quarter,dfReturns$Year,dfReturns$Ranking),
>      fvs_weighted.mean);
>
> And various other ways to get the tmp_byRet object into a matrix were
> tried, eg. unlist(), a loop like this:
> dfRet <- data.frame(tmp_byRet);
> for(i in 1:dim(dfRet)[2]){
>        dfRet[ ,i] <- as.vector(dfRet[ ,i]);
> }
> In each case, I got some error or the other.
>
> So, please help me get unstuck. How can I get the tmp_byRet() object
> into a matrix or a dataframe?
>
> --