I continue to have great frustrations with NA values--in particular making summary calculations on rows or cols of a matrix containing them. For example, why does:> a = matrix(1:30,nrow=5) > is.na(a[c(1:2),c(3:4)]);a[,1] [,2] [,3] [,4] [,5] [,6] [1,] 1 6 NA NA 21 26 [2,] 2 7 NA NA 22 27 [3,] 3 8 13 18 23 28 [4,] 4 9 14 19 24 29 [5,] 5 10 15 20 25 30> apply(a[!is.na(a)],2,sum)give me this: "Error in apply(a[!is.na(a)], 2, sum) : dim(X) must have a positive length" when> dim(a)[1] 5 6 What is the trick to calculating summary values from rows or columns containing NAs? Drives me nuts. More nuts that is. Thanks. Jim Bouldin, PhD Research Ecologist Department of Plant Sciences, UC Davis Davis CA, 95616 530-554-1740
What do you expect a[!is.na(a)] to be? The a matrix has 30 elements, 5 rows, and 6 columns. You tell R to throw away 4 elements leaving you with 26, the computer (and most of the rest of us) doesn't know how to make a 5x6 matrix with only 26 elements. It could make a 2x13 or 13x2, but that is unlikely what you want, so it just returns a vector without dimensions which then confuses the apply function. Instead try:> apply(a,2,sum, na.rm=TRUE)Or> apply(a, 2, function(x) sum( x[!is.na(x)] ) )Hope this helps, -- Gregory (Greg) L. Snow Ph.D. Statistical Data Center Intermountain Healthcare greg.snow at imail.org 801.408.8111> -----Original Message----- > From: r-help-bounces at r-project.org [mailto:r-help-bounces at r- > project.org] On Behalf Of Jim Bouldin > Sent: Thursday, March 11, 2010 3:26 PM > To: R help > Subject: [R] NAs and row/column calculations > > > I continue to have great frustrations with NA values--in particular > making > summary calculations on rows or cols of a matrix containing them. For > example, why does: > > > a = matrix(1:30,nrow=5) > > is.na(a[c(1:2),c(3:4)]);a > [,1] [,2] [,3] [,4] [,5] [,6] > [1,] 1 6 NA NA 21 26 > [2,] 2 7 NA NA 22 27 > [3,] 3 8 13 18 23 28 > [4,] 4 9 14 19 24 29 > [5,] 5 10 15 20 25 30 > > apply(a[!is.na(a)],2,sum) > > give me this: > > "Error in apply(a[!is.na(a)], 2, sum) : dim(X) must have a positive > length" > > when > > > dim(a) > [1] 5 6 > > What is the trick to calculating summary values from rows or columns > containing NAs? Drives me nuts. More nuts that is. > > Thanks. > > > > > Jim Bouldin, PhD > Research Ecologist > Department of Plant Sciences, UC Davis > Davis CA, 95616 > 530-554-1740 > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting- > guide.html > and provide commented, minimal, self-contained, reproducible code.
On 12/03/2010, at 11:25 AM, Jim Bouldin wrote:> > I continue to have great frustrations with NA values--in particular making > summary calculations on rows or cols of a matrix containing them. For > example, why does: > >> a = matrix(1:30,nrow=5) >> is.na(a[c(1:2),c(3:4)]);a > [,1] [,2] [,3] [,4] [,5] [,6] > [1,] 1 6 NA NA 21 26 > [2,] 2 7 NA NA 22 27 > [3,] 3 8 13 18 23 28 > [4,] 4 9 14 19 24 29 > [5,] 5 10 15 20 25 30 >> apply(a[!is.na(a)],2,sum) > > give me this: > > "Error in apply(a[!is.na(a)], 2, sum) : dim(X) must have a positive length" > > when > >> dim(a) > [1] 5 6 > > What is the trick to calculating summary values from rows or columns > containing NAs? Drives me nuts. More nuts that is.When you do a[!is.na(a)] you get a ***vector*** --- not a matrix. ``Obviously''!!! The non-missing values of a cannot be arranged in a 5 x 6 matrix; there are only 26 of them. So (as my late Uncle Stanley would have said) ``What the hell do you expect?''. The ``trick'' is to remove the NAs at the summing stage: apply(a,2,sum,na.rm=TRUE) Not all that tricky. cheers, Rolf Turner ###################################################################### Attention:\ This e-mail message is privileged and confid...{{dropped:9}}
> > On 12/03/2010, at 11:25 AM, Jim Bouldin wrote: > > > > > I continue to have great frustrations with NA values--in particular > making > > summary calculations on rows or cols of a matrix containing them. For > > example, why does: > > > >> a = matrix(1:30,nrow=5) > >> is.na(a[c(1:2),c(3:4)]);a > > [,1] [,2] [,3] [,4] [,5] [,6] > > [1,] 1 6 NA NA 21 26 > > [2,] 2 7 NA NA 22 27 > > [3,] 3 8 13 18 23 28 > > [4,] 4 9 14 19 24 29 > > [5,] 5 10 15 20 25 30 > >> apply(a[!is.na(a)],2,sum) > > > > give me this: > > > > "Error in apply(a[!is.na(a)], 2, sum) : dim(X) must have a positive > length" > > > > when > > > >> dim(a) > > [1] 5 6 > > > > What is the trick to calculating summary values from rows or columns > > containing NAs? Drives me nuts. More nuts that is. > > When you do a[!is.na(a)] you get a ***vector*** --- not a matrix. > ``Obviously''!!!Well, obvious to you maybe, or someone who's done it before, but not to me. The non-missing values of a cannot be arranged in> a 5 x 6 matrix; there are only 26 of them. So (as my late Uncle > Stanley would have said) ``What the hell do you expect?''.Silly me, I expected, based on (1) previous experience doing summary calcs on subsets of a matrix using exactly that style of command, and (2) the fact that dim(a) returns: [1] 5 6, and (3) the fact that a help search under the "apply" function gives NO INDICATION of any possible use of the na.rm command, AND (4) a help search on "na.action" does not even mention na.rm, that:> apply(a[!is.na(a)],2,sum)would sum the non-NA elements of matrix a, by columns. Terribly faulty reasoning on my part, obviously.> > The ``trick'' is to remove the NAs at the summing stage: > > apply(a,2,sum,na.rm=TRUE) > > Not all that tricky. > > cheers, > > Rolf Turner > > ###################################################################### > Attention: > This e-mail message is privileged and confidential. If you are not the > intended recipient please delete the message and notify the sender. > Any views or opinions presented are solely those of the author. > > This e-mail has been scanned and cleared by MailMarshal > www.marshalsoftware.com > ###################################################################### >Jim Bouldin, PhD Research Ecologist Department of Plant Sciences, UC Davis Davis CA, 95616 530-554-1740