R help - May 2012 - removeing only rows/columns with "na" value from square ( symmetrical ) matrix.

I have some square matrices with na values in corresponding rows and
columns.

M<-matrix(1:2,10,10)
M[6,1:2]<-NA
M[10,9]<-NA
M<-as.matrix(as.dist(M))
print (M)

    1  2 3 4 5  6 7 8  9 10
1   0  2 1 2 1 NA 1 2  1  2
2   2  0 1 2 1 NA 1 2  1  2
3   1  1 0 2 1  2 1 2  1  2
4   2  2 2 0 1  2 1 2  1  2
5   1  1 1 1 0  2 1 2  1  2
6  NA NA 2 2 2  0 1 2  1  2
7   1  1 1 1 1  1 0 2  1  2
8   2  2 2 2 2  2 2 0  1  2
9   1  1 1 1 1  1 1 1  0 NA
10  2  2 2 2 2  2 2 2 NA  0


How do I remove just the row/column pair( in this trivial example row 6 and
10 and column 6 and 10) containing the NA values?

so that I end up with all rows/ columns that are not NA - e.g.

  1 2 3 4 5 7 8 9
1 0 2 1 2 1 1 2 1
2 2 0 1 2 1 1 2 1
3 1 1 0 2 1 1 2 1
4 2 2 2 0 1 1 2 1
5 1 1 1 1 0 1 2 1
7 1 1 1 1 1 0 2 1
8 2 2 2 2 2 2 0 1
9 1 1 1 1 1 1 1 0


if i use na omit I lose rows 1,2,6, and 9
which is not what I want.

thanks
-- 
Nevil Amos
Molecular Ecology Research Group
Australian Centre for Biodiversity
Monash University
CLAYTON VIC 3800
Australia

	[[alternative HTML version deleted]]

Your problem is not well-defined. In your example below, why not
remove rows 1,2,6, and 10, all of which contain NA's? Is the matrix
supposed to be symmetric? Do NA's always occur symmetrically?

You either need to rethink what you want to do or clarify your statement of it.

-- Bert

On Sun, May 20, 2012 at 7:17 AM, Nevil Amos <nevil.amos at monash.edu>
wrote:
> I have some square matrices with na values in corresponding rows and
> columns.
>
> M<-matrix(1:2,10,10)
> M[6,1:2]<-NA
> M[10,9]<-NA
> M<-as.matrix(as.dist(M))
> print (M)
>
> ? ?1 2 3 4 5 6 7 8 9 10
> 1 ? 0 ?2 1 2 1 NA 1 2 ?1 ?2
> 2 ? 2 ?0 1 2 1 NA 1 2 ?1 ?2
> 3 ? 1 ?1 0 2 1 ?2 1 2 ?1 ?2
> 4 ? 2 ?2 2 0 1 ?2 1 2 ?1 ?2
> 5 ? 1 ?1 1 1 0 ?2 1 2 ?1 ?2
> 6 ?NA NA 2 2 2 ?0 1 2 ?1 ?2
> 7 ? 1 ?1 1 1 1 ?1 0 2 ?1 ?2
> 8 ? 2 ?2 2 2 2 ?2 2 0 ?1 ?2
> 9 ? 1 ?1 1 1 1 ?1 1 1 ?0 NA
> 10 ?2 ?2 2 2 2 ?2 2 2 NA ?0
>
>
> How do I remove just the row/column pair( in this trivial example row 6 and
> 10 and column 6 and 10) containing the NA values?
>
> so that I end up with all rows/ columns that are not NA - e.g.
>
> ?1 2 3 4 5 7 8 9
> 1 0 2 1 2 1 1 2 1
> 2 2 0 1 2 1 1 2 1
> 3 1 1 0 2 1 1 2 1
> 4 2 2 2 0 1 1 2 1
> 5 1 1 1 1 0 1 2 1
> 7 1 1 1 1 1 0 2 1
> 8 2 2 2 2 2 2 0 1
> 9 1 1 1 1 1 1 1 0
>
>
> if i use na omit I lose rows 1,2,6, and 9
> which is not what I want.
>
> thanks
> --
> Nevil Amos
> Molecular Ecology Research Group
> Australian Centre for Biodiversity
> Monash University
> CLAYTON VIC 3800
> Australia
>
> ? ? ? ?[[alternative HTML version deleted]]
>
> ______________________________________________
> R-help at r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.


-- 

Bert Gunter
Genentech Nonclinical Biostatistics

Internal Contact Info:
Phone: 467-7374
Website:
http://pharmadevelopment.roche.com/index/pdb/pdb-functional-groups/pdb-biostatistics/pdb-ncb-home.htm

On Sun, May 20, 2012 at 10:17 AM, Nevil Amos <nevil.amos at monash.edu>
wrote:
> I have some square matrices with na values in corresponding rows and
> columns.
>
> M<-matrix(1:2,10,10)
> M[6,1:2]<-NA
> M[10,9]<-NA
> M<-as.matrix(as.dist(M))
> print (M)
>
> ? ?1 2 3 4 5 6 7 8 9 10
> 1 ? 0 ?2 1 2 1 NA 1 2 ?1 ?2
> 2 ? 2 ?0 1 2 1 NA 1 2 ?1 ?2
> 3 ? 1 ?1 0 2 1 ?2 1 2 ?1 ?2
> 4 ? 2 ?2 2 0 1 ?2 1 2 ?1 ?2
> 5 ? 1 ?1 1 1 0 ?2 1 2 ?1 ?2
> 6 ?NA NA 2 2 2 ?0 1 2 ?1 ?2
> 7 ? 1 ?1 1 1 1 ?1 0 2 ?1 ?2
> 8 ? 2 ?2 2 2 2 ?2 2 0 ?1 ?2
> 9 ? 1 ?1 1 1 1 ?1 1 1 ?0 NA
> 10 ?2 ?2 2 2 2 ?2 2 2 NA ?0
>
>
> How do I remove just the row/column pair( in this trivial example row 6 and
> 10 and column 6 and 10) containing the NA values?
>
> so that I end up with all rows/ columns that are not NA - e.g.
>
> ?1 2 3 4 5 7 8 9
> 1 0 2 1 2 1 1 2 1
> 2 2 0 1 2 1 1 2 1
> 3 1 1 0 2 1 1 2 1
> 4 2 2 2 0 1 1 2 1
> 5 1 1 1 1 0 1 2 1
> 7 1 1 1 1 1 0 2 1
> 8 2 2 2 2 2 2 0 1
> 9 1 1 1 1 1 1 1 0
>
Try this:

ix <- na.action(na.omit(replace(M, upper.tri(M), 0)))
M[-ix, -ix]


-- 
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com

On Sun, May 20, 2012 at 10:52 AM, Gabor Grothendieck
<ggrothendieck at gmail.com> wrote:
> On Sun, May 20, 2012 at 10:17 AM, Nevil Amos <nevil.amos at
monash.edu> wrote:
>> I have some square matrices with na values in corresponding rows and
>> columns.
>>
>> M<-matrix(1:2,10,10)
>> M[6,1:2]<-NA
>> M[10,9]<-NA
>> M<-as.matrix(as.dist(M))
>> print (M)
>>
>> ? ?1 2 3 4 5 6 7 8 9 10
>> 1 ? 0 ?2 1 2 1 NA 1 2 ?1 ?2
>> 2 ? 2 ?0 1 2 1 NA 1 2 ?1 ?2
>> 3 ? 1 ?1 0 2 1 ?2 1 2 ?1 ?2
>> 4 ? 2 ?2 2 0 1 ?2 1 2 ?1 ?2
>> 5 ? 1 ?1 1 1 0 ?2 1 2 ?1 ?2
>> 6 ?NA NA 2 2 2 ?0 1 2 ?1 ?2
>> 7 ? 1 ?1 1 1 1 ?1 0 2 ?1 ?2
>> 8 ? 2 ?2 2 2 2 ?2 2 0 ?1 ?2
>> 9 ? 1 ?1 1 1 1 ?1 1 1 ?0 NA
>> 10 ?2 ?2 2 2 2 ?2 2 2 NA ?0
>>
>>
>> How do I remove just the row/column pair( in this trivial example row 6
and
>> 10 and column 6 and 10) containing the NA values?
>>
>> so that I end up with all rows/ columns that are not NA - e.g.
>>
>> ?1 2 3 4 5 7 8 9
>> 1 0 2 1 2 1 1 2 1
>> 2 2 0 1 2 1 1 2 1
>> 3 1 1 0 2 1 1 2 1
>> 4 2 2 2 0 1 1 2 1
>> 5 1 1 1 1 0 1 2 1
>> 7 1 1 1 1 1 0 2 1
>> 8 2 2 2 2 2 2 0 1
>> 9 1 1 1 1 1 1 1 0
>>
>
> Try this:
>
> ix <- na.action(na.omit(replace(M, upper.tri(M), 0)))
> M[-ix, -ix]
and here is a minor variation which is slightly shorter:

ix <- complete.cases(replace(M, upper.tri(M), 0))
M[ix, ix]

-- 
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com

On Sun, May 20, 2012 at 10:54 AM, Gabor Grothendieck
<ggrothendieck at gmail.com> wrote:
> On Sun, May 20, 2012 at 10:52 AM, Gabor Grothendieck
> <ggrothendieck at gmail.com> wrote:
>> On Sun, May 20, 2012 at 10:17 AM, Nevil Amos <nevil.amos at
monash.edu> wrote:
>>> I have some square matrices with na values in corresponding rows
and
>>> columns.
>>>
>>> M<-matrix(1:2,10,10)
>>> M[6,1:2]<-NA
>>> M[10,9]<-NA
>>> M<-as.matrix(as.dist(M))
>>> print (M)
>>>
>>> ? ?1 2 3 4 5 6 7 8 9 10
>>> 1 ? 0 ?2 1 2 1 NA 1 2 ?1 ?2
>>> 2 ? 2 ?0 1 2 1 NA 1 2 ?1 ?2
>>> 3 ? 1 ?1 0 2 1 ?2 1 2 ?1 ?2
>>> 4 ? 2 ?2 2 0 1 ?2 1 2 ?1 ?2
>>> 5 ? 1 ?1 1 1 0 ?2 1 2 ?1 ?2
>>> 6 ?NA NA 2 2 2 ?0 1 2 ?1 ?2
>>> 7 ? 1 ?1 1 1 1 ?1 0 2 ?1 ?2
>>> 8 ? 2 ?2 2 2 2 ?2 2 0 ?1 ?2
>>> 9 ? 1 ?1 1 1 1 ?1 1 1 ?0 NA
>>> 10 ?2 ?2 2 2 2 ?2 2 2 NA ?0
>>>
>>>
>>> How do I remove just the row/column pair( in this trivial example
row 6 and
>>> 10 and column 6 and 10) containing the NA values?
>>>
>>> so that I end up with all rows/ columns that are not NA - e.g.
>>>
>>> ?1 2 3 4 5 7 8 9
>>> 1 0 2 1 2 1 1 2 1
>>> 2 2 0 1 2 1 1 2 1
>>> 3 1 1 0 2 1 1 2 1
>>> 4 2 2 2 0 1 1 2 1
>>> 5 1 1 1 1 0 1 2 1
>>> 7 1 1 1 1 1 0 2 1
>>> 8 2 2 2 2 2 2 0 1
>>> 9 1 1 1 1 1 1 1 0
>>>
>>
>> Try this:
>>
>> ix <- na.action(na.omit(replace(M, upper.tri(M), 0)))
>> M[-ix, -ix]
>
> and here is a minor variation which is slightly shorter:
>
> ix <- complete.cases(replace(M, upper.tri(M), 0))
> M[ix, ix]
>
Please keep all follow ups on the original thread.

Here is a greedy algorithm to iteratively drop the column and row with
the most NAs.

dropNA <- function(M) {
   while(any(is.na(M))) {
      ix <- which.max(colSums(is.na(M)))
      M <- M[-ix, -ix]
   }
   M
}

dropNA(M)

-- 
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com

Hello,

Try

while(TRUE){
     ix <- apply(M, 2, function(x) sum(is.na(x)))
     if(all(ix == 0)) break
     ix <- max(which(ix == max(ix)))
     M <- M[-ix , -ix]
}
M

Note that in the original there's really no difference between columns 9 
and 10.
If in the above code you use 'min', column 9 is removed and it's
still a
minimum of removals.
(Like in any case of a tie.)

Hope this helps,

Rui Barradas

Em 21-05-2012 11:00, Nevil Amos <nevil.amos at monash.edu>
escreveu:
> Date: Mon, 21 May 2012 00:17:10 +1000
> From: Nevil Amos<nevil.amos at monash.edu>
> To:r-help at r-project.org
> Subject: [R] removeing only rows/columns with "na" value from
square (
> 	symmetrical ) matrix.
> Message-ID:
> 	<CAGUDtZJOW7x3sjZsnqf7uAQN6gKFg+EMqq=-HBqKgcwgGaJu6g at
mail.gmail.com>
> Content-Type: text/plain
>
> I have some square matrices with na values in corresponding rows and
> columns.
>
> M<-matrix(1:2,10,10)
> M[6,1:2]<-NA
> M[10,9]<-NA
> M<-as.matrix(as.dist(M))
> print (M)
>
>      1  2 3 4 5  6 7 8  9 10
> 1   0  2 1 2 1 NA 1 2  1  2
> 2   2  0 1 2 1 NA 1 2  1  2
> 3   1  1 0 2 1  2 1 2  1  2
> 4   2  2 2 0 1  2 1 2  1  2
> 5   1  1 1 1 0  2 1 2  1  2
> 6  NA NA 2 2 2  0 1 2  1  2
> 7   1  1 1 1 1  1 0 2  1  2
> 8   2  2 2 2 2  2 2 0  1  2
> 9   1  1 1 1 1  1 1 1  0 NA
> 10  2  2 2 2 2  2 2 2 NA  0
>
>
> How do I remove just the row/column pair( in this trivial example row 6 and
> 10 and column 6 and 10) containing the NA values?
>
> so that I end up with all rows/ columns that are not NA - e.g.
>
>    1 2 3 4 5 7 8 9
> 1 0 2 1 2 1 1 2 1
> 2 2 0 1 2 1 1 2 1
> 3 1 1 0 2 1 1 2 1
> 4 2 2 2 0 1 1 2 1
> 5 1 1 1 1 0 1 2 1
> 7 1 1 1 1 1 0 2 1
> 8 2 2 2 2 2 2 0 1
> 9 1 1 1 1 1 1 1 0
>
>
> if i use na omit I lose rows 1,2,6, and 9
> which is not what I want.
>
> thanks
> -- Nevil Amos Molecular Ecology Research Group Australian Centre for 
> Biodiversity Monash University CLAYTON VIC 3800 Australia 
> [[alternative HTML version deleted]]