R help - Sep 2011 - Results of applying na.omit on zoo object

I read in ?na.omit that it returns the object with incomplete cases
removed. I interpret this to mean that any zoo object row where any column
shows 'NA' will be removed from the data set.

   That's not what I need, since the 'NA' represents information in
my
context. However, what I would like to do is eliminate the rows where every
column is 'NA'. When I aggregate two factors by date in the zoo object,
the
rows being with the earliest possible date even if that's not appropriate
for the row factor pair.

   Is there a way to omit only those rows where all columns contain
'NA'?

Rich

On Sep 19, 2011, at 11:48 AM, Rich Shepard wrote:
>  I read in ?na.omit that it returns the object with incomplete cases
> removed. I interpret this to mean that any zoo object row where any column
> shows 'NA' will be removed from the data set.
> 
>  That's not what I need, since the 'NA' represents information
in my
> context. However, what I would like to do is eliminate the rows where every
> column is 'NA'. When I aggregate two factors by date in the zoo
object, the
> rows being with the earliest possible date even if that's not
appropriate
> for the row factor pair.
> 
>  Is there a way to omit only those rows where all columns contain
'NA'?
> 
> Rich


You can look at ?complete.cases for one approach, presuming that it will work on
zoo objects.

HTH

Marc Schwartz

On Mon, 19 Sep 2011, Rich Shepard wrote:
> When I aggregate two factors by date in the zoo object, the rows being ...
                                                                    ^^^^^

   That should be 'begin'.

Mea culpa!

Rich

On Sep 19, 2011, at 12:19 PM, Marc Schwartz wrote:
> 
> On Sep 19, 2011, at 12:14 PM, Marc Schwartz wrote:
> 
>> 
>> On Sep 19, 2011, at 12:01 PM, Rich Shepard wrote:
>> 
>>> On Mon, 19 Sep 2011, Marc Schwartz wrote:
>>> 
>>>> You can look at ?complete.cases for one approach, presuming
that it will
>>>> work on zoo objects.
>>> 
>>> Marc,
>>> 
>>> That's the opposite of what I want. It returns only rows with
no missing
>>> data. I'm looking for something that will return rows with
_only_ missing
>>> data, and drop them in the bit bucket.
>>> 
>>> Thanks,
>>> 
>>> Rich
>> 
>> 
>> Rich,
>> 
>> OK, I mis-read your post.
>> 
>> Depending upon the underlying structure/class of the zoo object (eg.
matrix versus data frame) and presuming that there are no functions in zoo that
provide this specific functionality, you may have to create a function that goes
row-by-row looking for all NA's.
>> 
>> Possibly something along the lines of the following for a matrix:
>> 
>> zoo.object[apply(zoo.object, 1, function(x) all(is.na(x))), ]
>> 
>> 
>> That won't work for a data frame class object, so you might have to
loop over the rows with a for() loop or sapply():
>> 
>> zoo.object[sapply(rownames(zoo.object), function(x) all(is.na(x))), ]
>> 
>> 
>> Both of the above are untested.
Rich,

One more time?re-post:

I forgot a critical '!' in both. They should be

 zoo.object[!apply(zoo.object, 1, function(x) all(is.na(x))), ]

and the DF approach is not correct either. It should be:

  zoo.object[!sapply(rownames(zoo.object), function(x) all(is.na(zoo.object[x,
]))), ]

I clearly have not had enough caffeine yet today?

Marc

On Mon, Sep 19, 2011 at 8:34 PM, Rich Shepard <rshepard at
appl-ecosys.com> wrote:
> On Mon, 19 Sep 2011, Gabor Grothendieck wrote:
>
>> names(z) gives the column names of zoo object z.
>
>> # get a list of date ranges
>> lapply(1:ncol(z), function(i) range(time(na.omit(z[, i]))))
>
If z has column names (but do not use this if it does not) then the
following improvement will display them in the output:
> sapply(names(z), function(nm) range(time(na.omit(z[, nm]))), simplify =
FALSE)
$a
[1] "2011-09-21" "2011-09-23"

$b
[1] "2011-09-22" "2011-09-24"



-- 
Statistics & Software Consulting
GKX Group, GKX Associates Inc.
tel: 1-877-GKX-GROUP
email: ggrothendieck at gmail.com