R devel - Sep 2007 - Date vs date (long)

Peter et al

  Thanks for the comments on dates.  Some of the respondents missed the point,
by showing ways that I could work around the problems, when my main argument
is that one shouldn't have to work around problems.  So I hereto present
round 2 of the debate.

 1 Postulates

   a. In my 35 year computing experience, I think that nothing frustrates me
more than a computer program that tries to keep me from doing something
"for my own protection", when I know quite well what I am doing.  So
postulate
1 is a Bayesian sort of thing: the loss function is so large (hopping mad
user) that one should be very cautious about creating a taboo.

   b. The S language's primary success is as a tool.  Tools get used in ways
that the originator never thought of.  Alternate use is not wrong --- in
fact you want to foster it.  (My farm backround plays a role here.  You
wouldn't
believe the number of things I've fixed with a hammer and/or wrench, when
the
goal was not to get it done "right", but just to get whatever done and
get the
crop in.)

  
 2 Key question

  Both a data and a time-span object consist of a numeric value along with
ancillary information about how to interpret that value.  For simplicity call
the latter "attributes" (ignoring whether they are implemented using
the attr
function or slots or whatever).
  For some operations is is fairly clear what do to with both the attr and the
numeric part, e.g., date + 1 is the next day.  No problem here.

  For other operations, e.g., timespan^2 it is only clear that the result is no
longer a timespan, but not what class it should be.  I firmly believe that the
right result is to toss the attribute and return the number.  This makes
the tool optimally useful.  Returning an error message is an unneccesary
and controlling  response: what good did the "not legal" message do
me?
There are of course many cases where an error message is the only choice,
because I can't see what to do with either the number or the attribute, 
e.g.  date + string.

  The key question is then "what is the right philosphy", flexible
tool or
rigorous control?  Rigorous control languages have not fared well historically.

3 Hard cases

  The hardest are cases where the right return value is unclear.  An example
is (date + 1.73) : should one return a true date, which is integer, allow
an invalid internal value that is "fixed" at print time, return a
numeric,
or an error message?

   I put (timespan/constant) in this category.  The author has no hint as to
whether the constant is unitless or not.  In the medical research environment
converstions back and forth from days to months and years are very common,
greatly outmassing division of an iterval into pieces, so if I had to guess
I would assume that I had to drop the units; another environment might be
just the opposite.

4 Response to particular points:

Peter D, 9/14
  a. as.Date(x)
Peter suggests (as.Date('1960-1-1') + x).  This is a really good idea,
as it
makes the code both origin independent and clearer.

  b. "I'd advise against numeric operation on difftime objects in
general,
because of the unspecified units."
  If I carry this idea forward, the R should insist that I specify units for
any variable that corresponds to a physical quantity, e.g. "height" or
"weight", so that it can slap my hands with an error message when I
type

	bodyMassIndex = weight/ height^2

or cause plot(height^2, weight) to fail.  This would go a long way towards
making R the most frustrating program available.  (An Microsoft gives some
stiff competition in that area!)

 c. 
"It is assumed that the divisor is unit-less. 
Convert to numeric first to avoid this. (The idea has been raised to 
introduce new units: epiyears and epimonths, in which case you might do

x <- as.Date('2007-9-14') - as.Date('1953-3-10')
units(x) <- "epiyears"

which would give you the age in years for those purposes where you don't 
care missing the exact birthday by a day or so.)"

   As I said, division is a hard case with no clear answer.  The creation of
other unit schemes is silly --- why in the world would I voluntarily put on
a straightjacket?

d. 
>> as.Date('09Sep2007')
>>     
> Error in fromchar(x) : character string is not in a standard unambiguous 
format

  My off-the-cuff suggestion is to make the message honest
	Error in fromchar(x): program is not able to divine the correct format

The problem is not that the format is necessarily wrong or ambiguous, but that
the program can't guess.  (Which is no real fault of the program - such 
a recognition is a hard problem.  It's ok to ask me for a format string).

--
Hadley Wickham
"Why not just always use seconds for difftime objects?  An attribute
could control how it was formatted, but would be independent of the
underlying representation."

  This misses the point.

-------

Gabor Grothendieck 

as.Date(10)
You can define as.Date.numeric in your package and then it will work.  zoo
has done that.

library(zoo)
as.Date(10)

  This is also a nice idea.  Although adding to a package is possible, it is
now very hard to take away, given namespaces.  That is, I can't define my
own Math.Date to do away with the creation of timespan objects.  Am I
correct?  Is it also true that adding methods is hard if one uses version 4
classes?

  The rest of Gabor's comments are workarounds for the problem I raised.
But I don't want to have to wrap "as.numeric" around all of my
date
calculations.

-----------
Brian Ripley

"It fails by design.  Using sqrt() on a measurement that has an arbitrary 
origin would not have been good design."

  Ah, the classic Unix response of "that's not a bug, it's a
feature".

  What is interesting is that this is almost precisely the response I
got when I first argued for a global na.action default.  John C (I think)
replied that, essentially, S SHOULD slap you alonside the head when
there were missing values.  They require careful thought wrt good analysis,
and allowing a global option was bad design because it would encourage bad
statistics.  The Insightful side of the debate said they didn't dare because
is might break something.  After getting nowhere with talking I finally
gave up and wrote my own version into the survival code.  This leverage 
eventually forced adoption of the idea.
   Not many (any?) people currently set na.action=na.fail because it is
a "better design". 
------------------

Historically, languages designed for other people to use have been bad: Cobol,
PL/I, Pascal, Ada, C++. The good languages have been those that were designed 
for their own creators: C, Perl, Smalltalk, Lisp. (Paul Graham)

   


   Terry Therneau

Terry Therneau wrote:
>   b. "I'd advise against numeric operation on difftime objects in
general,
> because of the unspecified units."
>   If I carry this idea forward, the R should insist that I specify units
for
> any variable that corresponds to a physical quantity, e.g.
"height" or
> "weight", so that it can slap my hands with an error message when
I type
>
> 	bodyMassIndex = weight/ height^2
>
> or cause plot(height^2, weight) to fail.  This would go a long way towards
> making R the most frustrating program available.  (An Microsoft gives some
> stiff competition in that area!)
>   
That's not the point. The point is that 2 weeks is 14 days, so do you 
want sqrt(2) or sqrt(14)? It is not my design to have this 
variable-units encoding of difftimes, but as it is there, it  is better 
to play along than to pretend that it is something else. (Once you go to 
faster time scales than in epidemiology, this becomes quite crucial 
because the units chosen can depend on the actual differences computed!)
>  c. 
> "It is assumed that the divisor is unit-less. 
> Convert to numeric first to avoid this. (The idea has been raised to 
> introduce new units: epiyears and epimonths, in which case you might do
>
> x <- as.Date('2007-9-14') - as.Date('1953-3-10')
> units(x) <- "epiyears"
>
> which would give you the age in years for those purposes where you
don't
> care missing the exact birthday by a day or so.)"
>
>    As I said, division is a hard case with no clear answer.  The creation
of
> other unit schemes is silly --- why in the world would I voluntarily put on
> a straightjacket?
>   
We'll put it on for you...

It makes sense to calculate half a difftime or a 12th or a 100th of a 
difftime. You were asking the system to magically conclude that a 
365.25th of a difftime has a different meaning, a units conversion. This 
is the sort of thing that humans can discern, but not machines. The 
design is that you change units by using units(x)<-. Unfortunately the 
largest regular unit is "weeks", hence the suggestion of
"epiyears".
> d. 
>   
>>> as.Date('09Sep2007')
>>>     
>>>       
>> Error in fromchar(x) : character string is not in a standard
unambiguous
>>     
> format
>
>   My off-the-cuff suggestion is to make the message honest
> 	Error in fromchar(x): program is not able to divine the correct format
>   
Heh. Pretty close.  Now what is a suitable eufemism for
"divine"?
> The problem is not that the format is necessarily wrong or ambiguous, but
that
> the program can't guess.  (Which is no real fault of the program - such
> a recognition is a hard problem.  It's ok to ask me for a format
string).
>
> --
> Hadley Wickham
> "Why not just always use seconds for difftime objects?  An attribute
> could control how it was formatted, but would be independent of the
> underlying representation."
>
>   This misses the point.
>   
No. It _is_ the point.  The design is that the numeric value of a 
difftime is nonsensical without knowing the units. This might be 
different, although as Brian indicated, the choice is deliberate, and 
some deep thinking was involved.
> -------
>
> Gabor Grothendieck 
>
> as.Date(10)
> You can define as.Date.numeric in your package and then it will work.  zoo
> has done that.
>
> library(zoo)
> as.Date(10)
>
>   This is also a nice idea.  Although adding to a package is possible, it
is
> now very hard to take away, given namespaces.  That is, I can't define
my
> own Math.Date to do away with the creation of timespan objects.  Am I
> correct?  Is it also true that adding methods is hard if one uses version 4
> classes?
>
>   The rest of Gabor's comments are workarounds for the problem I
raised.
> But I don't want to have to wrap "as.numeric" around all of
my date
> calculations.
>
>   
Just get used to it, I'd say.
> -----------
> Brian Ripley
>
> "It fails by design.  Using sqrt() on a measurement that has an
arbitrary
> origin would not have been good design."
>
>   Ah, the classic Unix response of "that's not a bug, it's a
feature".
>
>   What is interesting is that this is almost precisely the response I
> got when I first argued for a global na.action default.  John C (I think)
> replied that, essentially, S SHOULD slap you alonside the head when
> there were missing values.  They require careful thought wrt good analysis,
> and allowing a global option was bad design because it would encourage bad
> statistics.  The Insightful side of the debate said they didn't dare
because
> is might break something.  After getting nowhere with talking I finally
> gave up and wrote my own version into the survival code.  This leverage 
> eventually forced adoption of the idea.
>    Not many (any?) people currently set na.action=na.fail because it is
> a "better design". 
> ------------------
>
> Historically, languages designed for other people to use have been bad:
Cobol,
> PL/I, Pascal, Ada, C++. The good languages have been those that were
designed
> for their own creators: C, Perl, Smalltalk, Lisp. (Paul Graham)
>   
Each of those that I know had its share of trouble with users who relied 
on implementation details and protested loudly when their programs 
stopped working. (In the C case, relying on function arguments being 
stored consecutively on a stack was a common source of grief when 
porting programs to SPARC machines, for instance. Or assuming that 
doubles could be stored starting at arbitrary bytes. Or that pointers 
could be stored in long integers.)
>    
>
>
>    Terry Therneau
>   

-- 
   O__  ---- Peter Dalgaard             ?ster Farimagsgade 5, Entr.B
  c/ /'_ --- Dept. of Biostatistics     PO Box 2099, 1014 Cph. K
 (*) \(*) -- University of Copenhagen   Denmark          Ph:  (+45) 35327918
~~~~~~~~~~ - (p.dalgaard at biostat.ku.dk)                  FAX: (+45) 35327907

On 9/17/07, Terry Therneau <therneau at mayo.edu>
wrote:
> Gabor Grothendieck
>
> as.Date(10)
> You can define as.Date.numeric in your package and then it will work.  zoo
> has done that.
>
> library(zoo)
> as.Date(10)
>
>  This is also a nice idea.  Although adding to a package is possible, it is
> now very hard to take away, given namespaces.  That is, I can't define
my
> own Math.Date to do away with the creation of timespan objects.  Am I
> correct?  Is it also true that adding methods is hard if one uses version 4
> classes?
>
>  The rest of Gabor's comments are workarounds for the problem I raised.
> But I don't want to have to wrap "as.numeric" around all of
my date
> calculations.
You can define as.Date.numeric and Ops.Date, say, using S3 and these
will be added to the whatever is there but won't override the existing
+.Date and -.Date nor would you want them to or else the behavior would
be different depending on whether your package was there or not.  Also
namespaces should not be a problem since zoo uses namespaces and
it defined its own as.Date.numeric.

Try this:

Ops.Date <- function (e1, e2) {
    e <- if (missing(e2)) {
        NextMethod(.Generic)
    }
    else if (any(nchar(.Method) == 0)) {
        NextMethod(.Generic)
    }
    else {
        e1 <- as.numeric(e1)
	e2 <- as.numeric(e2)
        NextMethod(.Generic)
    }
    e
}

Sys.Date() / Sys.Date()
Sys.Date() + as.numeric(Sys.Date())
as.numeric(Sys.Date()) + as.numeric(Sys.Date())

Sys.Date() + Sys.Date() # error since its intercepted by +.Date

Thus you will have to issue some as.numeric calls but perhaps not too
many.

However, I think its better not to implement Ops.Date as above but
just leave the Date operations the way they are, extend it with
as.Date.numeric like zoo has done and force the user to use as.numeric
in other cases to make it clear from the code that  there is conversion
going on.  I have done a fair amount of Date manipulation and have not
found the as.numeric to be onerous.