R help - Jan 2009 - Mystery Error in midnightStandard

I wasn't even aware I was using midnightStandard.  You won't find it in
my
script.

Here is the relevant loop:

date1 = timeDate(charvec = Sys.Date(), format = "%Y-%m-%d")
date1
dow = 3;
for (i in 1:length(V4) ) {
  x = read.csv(as.character(V4[[i]]), header = FALSE, na.strings="");
  y = x[,1];
  year = V2[[i]];
  week = V3[[i]];
  dtstr = sprintf("%i-%i-%i",year,week,dow);
  date2 = timeDate(dtstr, format = "%Y-%U-%w");
  resultsdataframe$dt[[i]] <- difftimeDate(date1,date2,units =
"weeks");
  fp = fitdistr(y,"exponential");
  print(c(V1[[i]],V2[[i]],V3[[i]],fp$estimate,fp$sd));
  print(c(year,week,date2,resultsdataframe$dt[[i]]));
  resultsdataframe$estimate[[i]] <- fp$estimate;
  resultsdataframe$sd[[i]] <- fp$sd;
}

It fails with a little more than 100 records left in V4.

The full error message is:

Error in midnightStandard(charvec, format) :
  'charvec' has non-NA entries of different number of characters

Until it fails, date2 and resultsdataframe$dt[[i]] get correct values.

str() produces no surprises:
> str(resultsdataframe);
'data.frame':    303 obs. of  6 variables:
 $ mid     : int  171 206 206 206 206 206 206 206 206 218 ...
 $ year    : int  2008 2008 2008 2008 2008 2008 2008 2008 2008 2008 ...
 $ week    : int  16 17 18 19 21 26 31 35 51 40 ...
 $ dt      : num  39.9 38.9 37.9 36.9 34.9 ...
 $ estimate: num  Inf 0.25 Inf 0.0408 0.2 ...
 $ sd      : num  Inf 0.1768 Inf 0.0289 0.1414 ...

I would assume the error is related to my new code that manipulates dates,
as it doesn't occur in the earlier version that did not manipulate dates
(the relevant work being done, albeit very slowly, within the DB).

FTR: The year and week values are generated by MySQL using the YEAR and WEEK
functions applied to timestamps.  I do not know if it is relevant, but the
week value, at the point of failure, is 0 (a value that does not occur
earlier in the dataset, but several times subsequently), and I do not see
how a value of 0 for the week (legitimate in posix date formats) could
produce the error message I get.

Any thoughts on what is really wrong, and how to fix it?

Thanks

Ted

	[[alternative HTML version deleted]]

>>>> "TB" == Ted Byers <r.ted.byers at gmail.com>
>>>> on Tue, 27 Jan 2009 16:00:27 -0500
   TB> I wasn't even aware I was using midnightStandard.  You won't
   TB> find it in my
   TB> script.
   TB>
   TB> Here is the relevant loop:
   TB>
   TB> date1 = timeDate(charvec = Sys.Date(), format = %Y-%m-%d)
   TB> date1
   TB> dow = 3;
   TB> for (i in 1:length(V4) ) {
   TB> x = read.csv(as.character(V4[[i]]), header = FALSE,
   TB> na.strings=);
   TB> y = x[,1];
   TB> year = V2[[i]];
   TB> week = V3[[i]];
   TB> dtstr = sprintf(%i-%i-%i,year,week,dow);
   TB> date2 = timeDate(dtstr, format = %Y-%U-%w);
   TB> resultsdataframe[[i]] <- difftimeDate(date1,date2,units    TB>
weeks);
   TB> fp = fitdistr(y,exponential);
   TB> print(c(V1[[i]],V2[[i]],V3[[i]],fp,fp));
   TB> print(c(year,week,date2,resultsdataframe[[i]]));
   TB> resultsdataframe[[i]] <- fp;
   TB> resultsdataframe[[i]] <- fp;
   TB> }
   TB>
   TB> It fails with a little more than 100 records left in V4.
   TB>
   TB> The full error message is:
   TB>
   TB> Error in midnightStandard(charvec, format) :
   TB> 'charvec' has non-NA entries of different number of characters

timeDate() uses the midnight standard. The function 'midnightStandard'
assumes that all entries in 'charvec' have the same 'format'.
Can you
please check if this is the case?

This is all I can say from the information you provided. Please give us
a reproducible example.

We can continue this discussion off-list.

regards,
Yohan

   TB>
   TB> Until it fails, date2 and resultsdataframe[[i]] get correct
   TB> values.
   TB>
   TB> str() produces no surprises:
   TB>
   TB> > str(resultsdataframe);
   TB> 'data.frame': 303 obs. of 6 variables:
   TB> $ mid : int 171 206 206 206 206 206 206 206 206 218 ...
   TB> $ year : int 2008 2008 2008 2008 2008 2008 2008 2008 2008
   TB> 2008 ...
   TB> $ week : int 16 17 18 19 21 26 31 35 51 40 ...
   TB> $ dt : num 39.9 38.9 37.9 36.9 34.9 ...
   TB> $ estimate: num Inf 0.25 Inf 0.0408 0.2 ...
   TB> $ sd : num Inf 0.1768 Inf 0.0289 0.1414 ...
   TB>
   TB> I would assume the error is related to my new code that
   TB> manipulates dates,
   TB> as it doesn't occur in the earlier version that did not
   TB> manipulate dates
   TB> (the relevant work being done, albeit very slowly, within
   TB> the DB).
   TB>
   TB> FTR: The year and week values are generated by MySQL using
   TB> the YEAR and WEEK
   TB> functions applied to timestamps.  I do not know if it is
   TB> relevant, but the
   TB> week value, at the point of failure, is 0 (a value that does
   TB> not occur
   TB> earlier in the dataset, but several times subsequently),
   TB> and I do not see
   TB> how a value of 0 for the week (legitimate in posix date
   TB> formats) could
   TB> produce the error message I get.
   TB>
   TB> Any thoughts on what is really wrong, and how to fix it?
   TB>
   TB> Thanks
   TB>
   TB> Ted




-- 
PhD student
Swiss Federal Institute of Technology
Zurich

www.ethz.ch