thr3ads.net - R help - [R] Problem with strptime generating missing values where none appear to exist [Feb 2010]

If this information is useful, please help other people find it:
Share via:

Jonathan Williams

2010-Feb-23 18:26 UTC

[R] Problem with strptime generating missing values where none appear to exist

Dear R Helpers,

I am having difficulty with strptime. I wish to find the differences between
two vectors of times. I have apparently no difficulty to convert the vectors
to the appropriate format using strptime. But, then difftime does not
calculate all the differences.

Here is the code and output:-

dob=strptime(as.character(datx$BDT),'%d-%b-%y'); dob$year=dob$year-100
sdate=strptime(as.character(datx$SDT),'%d-%b-%y')
head(dob); head(sdate)
[1] "1922-07-14" "1922-07-14" "1922-07-14"
"1922-07-14" "1921-03-23"
"1921-03-23"
[1] "2001-08-27" "2001-08-27" "2001-08-27"
"2001-08-27" "2001-08-20"
"2001-08-20"> str(dob) POSIXlt[1:9], format: "1922-07-14" "1922-07-14"
"1922-07-14" "1922-07-14"
"1921-03-23" "1921-03-23" "1921-03-23"
"1927-08-27" "1927-08-27"
"1927-08-27" "1927-08-27" "1940-04-05"
"1940-04-05" "1940-04-05"
"1940-04-05" ...> str(sdate) POSIXlt[1:9], format: "2001-08-27" "2001-08-27"
"2001-08-27" "2001-08-27"
"2001-08-20" "2001-08-20" "2001-08-20"
"2001-11-26" "2001-11-26"
"2001-11-26" "2001-11-26" "2002-05-20"
"2002-05-20" "2002-05-20"
"2002-05-20" ...

table(is.na(sdate))

FALSE 
  812 

table(is.na(dob))

FALSE  TRUE 
  743    69 
But if I now look at each component of dob separately, none is missing

for (i in 1:length(dob)) {print(names(dob)[i]);
print(table(is.na(dob[[i]])))}

[1] "sec"

FALSE 
  812 
[1] "min"

FALSE 
  812 
[1] "hour"

FALSE 
  812 
[1] "mday"

FALSE 
  812 
[1] "mon"

FALSE 
  812 
[1] "year"

FALSE 
  812 
[1] "wday"

FALSE 
  812 
[1] "yday"

FALSE 
  812 
[1] "isdst"

FALSE 
  812 

Additionally, there are no NA values in any component of dob on direct
visual inspection. For example, here is dob$mon
dob$mon
[1]  6  6  6  6  2  2  2  7  7  7  7  3  3  3  3 11 11 11 11  7  7  7  7  7
7  7  7  4  4  4  4  4  4  4  4 11 11 11 11  7  7  7  7 11 11 11 11  6  6  6
6  9  9  9  9  3  3  3  3  6  6  6  6  8  8  8  8  7  7  7  7  4  4  4  4  4
 [77]  4  4  4  7  7  7  7 10 10 11 11 11 11  8  8  8  8  0  0  0  0  0 10
10 10  7  7  7  7  3  3  3  3  2  2  2  2  2  6  6  6  6  6  5  5  5  5  4
4  4  4  4 11 11 11 11  4  4  4  4  4  3  3  3  3  3  7  7  7  7  7  7  7  7
8  8
[153]  8  8  8  8  8  8  7  7  7  7  6  6 10 10 10 10  4  4  4  4  4  4  4
4 10 10 10 10 11 11 11 11  5  5  5  5  5  5  5  5  3  3  3  3  5  5  0  0  0
0  2  2  2  2  6  6  6  6  0  0  0  0  3  3  3  3  6  6  6  6  8  8  8  8  8
7
[229]  7  7  7  7  7  7  7  7  8  8  8  8  4  4  4  4 10 10 10 10  2  2  2
2  0  0  0  0  0  0  1  1  1  1  4  4  4  4  2  2  2  2  2  8  8  8  8 11 11
11 11  8  8  8  8  4  4  4  4  5  5  5  5  8  8  8  8  0  0  0  0  1  1  1
1  1
[305]  1  1  1  4  4  4  4  5  5  5  5  7  7  7  7  5  5  5  5  3  3  3  3
1  1  1  1  0  0  0  0  3  3  3  3  6  6  6  6  3  3  3  5  5 11 11 11  5  5
5  5  0  0  0  0 10 10 10 10  4  4  4  4  6  6  6  6  7  7  7  7  4  4  4  4
1
[381]  1  1  1  7  7  7  7  3  3  3  3  7  7  7  7  5  5  5  5  9  9  9  9
11 11 11 11 10 10 10 10  0  0  0  0  5  5  5  5  3  3  3  3  7  7  7  7  0
0  0  0  6  6  6  6  8  8  8  8  8  8  8  8  3  3  3  3  5  5  5  5 10 10 10
10  3
[457]  3  3  3  8  8  8  8  0  0  0  0 11 11 11 11  2  2  2  2  7  7  7  7
0  0  0  0  0  1  1  1  1  5  5  5  5  7  7  7  7  7  7  7  7  5  5  5  5  9
9  9  9  5  5  5  5  6  6  6  6  8  8  8  8 11 11 11 11  3  3  3  3  6  6  6
6
[533]  3  3  3  3  6  6  6  6  8  8  8  8  9  9  9  9  2  2  2  2  1  1  1
1  2  2  2  2  4  4  4  7  7  7  8  8  8  8  3  3  3  3  1  1  1  1  1  9  9
9  9  8  8  8  8 11 11 11 11  6  6  6  6  3  3  3  3 10 10 10  8  8  8  0  0
0
[609]  0  3  3  3  3  3  0  0  0  0  3  3  3  3  5  5  5  5 10 10 10 10 10
10 10 10  2  2  2  2  2  3  3  3  3  4  4  4 10 10 10 10  2  2  2  2  3  3
3  3  2  2  2  2  2  2  6  6  6  6  4  4  4  4 11 11 11 11  0  0  0  0 11 11
11 11
[685]  5  5  5  5  8  8  8  8  8  8  8  8  7  7  7  7  3  3  3  3  5  5  5
5 11 11 11 11  3  3  3  9  9  5  5  5  5  8  8  8  8  2  2  2  2  5  5  5  5
2  2  2  2 10 10 10 10  4  4  4 11 11 11 11  8  8  8  9  9  9  1  1  1  1  8
8
[761]  8  8  2  2  2 11 11 11 11  2  2  2  2  2  2  2  2  6  6  6  6 11 11
11 11  2  2  2 11 11 11  9  9  9  9  2  2  2  2  7  7  7  7 11 11 11  2  2
2  3  3  3

All the dob components are equally complete, including isdst.

However, when I then try to compute difftime(sdate,dob), 69 values are
missing:-
Time differences in days
  [1] 28899.00 28899.00 28899.00 28899.00 29369.96 29369.96 29369.96
27120.04 27120.04 27120.04 27120.04 22690.00 22690.00 22690.00 22690.00
28905.00 28905.00 28905.00 28905.00 31207.04 31207.04 31207.04 31207.04
31209.04 31209.04
 [26] 31209.04 31209.04 26323.00 26323.00 26323.00 26323.00 26338.00
26338.00 26338.00 26338.00 27310.96 27310.96 27310.96 27310.96 23588.04
23588.04 23588.04 23588.04 25255.00 25255.00 25255.00 25255.00 23752.00
23752.00 23752.00
 [51] 23752.00 29607.04 29607.04 29607.04 29607.04 27993.04 27993.04
27993.04 27993.04 28384.04 28384.04 28384.04 28384.04 26176.00 26176.00
26176.00 26176.00 28986.04 28986.04 28986.04 28986.04 28689.04 28689.04
28689.04 28689.04
 [76] 23722.00 23722.00 23722.00 23722.00 27353.00 27353.00 27353.00
27353.00 26303.00 26303.00 28803.96 28803.96 28803.96 28803.96 28564.04
28564.04 28564.04 28564.04 29826.96 29826.96 29826.96 29826.96 29826.96
30410.00 30410.00
[101] 30410.00 26490.04 26490.04 26490.04 26490.04       NA       NA
NA       NA 29765.96 29765.96 29765.96 29765.96 29765.96 26325.00 26325.00
26325.00 26325.00 26325.00 28824.00 28824.00 28824.00 28824.00 26808.00
26808.00
[126] 26808.00 26808.00 26808.00 28628.96 28628.96 28628.96 28628.96
23807.00 23807.00 23807.00 23807.00 23807.00       NA       NA       NA
NA       NA 25668.04 25668.04 25668.04 25668.04 28654.04 28654.04 28654.04
28654.04
[151] 21711.04 21711.04 21711.04 21711.04 27167.04 27167.04 27167.04
27167.04 24296.04 24296.04 24296.04 24296.04 30540.04 30540.04 25330.00
25330.00 25330.00 25330.00 25579.00 25579.00 25579.00 25579.00 29127.04
29127.04 29127.04
[176] 29127.04 29896.96 29896.96 29896.96 29896.96 25992.00 25992.00
25992.00 25992.00 26625.00 26625.00 26625.00 26625.00 30121.04 30121.04
30121.04 30121.04 21801.04 21801.04 21801.04 21801.04 31274.04 31274.04
25907.00 25907.00
[201] 25907.00 25907.00 28516.00 28516.00 28516.00 28516.00 28943.00
28943.00 28943.00 28943.00 29847.96 29847.96 29847.96 29847.96 30529.04
30529.04 30529.04 30529.04 30527.04 30527.04 30527.04 30527.04 29434.00
29434.00 29434.00
[226] 29434.00 29434.00 28631.04 28631.04 28631.04 28631.04 25761.04
25761.04 25761.04 25761.04 25761.04 26127.04 26127.04 26127.04 26127.04
26027.00 26027.00 26027.00 26027.00 28987.00 28987.00 28987.00 28987.00
29232.00 29232.00
[251] 29232.00 29232.00 26109.96 26109.96 26109.96 26109.96 31339.00
31339.00 29235.00 29235.00 29235.00 29235.00 28092.00 28092.00 28092.00
28092.00 30209.00 30209.00 30209.00 30209.00 30209.00 30281.00 30281.00
30281.00 30281.00
[276] 26880.96 26880.96 26880.96 26880.96 25691.04 25691.04 25691.04
25691.04 22938.04 22938.04 22938.04 22938.04 25878.00 25878.00 25878.00
25878.00 24470.00 24470.00 24470.00 24470.00 26046.96 26046.96 26046.96
26046.96 26763.96
[301] 26763.96 26763.96 26763.96 25720.96 25720.96 25720.96 25720.96
29214.00 29214.00 29214.00 29214.00 26992.00 26992.00 26992.00 26992.00
30659.00 30659.00 30659.00 30659.00 25600.00 25600.00 25600.00 25600.00
26842.00 26842.00
[326] 26842.00 26842.00 25541.00 25541.00 25541.00 25541.00 27386.00
27386.00 27386.00 27386.00 30302.04 30302.04 30302.04 30302.04 28059.00
28059.00 28059.00 28059.00       NA       NA       NA 25657.00 25657.00
NA       NA
[351]       NA 24835.00 24835.00 24835.00 24835.00 29340.96 29340.96
29340.96 29340.96 26473.96 26473.96 26473.96 26473.96 28873.00 28873.00
28873.00 28873.00 27690.00 27690.00 27690.00 27690.00 26554.00 26554.00
26554.00 26554.00
[376] 28876.00 28876.00 28876.00 28876.00 27156.96 27156.96 27156.96
27156.96 26577.00 26577.00 26577.00 26577.00 27471.00 27471.00 27471.00
27471.00 27323.00 27323.00 27323.00 27323.00 29232.00 29232.00 29232.00
29232.00       NA
[401]       NA       NA       NA 26523.96 26523.96 26523.96 26523.96
26538.96 26538.96 26538.96 26538.96 24374.96 24374.96 24374.96 24374.96
30798.00 30798.00 30798.00 30798.00       NA       NA       NA       NA
22775.04 22775.04
[426] 22775.04 22775.04 28464.00 28464.00 28464.00 28464.00 25763.04
25763.04 25763.04 25763.04 30114.04 30114.04 30114.04 30114.04 26864.04
26864.04 26864.04 26864.04       NA       NA       NA       NA 26945.04
26945.04 26945.04
[451] 26945.04 29528.96 29528.96 29528.96 29528.96 29058.04 29058.04
29058.04 29058.04 29456.00 29456.00 29456.00 29456.00 26450.96 26450.96
26450.96 26450.96 22837.96 22837.96 22837.96 22837.96 24222.96 24222.96
24222.96 24222.96
[476] 29592.00 29592.00 29592.00 29592.00 26573.00 26573.00 26573.00
26573.00 26573.00 24811.00 24811.00 24811.00 24811.00 24834.00 24834.00
24834.00 24834.00 31312.00 31312.00 31312.00 31312.00 23337.00 23337.00
23337.00 23337.00
[501] 26422.00 26422.00 26422.00 26422.00 22664.04 22664.04 22664.04
22664.04 23192.04 23192.04 23192.04 23192.04 27557.04 27557.04 27557.04
27557.04 23449.04 23449.04 23449.04 23449.04 27799.00 27799.00 27799.00
27799.00 28747.04
[526] 28747.04 28747.04 28747.04 24660.04 24660.04 24660.04 24660.04
NA       NA       NA       NA 24683.04 24683.04 24683.04 24683.04 26576.00
26576.00 26576.00 26576.00       NA       NA       NA       NA 28897.96
28897.96
[551] 28897.96 28897.96 25997.96 25997.96 25997.96 25997.96 24594.96
24594.96 24594.96 24594.96 25965.00 25965.00 25965.00 30139.04 30139.04
30139.04 26104.04 26104.04 26104.04 26104.04 26255.04 26255.04 26255.04
26255.04 28887.00
[576] 28887.00 28887.00 28887.00 28887.00       NA       NA       NA
NA 25470.00 25470.00 25470.00 25470.00 20677.96 20677.96 20677.96 20677.96
29227.00 29227.00 29227.00 29227.00       NA       NA       NA       NA
29543.96
[601] 29543.96 29543.96 31080.00 31080.00 31080.00 27710.00 27710.00
27710.00 27710.00       NA       NA       NA       NA       NA 29903.00
29903.00 29903.00 29903.00       NA       NA       NA       NA 24147.00
24147.00 24147.00
[626] 24147.00 23316.96 23316.96 23316.96 23316.96 27096.00 27096.00
27096.00 27096.00 25543.00 25543.00 25543.00 25543.00 25543.00       NA
NA       NA       NA 25131.04 25131.04 25131.04 29565.96 29565.96 29565.96
29565.96
[651] 28070.00 28070.00 28070.00 28070.00 28774.04 28774.04 28774.04
28774.04 28073.00 28073.00 28130.00 28130.00 28130.00 28130.00 20038.00
20038.00 20038.00 20038.00 27298.04 27298.04 27298.04 27298.04 27793.00
27793.00 27793.00
[676] 27793.00 25586.00 25586.00 25586.00 25586.00 26000.00 26000.00
26000.00 26000.00 30577.04 30577.04 30577.04 30577.04 27194.04 27194.04
27194.04 27194.04 23156.04 23156.04 23156.04 23156.04 23978.04 23978.04
23978.04 23978.04
[701]       NA       NA       NA       NA 24391.00 24391.00 24391.00
24391.00 27152.96 27152.96 27152.96 27152.96 28852.00 28852.00 28852.00
25419.00 25419.00 29212.00 29212.00 29212.00 29212.00 23660.00 23660.00
23660.00 23660.00
[726] 26022.96 26022.96 26022.96 26022.96 25566.00 25566.00 25566.00
25566.00 25336.96 25336.96 25336.96 25336.96 26931.96 26931.96 26931.96
26931.96 26758.00 26758.00 26758.00 26537.96 26537.96 26537.96 26537.96
27026.00 27026.00
[751] 27026.00       NA       NA       NA 24349.96 24349.96 24349.96
24349.96 25960.00 25960.00 25960.00 25960.00 27276.00 27276.00 27276.00
26826.96 26826.96 26826.96 26826.96 26428.96 26428.96 26428.96 26428.96
26780.96 26780.96
[776] 26780.96 26780.96 26301.00 26301.00 26301.00 26301.00 28385.96
28385.96 28385.96 28385.96 27210.96 27210.96 27210.96 23704.00 23704.00
23704.00 24160.04 24160.04 24160.04 24160.04 25703.96 25703.96 25703.96
25703.96 25269.00
[801] 25269.00 25269.00 25269.00 29886.96 29886.96 29886.96       NA
NA       NA       NA       NA       NA
attr(,"tzone")
[1] ""

Here are the values of sdate and dob that relate to the missing values in
difftime(sdate,dob)
> sdate[is.na(difftime(sdate,dob))] [1] "2002-02-28" "2002-02-28" "2002-02-28"
"2002-02-28" "2002-07-30"
"2002-07-30" "2002-07-30" "2002-07-30"
"2002-07-30" "2003-06-17"
"2003-06-17" "2003-06-17" "2003-10-30"
"2003-10-30" "2003-10-30"
"2002-07-22" "2002-07-22"
[18] "2002-07-22" "2002-07-22" "2002-12-18"
"2002-12-18" "2002-12-18"
"2002-12-18" "2003-03-10" "2003-03-10"
"2003-03-10" "2003-03-10"
"2003-02-05" "2003-02-05" "2003-02-05"
"2003-02-05" "2003-03-19"
"2003-03-19" "2003-03-19"
[35] "2003-03-19" "2003-05-29" "2003-05-29"
"2003-05-29" "2003-05-29"
"2003-08-13" "2003-08-13" "2003-08-13"
"2003-08-13" "2003-11-03"
"2003-11-03" "2003-11-03" "2003-11-03"
"2003-11-03" "2002-06-25"
"2002-06-25" "2002-06-25"
[52] "2002-06-25" "2003-04-10" "2003-04-10"
"2003-04-10" "2003-04-10"
"2003-04-03" "2003-04-03" "2003-04-03"
"2003-04-03" "2003-10-15"
"2003-10-15" "2003-10-15" "2003-11-21"
"2003-11-21" "2003-11-21"
"2003-12-04" "2003-12-04"
[69] "2003-12-04"> dob[is.na(difftime(sdate,dob))] [1] "1927-04-03" "1927-04-03" "1927-04-03"
"1927-04-03" "1925-04-11"
"1925-04-11" "1925-04-11" "1925-04-11"
"1925-04-11" "1939-04-03"
"1939-04-03" "1939-04-03" "1940-12-30"
"1940-12-30" "1940-12-30"
"1917-10-14" "1917-10-14"
[18] "1917-10-14" "1917-10-14" "1925-04-16"
"1925-04-16" "1925-04-16"
"1925-04-16" "1927-04-05" "1927-04-05"
"1927-04-05" "1927-04-05"
"1939-04-08" "1939-04-08" "1939-04-08"
"1939-04-08" "1938-10-24"
"1938-10-24" "1938-10-24"
[35] "1938-10-24" "1930-10-16" "1930-10-16"
"1930-10-16" "1930-10-16"
"1923-04-17" "1923-04-17" "1923-04-17"
"1923-04-17" "1929-04-17"
"1929-04-17" "1929-04-17" "1929-04-17"
"1929-04-17" "1925-04-11"
"1925-04-11" "1925-04-11"
[52] "1925-04-11" "1931-04-02" "1931-04-02"
"1931-04-02" "1931-04-02"
"1929-04-18" "1929-04-18" "1929-04-18"
"1929-04-18" "1917-10-22"
"1917-10-22" "1917-10-22" "1928-03-28"
"1928-03-28" "1928-03-28"
"1928-04-09" "1928-04-09"
[69] "1928-04-09"

The values of dob here do not differ in any obvious way from those in the
rest of the dob vector, where difftime(sdate,sob) gives sensible results.

If I try to recompute the difftime, the result is the same.

s1=sdate[is.na(difftime(sdate,dob))]
d1=dob[is.na(difftime(sdate,dob))]
difftime(s1,d1)
Time differences in secs
 [1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
attr(,"tzone")
[1] ""

However, if I now create the first value of each missing vector manually,
then difftime works:-

js1=strptime('2002-02-28','%Y-%m-%d'); js1
#[1] "2002-02-28"
jb1=strptime('1927-04-03','%Y-%m-%d'); jb1
#[1] "1927-04-03"
difftime(js1,jb1)
#Time difference of 27360 days

So, it appears that strptime is handling these values differently in the
vector, but manages them correctly one by one.

I'm sorry if I'm being silly, but I can't see the problem. I'd
be VERY
grateful if someone could help me to find it and fix it.

With many thanks in advance for your thoughts,

Jonathan Williams

Don MacQueen

2010-Feb-23 21:25 UTC

head link

[R] Problem with strptime generating missing values where none appear to exist

What happens if you do all that NA checking on dob  *before* 
subtracting 100 from dob$year?

What happens if you use difftime() before subtracting the 100?

Do you get any NAs if you convert dob to POSIXct?

(these are just investigative ideas, obviously)

-Don

At 6:26 PM +0000 2/23/10, Jonathan Williams wrote:>Dear R Helpers,
>
>I am having difficulty with strptime. I wish to find the differences between
>two vectors of times. I have apparently no difficulty to convert the vectors
>to the appropriate format using strptime. But, then difftime does not
>calculate all the differences.
>
>Here is the code and output:-
>
>dob=strptime(as.character(datx$BDT),'%d-%b-%y');
dob$year=dob$year-100
>sdate=strptime(as.character(datx$SDT),'%d-%b-%y')
>head(dob); head(sdate)
>[1] "1922-07-14" "1922-07-14" "1922-07-14"
"1922-07-14" "1921-03-23"
>"1921-03-23"
>[1] "2001-08-27" "2001-08-27" "2001-08-27"
"2001-08-27" "2001-08-20"
>"2001-08-20"
>>  str(dob)
>  POSIXlt[1:9], format: "1922-07-14" "1922-07-14"
"1922-07-14" "1922-07-14"
>"1921-03-23" "1921-03-23" "1921-03-23"
"1927-08-27" "1927-08-27"
>"1927-08-27" "1927-08-27" "1940-04-05"
"1940-04-05" "1940-04-05"
>"1940-04-05" ...
>>  str(sdate)
>  POSIXlt[1:9], format: "2001-08-27" "2001-08-27"
"2001-08-27" "2001-08-27"
>"2001-08-20" "2001-08-20" "2001-08-20"
"2001-11-26" "2001-11-26"
>"2001-11-26" "2001-11-26" "2002-05-20"
"2002-05-20" "2002-05-20"
>"2002-05-20" ...
>
>table(is.na(sdate))
>
>FALSE
>   812
>
>table(is.na(dob))
>
>FALSE  TRUE
>   743    69
>But if I now look at each component of dob separately, none is missing
>
>for (i in 1:length(dob)) {print(names(dob)[i]);
>print(table(is.na(dob[[i]])))}
>
>[1] "sec"
>
>FALSE
>   812
>[1] "min"
>
>FALSE
>   812
>[1] "hour"
>
>FALSE
>   812
>[1] "mday"
>
>FALSE
>   812
>[1] "mon"
>
>FALSE
>   812
>[1] "year"
>
>FALSE
>   812
>[1] "wday"
>
>FALSE
>   812
>[1] "yday"
>
>FALSE
>   812
>[1] "isdst"
>
>FALSE
>   812
>
>Additionally, there are no NA values in any component of dob on direct
>visual inspection. For example, here is dob$mon
>dob$mon
>[1]  6  6  6  6  2  2  2  7  7  7  7  3  3  3  3 11 11 11 11  7  7  7  7  7
>7  7  7  4  4  4  4  4  4  4  4 11 11 11 11  7  7  7  7 11 11 11 11  6  6  6
>6  9  9  9  9  3  3  3  3  6  6  6  6  8  8  8  8  7  7  7  7  4  4  4  4  4
>  [77]  4  4  4  7  7  7  7 10 10 11 11 11 11  8  8  8  8  0  0  0  0  0 10
>10 10  7  7  7  7  3  3  3  3  2  2  2  2  2  6  6  6  6  6  5  5  5  5  4
>4  4  4  4 11 11 11 11  4  4  4  4  4  3  3  3  3  3  7  7  7  7  7  7  7  7
>8  8
>[153]  8  8  8  8  8  8  7  7  7  7  6  6 10 10 10 10  4  4  4  4  4  4  4
>4 10 10 10 10 11 11 11 11  5  5  5  5  5  5  5  5  3  3  3  3  5  5  0  0  0
>0  2  2  2  2  6  6  6  6  0  0  0  0  3  3  3  3  6  6  6  6  8  8  8  8  8
>7
>[229]  7  7  7  7  7  7  7  7  8  8  8  8  4  4  4  4 10 10 10 10  2  2  2
>2  0  0  0  0  0  0  1  1  1  1  4  4  4  4  2  2  2  2  2  8  8  8  8 11 11
>11 11  8  8  8  8  4  4  4  4  5  5  5  5  8  8  8  8  0  0  0  0  1  1  1
>1  1
>[305]  1  1  1  4  4  4  4  5  5  5  5  7  7  7  7  5  5  5  5  3  3  3  3
>1  1  1  1  0  0  0  0  3  3  3  3  6  6  6  6  3  3  3  5  5 11 11 11  5  5
>5  5  0  0  0  0 10 10 10 10  4  4  4  4  6  6  6  6  7  7  7  7  4  4  4  4
>1
>[381]  1  1  1  7  7  7  7  3  3  3  3  7  7  7  7  5  5  5  5  9  9  9  9
>11 11 11 11 10 10 10 10  0  0  0  0  5  5  5  5  3  3  3  3  7  7  7  7  0
>0  0  0  6  6  6  6  8  8  8  8  8  8  8  8  3  3  3  3  5  5  5  5 10 10 10
>10  3
>[457]  3  3  3  8  8  8  8  0  0  0  0 11 11 11 11  2  2  2  2  7  7  7  7
>0  0  0  0  0  1  1  1  1  5  5  5  5  7  7  7  7  7  7  7  7  5  5  5  5  9
>9  9  9  5  5  5  5  6  6  6  6  8  8  8  8 11 11 11 11  3  3  3  3  6  6  6
>6
>[533]  3  3  3  3  6  6  6  6  8  8  8  8  9  9  9  9  2  2  2  2  1  1  1
>1  2  2  2  2  4  4  4  7  7  7  8  8  8  8  3  3  3  3  1  1  1  1  1  9  9
>9  9  8  8  8  8 11 11 11 11  6  6  6  6  3  3  3  3 10 10 10  8  8  8  0  0
>0
>[609]  0  3  3  3  3  3  0  0  0  0  3  3  3  3  5  5  5  5 10 10 10 10 10
>10 10 10  2  2  2  2  2  3  3  3  3  4  4  4 10 10 10 10  2  2  2  2  3  3
>3  3  2  2  2  2  2  2  6  6  6  6  4  4  4  4 11 11 11 11  0  0  0  0 11 11
>11 11
>[685]  5  5  5  5  8  8  8  8  8  8  8  8  7  7  7  7  3  3  3  3  5  5  5
>5 11 11 11 11  3  3  3  9  9  5  5  5  5  8  8  8  8  2  2  2  2  5  5  5  5
>2  2  2  2 10 10 10 10  4  4  4 11 11 11 11  8  8  8  9  9  9  1  1  1  1  8
>8
>[761]  8  8  2  2  2 11 11 11 11  2  2  2  2  2  2  2  2  6  6  6  6 11 11
>11 11  2  2  2 11 11 11  9  9  9  9  2  2  2  2  7  7  7  7 11 11 11  2  2
>2  3  3  3
>
>All the dob components are equally complete, including isdst.
>
>However, when I then try to compute difftime(sdate,dob), 69 values are
>missing:-
>Time differences in days
>   [1] 28899.00 28899.00 28899.00 28899.00 29369.96 29369.96 29369.96
>27120.04 27120.04 27120.04 27120.04 22690.00 22690.00 22690.00 22690.00
>28905.00 28905.00 28905.00 28905.00 31207.04 31207.04 31207.04 31207.04
>31209.04 31209.04
>  [26] 31209.04 31209.04 26323.00 26323.00 26323.00 26323.00 26338.00
>26338.00 26338.00 26338.00 27310.96 27310.96 27310.96 27310.96 23588.04
>23588.04 23588.04 23588.04 25255.00 25255.00 25255.00 25255.00 23752.00
>23752.00 23752.00
>  [51] 23752.00 29607.04 29607.04 29607.04 29607.04 27993.04 27993.04
>27993.04 27993.04 28384.04 28384.04 28384.04 28384.04 26176.00 26176.00
>26176.00 26176.00 28986.04 28986.04 28986.04 28986.04 28689.04 28689.04
>28689.04 28689.04
>  [76] 23722.00 23722.00 23722.00 23722.00 27353.00 27353.00 27353.00
>27353.00 26303.00 26303.00 28803.96 28803.96 28803.96 28803.96 28564.04
>28564.04 28564.04 28564.04 29826.96 29826.96 29826.96 29826.96 29826.96
>30410.00 30410.00
>[101] 30410.00 26490.04 26490.04 26490.04 26490.04       NA       NA
>NA       NA 29765.96 29765.96 29765.96 29765.96 29765.96 26325.00 26325.00
>26325.00 26325.00 26325.00 28824.00 28824.00 28824.00 28824.00 26808.00
>26808.00
>[126] 26808.00 26808.00 26808.00 28628.96 28628.96 28628.96 28628.96
>23807.00 23807.00 23807.00 23807.00 23807.00       NA       NA       NA
>NA       NA 25668.04 25668.04 25668.04 25668.04 28654.04 28654.04 28654.04
>28654.04
>[151] 21711.04 21711.04 21711.04 21711.04 27167.04 27167.04 27167.04
>27167.04 24296.04 24296.04 24296.04 24296.04 30540.04 30540.04 25330.00
>25330.00 25330.00 25330.00 25579.00 25579.00 25579.00 25579.00 29127.04
>29127.04 29127.04
>[176] 29127.04 29896.96 29896.96 29896.96 29896.96 25992.00 25992.00
>25992.00 25992.00 26625.00 26625.00 26625.00 26625.00 30121.04 30121.04
>30121.04 30121.04 21801.04 21801.04 21801.04 21801.04 31274.04 31274.04
>25907.00 25907.00
>[201] 25907.00 25907.00 28516.00 28516.00 28516.00 28516.00 28943.00
>28943.00 28943.00 28943.00 29847.96 29847.96 29847.96 29847.96 30529.04
>30529.04 30529.04 30529.04 30527.04 30527.04 30527.04 30527.04 29434.00
>29434.00 29434.00
>[226] 29434.00 29434.00 28631.04 28631.04 28631.04 28631.04 25761.04
>25761.04 25761.04 25761.04 25761.04 26127.04 26127.04 26127.04 26127.04
>26027.00 26027.00 26027.00 26027.00 28987.00 28987.00 28987.00 28987.00
>29232.00 29232.00
>[251] 29232.00 29232.00 26109.96 26109.96 26109.96 26109.96 31339.00
>31339.00 29235.00 29235.00 29235.00 29235.00 28092.00 28092.00 28092.00
>28092.00 30209.00 30209.00 30209.00 30209.00 30209.00 30281.00 30281.00
>30281.00 30281.00
>[276] 26880.96 26880.96 26880.96 26880.96 25691.04 25691.04 25691.04
>25691.04 22938.04 22938.04 22938.04 22938.04 25878.00 25878.00 25878.00
>25878.00 24470.00 24470.00 24470.00 24470.00 26046.96 26046.96 26046.96
>26046.96 26763.96
>[301] 26763.96 26763.96 26763.96 25720.96 25720.96 25720.96 25720.96
>29214.00 29214.00 29214.00 29214.00 26992.00 26992.00 26992.00 26992.00
>30659.00 30659.00 30659.00 30659.00 25600.00 25600.00 25600.00 25600.00
>26842.00 26842.00
>[326] 26842.00 26842.00 25541.00 25541.00 25541.00 25541.00 27386.00
>27386.00 27386.00 27386.00 30302.04 30302.04 30302.04 30302.04 28059.00
>28059.00 28059.00 28059.00       NA       NA       NA 25657.00 25657.00
>NA       NA
>[351]       NA 24835.00 24835.00 24835.00 24835.00 29340.96 29340.96
>29340.96 29340.96 26473.96 26473.96 26473.96 26473.96 28873.00 28873.00
>28873.00 28873.00 27690.00 27690.00 27690.00 27690.00 26554.00 26554.00
>26554.00 26554.00
>[376] 28876.00 28876.00 28876.00 28876.00 27156.96 27156.96 27156.96
>27156.96 26577.00 26577.00 26577.00 26577.00 27471.00 27471.00 27471.00
>27471.00 27323.00 27323.00 27323.00 27323.00 29232.00 29232.00 29232.00
>29232.00       NA
>[401]       NA       NA       NA 26523.96 26523.96 26523.96 26523.96
>26538.96 26538.96 26538.96 26538.96 24374.96 24374.96 24374.96 24374.96
>30798.00 30798.00 30798.00 30798.00       NA       NA       NA       NA
>22775.04 22775.04
>[426] 22775.04 22775.04 28464.00 28464.00 28464.00 28464.00 25763.04
>25763.04 25763.04 25763.04 30114.04 30114.04 30114.04 30114.04 26864.04
>26864.04 26864.04 26864.04       NA       NA       NA       NA 26945.04
>26945.04 26945.04
>[451] 26945.04 29528.96 29528.96 29528.96 29528.96 29058.04 29058.04
>29058.04 29058.04 29456.00 29456.00 29456.00 29456.00 26450.96 26450.96
>26450.96 26450.96 22837.96 22837.96 22837.96 22837.96 24222.96 24222.96
>24222.96 24222.96
>[476] 29592.00 29592.00 29592.00 29592.00 26573.00 26573.00 26573.00
>26573.00 26573.00 24811.00 24811.00 24811.00 24811.00 24834.00 24834.00
>24834.00 24834.00 31312.00 31312.00 31312.00 31312.00 23337.00 23337.00
>23337.00 23337.00
>[501] 26422.00 26422.00 26422.00 26422.00 22664.04 22664.04 22664.04
>22664.04 23192.04 23192.04 23192.04 23192.04 27557.04 27557.04 27557.04
>27557.04 23449.04 23449.04 23449.04 23449.04 27799.00 27799.00 27799.00
>27799.00 28747.04
>[526] 28747.04 28747.04 28747.04 24660.04 24660.04 24660.04 24660.04
>NA       NA       NA       NA 24683.04 24683.04 24683.04 24683.04 26576.00
>26576.00 26576.00 26576.00       NA       NA       NA       NA 28897.96
>28897.96
>[551] 28897.96 28897.96 25997.96 25997.96 25997.96 25997.96 24594.96
>24594.96 24594.96 24594.96 25965.00 25965.00 25965.00 30139.04 30139.04
>30139.04 26104.04 26104.04 26104.04 26104.04 26255.04 26255.04 26255.04
>26255.04 28887.00
>[576] 28887.00 28887.00 28887.00 28887.00       NA       NA       NA
>NA 25470.00 25470.00 25470.00 25470.00 20677.96 20677.96 20677.96 20677.96
>29227.00 29227.00 29227.00 29227.00       NA       NA       NA       NA
>29543.96
>[601] 29543.96 29543.96 31080.00 31080.00 31080.00 27710.00 27710.00
>27710.00 27710.00       NA       NA       NA       NA       NA 29903.00
>29903.00 29903.00 29903.00       NA       NA       NA       NA 24147.00
>24147.00 24147.00
>[626] 24147.00 23316.96 23316.96 23316.96 23316.96 27096.00 27096.00
>27096.00 27096.00 25543.00 25543.00 25543.00 25543.00 25543.00       NA
>NA       NA       NA 25131.04 25131.04 25131.04 29565.96 29565.96 29565.96
>29565.96
>[651] 28070.00 28070.00 28070.00 28070.00 28774.04 28774.04 28774.04
>28774.04 28073.00 28073.00 28130.00 28130.00 28130.00 28130.00 20038.00
>20038.00 20038.00 20038.00 27298.04 27298.04 27298.04 27298.04 27793.00
>27793.00 27793.00
>[676] 27793.00 25586.00 25586.00 25586.00 25586.00 26000.00 26000.00
>26000.00 26000.00 30577.04 30577.04 30577.04 30577.04 27194.04 27194.04
>27194.04 27194.04 23156.04 23156.04 23156.04 23156.04 23978.04 23978.04
>23978.04 23978.04
>[701]       NA       NA       NA       NA 24391.00 24391.00 24391.00
>24391.00 27152.96 27152.96 27152.96 27152.96 28852.00 28852.00 28852.00
>25419.00 25419.00 29212.00 29212.00 29212.00 29212.00 23660.00 23660.00
>23660.00 23660.00
>[726] 26022.96 26022.96 26022.96 26022.96 25566.00 25566.00 25566.00
>25566.00 25336.96 25336.96 25336.96 25336.96 26931.96 26931.96 26931.96
>26931.96 26758.00 26758.00 26758.00 26537.96 26537.96 26537.96 26537.96
>27026.00 27026.00
>[751] 27026.00       NA       NA       NA 24349.96 24349.96 24349.96
>24349.96 25960.00 25960.00 25960.00 25960.00 27276.00 27276.00 27276.00
>26826.96 26826.96 26826.96 26826.96 26428.96 26428.96 26428.96 26428.96
>26780.96 26780.96
>[776] 26780.96 26780.96 26301.00 26301.00 26301.00 26301.00 28385.96
>28385.96 28385.96 28385.96 27210.96 27210.96 27210.96 23704.00 23704.00
>23704.00 24160.04 24160.04 24160.04 24160.04 25703.96 25703.96 25703.96
>25703.96 25269.00
>[801] 25269.00 25269.00 25269.00 29886.96 29886.96 29886.96       NA
>NA       NA       NA       NA       NA
>attr(,"tzone")
>[1] ""
>
>Here are the values of sdate and dob that relate to the missing values in
>difftime(sdate,dob)
>
>>  sdate[is.na(difftime(sdate,dob))]
>  [1] "2002-02-28" "2002-02-28" "2002-02-28"
"2002-02-28" "2002-07-30"
>"2002-07-30" "2002-07-30" "2002-07-30"
"2002-07-30" "2003-06-17"
>"2003-06-17" "2003-06-17" "2003-10-30"
"2003-10-30" "2003-10-30"
>"2002-07-22" "2002-07-22"
>[18] "2002-07-22" "2002-07-22" "2002-12-18"
"2002-12-18" "2002-12-18"
>"2002-12-18" "2003-03-10" "2003-03-10"
"2003-03-10" "2003-03-10"
>"2003-02-05" "2003-02-05" "2003-02-05"
"2003-02-05" "2003-03-19"
>"2003-03-19" "2003-03-19"
>[35] "2003-03-19" "2003-05-29" "2003-05-29"
"2003-05-29" "2003-05-29"
>"2003-08-13" "2003-08-13" "2003-08-13"
"2003-08-13" "2003-11-03"
>"2003-11-03" "2003-11-03" "2003-11-03"
"2003-11-03" "2002-06-25"
>"2002-06-25" "2002-06-25"
>[52] "2002-06-25" "2003-04-10" "2003-04-10"
"2003-04-10" "2003-04-10"
>"2003-04-03" "2003-04-03" "2003-04-03"
"2003-04-03" "2003-10-15"
>"2003-10-15" "2003-10-15" "2003-11-21"
"2003-11-21" "2003-11-21"
>"2003-12-04" "2003-12-04"
>[69] "2003-12-04"
>>  dob[is.na(difftime(sdate,dob))]
>  [1] "1927-04-03" "1927-04-03" "1927-04-03"
"1927-04-03" "1925-04-11"
>"1925-04-11" "1925-04-11" "1925-04-11"
"1925-04-11" "1939-04-03"
>"1939-04-03" "1939-04-03" "1940-12-30"
"1940-12-30" "1940-12-30"
>"1917-10-14" "1917-10-14"
>[18] "1917-10-14" "1917-10-14" "1925-04-16"
"1925-04-16" "1925-04-16"
>"1925-04-16" "1927-04-05" "1927-04-05"
"1927-04-05" "1927-04-05"
>"1939-04-08" "1939-04-08" "1939-04-08"
"1939-04-08" "1938-10-24"
>"1938-10-24" "1938-10-24"
>[35] "1938-10-24" "1930-10-16" "1930-10-16"
"1930-10-16" "1930-10-16"
>"1923-04-17" "1923-04-17" "1923-04-17"
"1923-04-17" "1929-04-17"
>"1929-04-17" "1929-04-17" "1929-04-17"
"1929-04-17" "1925-04-11"
>"1925-04-11" "1925-04-11"
>[52] "1925-04-11" "1931-04-02" "1931-04-02"
"1931-04-02" "1931-04-02"
>"1929-04-18" "1929-04-18" "1929-04-18"
"1929-04-18" "1917-10-22"
>"1917-10-22" "1917-10-22" "1928-03-28"
"1928-03-28" "1928-03-28"
>"1928-04-09" "1928-04-09"
>[69] "1928-04-09"
>
>The values of dob here do not differ in any obvious way from those in the
>rest of the dob vector, where difftime(sdate,sob) gives sensible results.
>
>If I try to recompute the difftime, the result is the same.
>
>s1=sdate[is.na(difftime(sdate,dob))]
>d1=dob[is.na(difftime(sdate,dob))]
>difftime(s1,d1)
>Time differences in secs
>  [1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
NA
>NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
>NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA
>attr(,"tzone")
>[1] ""
>
>However, if I now create the first value of each missing vector manually,
>then difftime works:-
>
>js1=strptime('2002-02-28','%Y-%m-%d'); js1
>#[1] "2002-02-28"
>jb1=strptime('1927-04-03','%Y-%m-%d'); jb1
>#[1] "1927-04-03"
>difftime(js1,jb1)
>#Time difference of 27360 days
>
>So, it appears that strptime is handling these values differently in the
>vector, but manages them correctly one by one.
>
>I'm sorry if I'm being silly, but I can't see the problem.
I'd be VERY
>grateful if someone could help me to find it and fix it.
>
>With many thanks in advance for your thoughts,
>
>Jonathan Williams
>
>______________________________________________
>R-help at r-project.org mailing list
>https://*stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
http://*www.*R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

-- 
--------------------------------------
Don MacQueen
Environmental Protection Department
Lawrence Livermore National Laboratory
Livermore, CA, USA
925-423-1062

Apparently Analagous Threads

Search for more maybe matching threads

R help - Feb 2010 - Problem with strptime generating missing values where none appear to exist

[R] Problem with strptime generating missing values where none appear to exist

[R] Problem with strptime generating missing values where none appear to exist

Apparently Analagous Threads