R help - Nov 2004 - Data Censoring and Normality Tests

Hello,

I would like to know if there is a function in R that will test for 
normality and handle censored data sets.  Currently, I evaluate each 
censored data set by the extent to which a normal scores plot 
approximate a straight line.  For complete data sets I use 
shapiro.test().

Below is an example of a censored data set.

data1<-c(0.00, 0.00, 0.00, 5.86, 5.17,  8.17,  5.12, 4.92, 7.08, 5.73, 
5.44, 6.61, 6.34, 6.23, 5.97, 5.86, 5.15, 7.98, 6.72, 5.15, 3.58, 6.86, 
6.12, 4.58, 6.07, 5.38, 5.21, 3.78)

The zero values occur because I cannot detect a pathogen with a value 
below 2.4.  Using shapiro.test(), it seems that the data are different 
from a normal distribution.  However, the normal scores plot suggest 
otherwise.

Using R version 1.9.1

Thanks in advance.

Ken
________________________________________________________
Kenneth E. Frost
Research Assistant
University of Wisconsin - Madison
Dept. of Plant Pathology
1630 Linden Dr.
Madison, WI 53706
kef at plantpath.wisc.edu

I don't know much about censored data, but what you described doesn't
sound
like censored data, but rather truncated data.  

I think normality tests are over-(ab)used, and would suggest inspection of
normal QQ plots instead (as you already do).  For formal tests, you might be
able to cook up something that test for truncated normality.

Cheers,
Andy
> From: Kenneth Frost
> 
> Hello,
> 
> I would like to know if there is a function in R that will test for 
> normality and handle censored data sets.  Currently, I evaluate each 
> censored data set by the extent to which a normal scores plot 
> approximate a straight line.  For complete data sets I use 
> shapiro.test().
> 
> Below is an example of a censored data set.
> 
> data1<-c(0.00, 0.00, 0.00, 5.86, 5.17,  8.17,  5.12, 4.92, 
> 7.08, 5.73, 
> 5.44, 6.61, 6.34, 6.23, 5.97, 5.86, 5.15, 7.98, 6.72, 5.15, 
> 3.58, 6.86, 
> 6.12, 4.58, 6.07, 5.38, 5.21, 3.78)
> 
> The zero values occur because I cannot detect a pathogen with a value 
> below 2.4.  Using shapiro.test(), it seems that the data are 
> different 
> from a normal distribution.  However, the normal scores plot suggest 
> otherwise.
> 
> Using R version 1.9.1
> 
> Thanks in advance.
> 
> Ken
> ________________________________________________________
> Kenneth E. Frost
> Research Assistant
> University of Wisconsin - Madison
> Dept. of Plant Pathology
> 1630 Linden Dr.
> Madison, WI 53706
> kef at plantpath.wisc.edu
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide! 
> http://www.R-project.org/posting-guide.html
> 
>

I stand corrected!  Hope Bert doesn't mind me CC'ing the list.

Andy
> From: Berton Gunter
> 
> No Andy, they're (left) censored alright: their values are 
> known to be below
> 2.4, but not how far below. Truncated data "arise when 
> observations are
> actually observed only when they take on values in a 
> particular range (above
> 2.4, say). For observations outside that range, their EXISTENCE is not
> known." -- Meeker and Escobar, "Statistical Methods for 
> Reliability Data."
> 
> Cheers,
> Bert
>  
>  
> 
> > -----Original Message-----
> > From: r-help-bounces at stat.math.ethz.ch 
> > [mailto:r-help-bounces at stat.math.ethz.ch] On Behalf Of Liaw, Andy
> > Sent: Tuesday, November 09, 2004 3:12 PM
> > To: 'Kenneth Frost'; r-help at stat.math.ethz.ch
> > Subject: RE: [R] Data Censoring and Normality Tests
> > 
> > I don't know much about censored data, but what you described 
> > doesn't sound
> > like censored data, but rather truncated data.  
> > 
> > I think normality tests are over-(ab)used, and would suggest 
> > inspection of
> > normal QQ plots instead (as you already do).  For formal 
> > tests, you might be
> > able to cook up something that test for truncated normality.
> > 
> > Cheers,
> > Andy
> > 
> > > From: Kenneth Frost
> > > 
> > > Hello,
> > > 
> > > I would like to know if there is a function in R that 
> will test for 
> > > normality and handle censored data sets.  Currently, I 
> > evaluate each 
> > > censored data set by the extent to which a normal scores plot 
> > > approximate a straight line.  For complete data sets I use 
> > > shapiro.test().
> > > 
> > > Below is an example of a censored data set.
> > > 
> > > data1<-c(0.00, 0.00, 0.00, 5.86, 5.17,  8.17,  5.12, 4.92, 
> > > 7.08, 5.73, 
> > > 5.44, 6.61, 6.34, 6.23, 5.97, 5.86, 5.15, 7.98, 6.72, 5.15, 
> > > 3.58, 6.86, 
> > > 6.12, 4.58, 6.07, 5.38, 5.21, 3.78)
> > > 
> > > The zero values occur because I cannot detect a pathogen 
> > with a value 
> > > below 2.4.  Using shapiro.test(), it seems that the data are 
> > > different 
> > > from a normal distribution.  However, the normal scores 
> > plot suggest 
> > > otherwise.
> > > 
> > > Using R version 1.9.1
> > > 
> > > Thanks in advance.
> > > 
> > > Ken
> > > ________________________________________________________
> > > Kenneth E. Frost
> > > Research Assistant
> > > University of Wisconsin - Madison
> > > Dept. of Plant Pathology
> > > 1630 Linden Dr.
> > > Madison, WI 53706
> > > kef at plantpath.wisc.edu
> > > 
> > > ______________________________________________
> > > R-help at stat.math.ethz.ch mailing list
> > > https://stat.ethz.ch/mailman/listinfo/r-help
> > > PLEASE do read the posting guide! 
> > > http://www.R-project.org/posting-guide.html
> > > 
> > >
> > 
> > ______________________________________________
> > R-help at stat.math.ethz.ch mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide! 
> > http://www.R-project.org/posting-guide.html
> > 
>