If I have data that I feed into shapio.test and jarque.bera.test yet they seem to disagree. What do I use for a decision? For my data set I have p.value of 0.05496421 returned from the shapiro.test and 0.882027 returned from the jarque.bera.test. I have included the data set below. Thank you. Kevin "Category","Period","Residual" "CHILD HATS, WIGS & MASKS",1/1/2005,-0.449735723758323 "CHILD HATS, WIGS & MASKS",2/1/2005,0.281461045050074 "CHILD HATS, WIGS & MASKS",3/1/2005,0.591383050911335 "CHILD HATS, WIGS & MASKS",4/1/2005,0.239998659520616 "CHILD HATS, WIGS & MASKS",5/1/2005,0.00343879474063987 "CHILD HATS, WIGS & MASKS",6/1/2005,-2.64372061292663 "CHILD HATS, WIGS & MASKS",7/1/2005,0.381630655290173 "CHILD HATS, WIGS & MASKS",8/1/2005,-1.79543281552347 "CHILD HATS, WIGS & MASKS",9/1/2005,1.90631012440313 "CHILD HATS, WIGS & MASKS",10/1/2005,-0.256232543929779 "CHILD HATS, WIGS & MASKS",11/1/2005,1.83452602676812 "CHILD HATS, WIGS & MASKS",12/1/2005,-1.06869719416837 "CHILD HATS, WIGS & MASKS",1/1/2006,1.04378655286183 "CHILD HATS, WIGS & MASKS",2/1/2006,0.232655831328322 "CHILD HATS, WIGS & MASKS",3/1/2006,-0.939084802643773 "CHILD HATS, WIGS & MASKS",4/1/2006,0.854132879285335 "CHILD HATS, WIGS & MASKS",5/1/2006,-1.71217066877156 "CHILD HATS, WIGS & MASKS",6/1/2006,1.28040273099582 "CHILD HATS, WIGS & MASKS",7/1/2006,-0.386415431325857 "CHILD HATS, WIGS & MASKS",8/1/2006,-0.769127669783483 "CHILD HATS, WIGS & MASKS",9/1/2006,-0.810996835089867 "CHILD HATS, WIGS & MASKS",10/1/2006,0.0477292147635991 "CHILD HATS, WIGS & MASKS",11/1/2006,0.294672848750557 "CHILD HATS, WIGS & MASKS",12/1/2006,-0.0841330473924862 "CHILD HATS, WIGS & MASKS",1/1/2007,0.231663729192233 "CHILD HATS, WIGS & MASKS",2/1/2007,-0.601790650547443 "CHILD HATS, WIGS & MASKS",3/1/2007,0.285635768516625 "CHILD HATS, WIGS & MASKS",4/1/2007,-0.963154959558619 "CHILD HATS, WIGS & MASKS",5/1/2007,1.52188112949994 "CHILD HATS, WIGS & MASKS",6/1/2007,-0.826092842933196 "CHILD HATS, WIGS & MASKS",7/1/2007,1.91937201229077 "CHILD HATS, WIGS & MASKS",8/1/2007,-0.317789483136924 "CHILD HATS, WIGS & MASKS",9/1/2007,-0.865011007394312 "CHILD HATS, WIGS & MASKS",10/1/2007,-0.0281604973711276 "CHILD HATS, WIGS & MASKS",11/1/2007,-0.123887049811822 "CHILD HATS, WIGS & MASKS",12/1/2007,-0.0327727730592468 "CHILD HATS, WIGS & MASKS",1/1/2008,-0.0654939600771254 "CHILD HATS, WIGS & MASKS",2/1/2008,0.279247739913908 "CHILD HATS, WIGS & MASKS",3/1/2008,0.167606602923418 "CHILD HATS, WIGS & MASKS",4/1/2008,0.189533097427477 "CHILD HATS, WIGS & MASKS",5/1/2008,0.402062194225847 "CHILD HATS, WIGS & MASKS",6/1/2008,1.97150984262995 "CHILD HATS, WIGS & MASKS",7/1/2008,-2.27538477532968 "CHILD HATS, WIGS & MASKS",8/1/2008,1.89091792097945 "CHILD HATS, WIGS & MASKS",9/1/2008,0.0251732151287081 "CHILD HATS, WIGS & MASKS",10/1/2008,-0.2349741808124 "CHILD HATS, WIGS & MASKS",11/1/2008,-0.659332058368173 "CHILD HATS, WIGS & MASKS",12/1/2008,0.127284768034285 "CHILD HATS, WIGS & MASKS",1/1/2009,-1.42838560676513 "CHILD HATS, WIGS & MASKS",2/1/2009,0.617689775286461 "CHILD HATS, WIGS & MASKS",3/1/2009,-0.034243005247084 "CHILD HATS, WIGS & MASKS",4/1/2009,-0.304574261133836 "CHILD HATS, WIGS & MASKS",5/1/2009,0.128679369916751 "CHILD HATS, WIGS & MASKS",6/1/2009,-0.657479389968652 "CHILD HATS, WIGS & MASKS",7/1/2009,0.608766068692517 "CHILD HATS, WIGS & MASKS",8/1/2009,1.92814770869400 "CHILD HATS, WIGS & MASKS",9/1/2009,-0.172644961366165 "CHILD HATS, WIGS & MASKS",10/1/2009,-0.453255508263169 "CHILD HATS, WIGS & MASKS",11/1/2009,-1.09903330959344
If I have data that I feed into shapio.test and jarque.bera.test yet they seem to disagree. What do I use for a decision? For my data set I have p.value of 0.05496421 returned from the shapiro.test and 0.882027 returned from the jarque.bera.test. I have included the data set below. Thank you. Kevin "Category","Period","Residual" "CHILD HATS, WIGS & MASKS",1/1/2005,-0.449735723758323 "CHILD HATS, WIGS & MASKS",2/1/2005,0.281461045050074 "CHILD HATS, WIGS & MASKS",3/1/2005,0.591383050911335 "CHILD HATS, WIGS & MASKS",4/1/2005,0.239998659520616 "CHILD HATS, WIGS & MASKS",5/1/2005,0.00343879474063987 "CHILD HATS, WIGS & MASKS",6/1/2005,-2.64372061292663 "CHILD HATS, WIGS & MASKS",7/1/2005,0.381630655290173 "CHILD HATS, WIGS & MASKS",8/1/2005,-1.79543281552347 "CHILD HATS, WIGS & MASKS",9/1/2005,1.90631012440313 "CHILD HATS, WIGS & MASKS",10/1/2005,-0.256232543929779 "CHILD HATS, WIGS & MASKS",11/1/2005,1.83452602676812 "CHILD HATS, WIGS & MASKS",12/1/2005,-1.06869719416837 "CHILD HATS, WIGS & MASKS",1/1/2006,1.04378655286183 "CHILD HATS, WIGS & MASKS",2/1/2006,0.232655831328322 "CHILD HATS, WIGS & MASKS",3/1/2006,-0.939084802643773 "CHILD HATS, WIGS & MASKS",4/1/2006,0.854132879285335 "CHILD HATS, WIGS & MASKS",5/1/2006,-1.71217066877156 "CHILD HATS, WIGS & MASKS",6/1/2006,1.28040273099582 "CHILD HATS, WIGS & MASKS",7/1/2006,-0.386415431325857 "CHILD HATS, WIGS & MASKS",8/1/2006,-0.769127669783483 "CHILD HATS, WIGS & MASKS",9/1/2006,-0.810996835089867 "CHILD HATS, WIGS & MASKS",10/1/2006,0.0477292147635991 "CHILD HATS, WIGS & MASKS",11/1/2006,0.294672848750557 "CHILD HATS, WIGS & MASKS",12/1/2006,-0.0841330473924862 "CHILD HATS, WIGS & MASKS",1/1/2007,0.231663729192233 "CHILD HATS, WIGS & MASKS",2/1/2007,-0.601790650547443 "CHILD HATS, WIGS & MASKS",3/1/2007,0.285635768516625 "CHILD HATS, WIGS & MASKS",4/1/2007,-0.963154959558619 "CHILD HATS, WIGS & MASKS",5/1/2007,1.52188112949994 "CHILD HATS, WIGS & MASKS",6/1/2007,-0.826092842933196 "CHILD HATS, WIGS & MASKS",7/1/2007,1.91937201229077 "CHILD HATS, WIGS & MASKS",8/1/2007,-0.317789483136924 "CHILD HATS, WIGS & MASKS",9/1/2007,-0.865011007394312 "CHILD HATS, WIGS & MASKS",10/1/2007,-0.0281604973711276 "CHILD HATS, WIGS & MASKS",11/1/2007,-0.123887049811822 "CHILD HATS, WIGS & MASKS",12/1/2007,-0.0327727730592468 "CHILD HATS, WIGS & MASKS",1/1/2008,-0.0654939600771254 "CHILD HATS, WIGS & MASKS",2/1/2008,0.279247739913908 "CHILD HATS, WIGS & MASKS",3/1/2008,0.167606602923418 "CHILD HATS, WIGS & MASKS",4/1/2008,0.189533097427477 "CHILD HATS, WIGS & MASKS",5/1/2008,0.402062194225847 "CHILD HATS, WIGS & MASKS",6/1/2008,1.97150984262995 "CHILD HATS, WIGS & MASKS",7/1/2008,-2.27538477532968 "CHILD HATS, WIGS & MASKS",8/1/2008,1.89091792097945 "CHILD HATS, WIGS & MASKS",9/1/2008,0.0251732151287081 "CHILD HATS, WIGS & MASKS",10/1/2008,-0.2349741808124 "CHILD HATS, WIGS & MASKS",11/1/2008,-0.659332058368173 "CHILD HATS, WIGS & MASKS",12/1/2008,0.127284768034285 "CHILD HATS, WIGS & MASKS",1/1/2009,-1.42838560676513 "CHILD HATS, WIGS & MASKS",2/1/2009,0.617689775286461 "CHILD HATS, WIGS & MASKS",3/1/2009,-0.034243005247084 "CHILD HATS, WIGS & MASKS",4/1/2009,-0.304574261133836 "CHILD HATS, WIGS & MASKS",5/1/2009,0.128679369916751 "CHILD HATS, WIGS & MASKS",6/1/2009,-0.657479389968652 "CHILD HATS, WIGS & MASKS",7/1/2009,0.608766068692517 "CHILD HATS, WIGS & MASKS",8/1/2009,1.92814770869400 "CHILD HATS, WIGS & MASKS",9/1/2009,-0.172644961366165 "CHILD HATS, WIGS & MASKS",10/1/2009,-0.453255508263169 "CHILD HATS, WIGS & MASKS",11/1/2009,-1.09903330959344
Hi there, Could you provide a minimum reproducible code, please. Bests milton On Tue, Dec 1, 2009 at 6:11 PM, <rkevinburton@charter.net> wrote:> If I have data that I feed into shapio.test and jarque.bera.test yet they > seem to disagree. What do I use for a decision? > > For my data set I have p.value of 0.05496421 returned from the shapiro.test > and 0.882027 returned from the jarque.bera.test. I have included the data > set below. > > Thank you. > > Kevin > > > "Category","Period","Residual" > "CHILD HATS, WIGS & MASKS",1/1/2005,-0.449735723758323 > "CHILD HATS, WIGS & MASKS",2/1/2005,0.281461045050074 > "CHILD HATS, WIGS & MASKS",3/1/2005,0.591383050911335 > "CHILD HATS, WIGS & MASKS",4/1/2005,0.239998659520616 > "CHILD HATS, WIGS & MASKS",5/1/2005,0.00343879474063987 > "CHILD HATS, WIGS & MASKS",6/1/2005,-2.64372061292663 > "CHILD HATS, WIGS & MASKS",7/1/2005,0.381630655290173 > "CHILD HATS, WIGS & MASKS",8/1/2005,-1.79543281552347 > "CHILD HATS, WIGS & MASKS",9/1/2005,1.90631012440313 > "CHILD HATS, WIGS & MASKS",10/1/2005,-0.256232543929779 > "CHILD HATS, WIGS & MASKS",11/1/2005,1.83452602676812 > "CHILD HATS, WIGS & MASKS",12/1/2005,-1.06869719416837 > "CHILD HATS, WIGS & MASKS",1/1/2006,1.04378655286183 > "CHILD HATS, WIGS & MASKS",2/1/2006,0.232655831328322 > "CHILD HATS, WIGS & MASKS",3/1/2006,-0.939084802643773 > "CHILD HATS, WIGS & MASKS",4/1/2006,0.854132879285335 > "CHILD HATS, WIGS & MASKS",5/1/2006,-1.71217066877156 > "CHILD HATS, WIGS & MASKS",6/1/2006,1.28040273099582 > "CHILD HATS, WIGS & MASKS",7/1/2006,-0.386415431325857 > "CHILD HATS, WIGS & MASKS",8/1/2006,-0.769127669783483 > "CHILD HATS, WIGS & MASKS",9/1/2006,-0.810996835089867 > "CHILD HATS, WIGS & MASKS",10/1/2006,0.0477292147635991 > "CHILD HATS, WIGS & MASKS",11/1/2006,0.294672848750557 > "CHILD HATS, WIGS & MASKS",12/1/2006,-0.0841330473924862 > "CHILD HATS, WIGS & MASKS",1/1/2007,0.231663729192233 > "CHILD HATS, WIGS & MASKS",2/1/2007,-0.601790650547443 > "CHILD HATS, WIGS & MASKS",3/1/2007,0.285635768516625 > "CHILD HATS, WIGS & MASKS",4/1/2007,-0.963154959558619 > "CHILD HATS, WIGS & MASKS",5/1/2007,1.52188112949994 > "CHILD HATS, WIGS & MASKS",6/1/2007,-0.826092842933196 > "CHILD HATS, WIGS & MASKS",7/1/2007,1.91937201229077 > "CHILD HATS, WIGS & MASKS",8/1/2007,-0.317789483136924 > "CHILD HATS, WIGS & MASKS",9/1/2007,-0.865011007394312 > "CHILD HATS, WIGS & MASKS",10/1/2007,-0.0281604973711276 > "CHILD HATS, WIGS & MASKS",11/1/2007,-0.123887049811822 > "CHILD HATS, WIGS & MASKS",12/1/2007,-0.0327727730592468 > "CHILD HATS, WIGS & MASKS",1/1/2008,-0.0654939600771254 > "CHILD HATS, WIGS & MASKS",2/1/2008,0.279247739913908 > "CHILD HATS, WIGS & MASKS",3/1/2008,0.167606602923418 > "CHILD HATS, WIGS & MASKS",4/1/2008,0.189533097427477 > "CHILD HATS, WIGS & MASKS",5/1/2008,0.402062194225847 > "CHILD HATS, WIGS & MASKS",6/1/2008,1.97150984262995 > "CHILD HATS, WIGS & MASKS",7/1/2008,-2.27538477532968 > "CHILD HATS, WIGS & MASKS",8/1/2008,1.89091792097945 > "CHILD HATS, WIGS & MASKS",9/1/2008,0.0251732151287081 > "CHILD HATS, WIGS & MASKS",10/1/2008,-0.2349741808124 > "CHILD HATS, WIGS & MASKS",11/1/2008,-0.659332058368173 > "CHILD HATS, WIGS & MASKS",12/1/2008,0.127284768034285 > "CHILD HATS, WIGS & MASKS",1/1/2009,-1.42838560676513 > "CHILD HATS, WIGS & MASKS",2/1/2009,0.617689775286461 > "CHILD HATS, WIGS & MASKS",3/1/2009,-0.034243005247084 > "CHILD HATS, WIGS & MASKS",4/1/2009,-0.304574261133836 > "CHILD HATS, WIGS & MASKS",5/1/2009,0.128679369916751 > "CHILD HATS, WIGS & MASKS",6/1/2009,-0.657479389968652 > "CHILD HATS, WIGS & MASKS",7/1/2009,0.608766068692517 > "CHILD HATS, WIGS & MASKS",8/1/2009,1.92814770869400 > "CHILD HATS, WIGS & MASKS",9/1/2009,-0.172644961366165 > "CHILD HATS, WIGS & MASKS",10/1/2009,-0.453255508263169 > "CHILD HATS, WIGS & MASKS",11/1/2009,-1.09903330959344 > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html<http://www.r-project.org/posting-guide.html> > and provide commented, minimal, self-contained, reproducible code. >[[alternative HTML version deleted]]
rkevinburton at charter.net wrote:> If I have data that I feed into shapio.test and jarque.bera.test yet they seem to disagree. What do I use for a decision?I feel those testing for normality in real applied work always have some masochistic attitude. This attitude is squared if they/you even try out a second test. There are three possible results: 1. Both tests do not reject: You still do not know if the underlying distribution is normal. Hence a useless result. 2. Both tests do reject: You know with some probability 1-\alpha that your data are not normal but you do not know what to do next now - except for things that are obvious when looking at you data - and you should have done that before the analysis anyway. If nothing was obvious before, you will hide your finding about the non-normality now - or use another method that might be less powerful given the difference from normality is not really big. 3. Your case, one test rejects, the other one not: Welcome, you finally arrived in hell. Best wishes, Uwe Ligges> For my data set I have p.value of 0.05496421 returned from the shapiro.test and 0.882027 returned from the jarque.bera.test. I have included the data set below. > > Thank you. > > Kevin > > > "Category","Period","Residual" > "CHILD HATS, WIGS & MASKS",1/1/2005,-0.449735723758323 > "CHILD HATS, WIGS & MASKS",2/1/2005,0.281461045050074 > "CHILD HATS, WIGS & MASKS",3/1/2005,0.591383050911335 > "CHILD HATS, WIGS & MASKS",4/1/2005,0.239998659520616 > "CHILD HATS, WIGS & MASKS",5/1/2005,0.00343879474063987 > "CHILD HATS, WIGS & MASKS",6/1/2005,-2.64372061292663 > "CHILD HATS, WIGS & MASKS",7/1/2005,0.381630655290173 > "CHILD HATS, WIGS & MASKS",8/1/2005,-1.79543281552347 > "CHILD HATS, WIGS & MASKS",9/1/2005,1.90631012440313 > "CHILD HATS, WIGS & MASKS",10/1/2005,-0.256232543929779 > "CHILD HATS, WIGS & MASKS",11/1/2005,1.83452602676812 > "CHILD HATS, WIGS & MASKS",12/1/2005,-1.06869719416837 > "CHILD HATS, WIGS & MASKS",1/1/2006,1.04378655286183 > "CHILD HATS, WIGS & MASKS",2/1/2006,0.232655831328322 > "CHILD HATS, WIGS & MASKS",3/1/2006,-0.939084802643773 > "CHILD HATS, WIGS & MASKS",4/1/2006,0.854132879285335 > "CHILD HATS, WIGS & MASKS",5/1/2006,-1.71217066877156 > "CHILD HATS, WIGS & MASKS",6/1/2006,1.28040273099582 > "CHILD HATS, WIGS & MASKS",7/1/2006,-0.386415431325857 > "CHILD HATS, WIGS & MASKS",8/1/2006,-0.769127669783483 > "CHILD HATS, WIGS & MASKS",9/1/2006,-0.810996835089867 > "CHILD HATS, WIGS & MASKS",10/1/2006,0.0477292147635991 > "CHILD HATS, WIGS & MASKS",11/1/2006,0.294672848750557 > "CHILD HATS, WIGS & MASKS",12/1/2006,-0.0841330473924862 > "CHILD HATS, WIGS & MASKS",1/1/2007,0.231663729192233 > "CHILD HATS, WIGS & MASKS",2/1/2007,-0.601790650547443 > "CHILD HATS, WIGS & MASKS",3/1/2007,0.285635768516625 > "CHILD HATS, WIGS & MASKS",4/1/2007,-0.963154959558619 > "CHILD HATS, WIGS & MASKS",5/1/2007,1.52188112949994 > "CHILD HATS, WIGS & MASKS",6/1/2007,-0.826092842933196 > "CHILD HATS, WIGS & MASKS",7/1/2007,1.91937201229077 > "CHILD HATS, WIGS & MASKS",8/1/2007,-0.317789483136924 > "CHILD HATS, WIGS & MASKS",9/1/2007,-0.865011007394312 > "CHILD HATS, WIGS & MASKS",10/1/2007,-0.0281604973711276 > "CHILD HATS, WIGS & MASKS",11/1/2007,-0.123887049811822 > "CHILD HATS, WIGS & MASKS",12/1/2007,-0.0327727730592468 > "CHILD HATS, WIGS & MASKS",1/1/2008,-0.0654939600771254 > "CHILD HATS, WIGS & MASKS",2/1/2008,0.279247739913908 > "CHILD HATS, WIGS & MASKS",3/1/2008,0.167606602923418 > "CHILD HATS, WIGS & MASKS",4/1/2008,0.189533097427477 > "CHILD HATS, WIGS & MASKS",5/1/2008,0.402062194225847 > "CHILD HATS, WIGS & MASKS",6/1/2008,1.97150984262995 > "CHILD HATS, WIGS & MASKS",7/1/2008,-2.27538477532968 > "CHILD HATS, WIGS & MASKS",8/1/2008,1.89091792097945 > "CHILD HATS, WIGS & MASKS",9/1/2008,0.0251732151287081 > "CHILD HATS, WIGS & MASKS",10/1/2008,-0.2349741808124 > "CHILD HATS, WIGS & MASKS",11/1/2008,-0.659332058368173 > "CHILD HATS, WIGS & MASKS",12/1/2008,0.127284768034285 > "CHILD HATS, WIGS & MASKS",1/1/2009,-1.42838560676513 > "CHILD HATS, WIGS & MASKS",2/1/2009,0.617689775286461 > "CHILD HATS, WIGS & MASKS",3/1/2009,-0.034243005247084 > "CHILD HATS, WIGS & MASKS",4/1/2009,-0.304574261133836 > "CHILD HATS, WIGS & MASKS",5/1/2009,0.128679369916751 > "CHILD HATS, WIGS & MASKS",6/1/2009,-0.657479389968652 > "CHILD HATS, WIGS & MASKS",7/1/2009,0.608766068692517 > "CHILD HATS, WIGS & MASKS",8/1/2009,1.92814770869400 > "CHILD HATS, WIGS & MASKS",9/1/2009,-0.172644961366165 > "CHILD HATS, WIGS & MASKS",10/1/2009,-0.453255508263169 > "CHILD HATS, WIGS & MASKS",11/1/2009,-1.09903330959344 > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.
On 01-Dec-09 23:11:20, rkevinburton at charter.net wrote:> If I have data that I feed into shapio.test and jarque.bera.test yet > they seem to disagree. What do I use for a decision? > > For my data set I have p.value of 0.05496421 returned from the > shapiro.test and 0.882027 returned from the jarque.bera.test. I have > included the data set below. > > Thank you. > > Kevin[Data snipped] The reason is that the Jarque-Bera test (JB) works with the squared skewness plus 1/4 of (kurtosis - 3)^2. For a Normal distribution, the skewness is zero and the kurtosis is 3. Hence large values of JB are evidence against the hypothesis that the distribution is Normal, in the basis that skewness and/or kurtosis depart from the values to be expected for a Normal distribution. However, it is perfectly possible to get skewness near 0, and kurtosis near 3, for manifestly non-Normal distributions. I get Skewness=0.014 and Kurtosis=3.32 for your data, both quite close to the Normal values. However, you only have to look at the histogram to see that the distribution has a distinctly non-Normal appearance: hist(x,breaks=20) The Shapiro-Wilk test, on the other hand, works (broadly speaking) in terms of standardised spacings between the order statistics of the sample, compared with what they should be from a Normal. It is therefore sensitive to features of the sample which are rather different to the features that the J-B test is sensitive to. Given the appearance of the histogram, it is to be expected that many of the spacings between order statistics are different from what are to be expected from a Normal distribution. Much of this, and also the insensitivity of the J-B test, arises from the clump of values at the top of the range (6 out of the 59 between 1.83 and 2.00). Leave these out and you get quite different results. The following is an explicit implementation of the J-B test, based on the Wikipedia description, and using the chi-squared(2) approximation for the P-value (and also returning the skewness and kurtosis): jarque.bera <- function(x){ m1 <- mean(x) ; m2 <- mean((x-m1)^2) m3 <- mean((x-m1)^3) ; m4 <- mean((x-m1)^4) n <- length(x) ; S <- m3/(m2^(3/2)) ; K <- m4/(m2^2) JB <- (n/6)*(S^2 + ((K-3)^2)/4) P <- 1-pchisq(JB,2) list(JB=JB,P=P,S=S,K=K) } For your original data x (as explicitly extracted by Ben Bolker): jarque.bera(x) # $JB # [1] 0.251065 # $P # [1] 0.882027 ##### (As you found yourself) # $S # [1] 0.01396711 # $K # [1] 3.318352 For the data excluding the 6 values above 1.83: jarque.bera(x[x<=1.83]) # $JB # [1] 6.047885 # $P # [1] 0.04860919 # $S # [1] -0.6831185 # $K # [1] 3.933842 So excluding these values has produced a distinct negative skewness and a kurtosis distinctly greater than 3. Hence those 6 values were primarily responsible for almost completely eliminating the skewness and the kurtosis of the remainder of the distribution, and hence frustrating the J-B test. Now compare with the Shapiro-Wilk test: shapiro.test(x) # Shapiro-Wilk normality test # data: x # W = 0.9608, p-value = 0.05496 so the S-W P-value 0.05496 for the full data is close to the J-B P-value 0.04861 for the reduced data. Now compare with the S-W test on the reduced data: shapiro.test(x[x <= 1.83]) # Shapiro-Wilk normality test # data: x[x <= 1.83] # W = 0.9595, p-value = 0.06968 The S-W P-vale has increased slightly (from 0.055 to 0.070), but the S-W test is still picking up the non-Normality in the reduced dataset. The summary is that the S-W test and the J-B test are looking at different aspects of the data. The J-B test depends only on two summary functions (skewness and kurtosis) as indices of non-Normality, while the S-W test is sensitive to a wider variation of the fine detail of the distribution of the data. The failure of the J-B test to detect the non-Normality in the data is primarily due to the fact that the 6 data values at the top end have, in effect, compensated for the marked skewness and kurtosis in the remainder of the data. The ultimate lesson from all this is that different tests test for different kinds of deperture from the Null Hypothesis. See also Uwe Ligges's remarks ... Ted. -------------------------------------------------------------------- E-Mail: (Ted Harding) <Ted.Harding at manchester.ac.uk> Fax-to-email: +44 (0)870 094 0861 Date: 02-Dec-09 Time: 11:53:22 ------------------------------ XFMail ------------------------------
rkevinburton wrote:> > If I have data that I feed into shapio.test and jarque.bera.test yet they > seem to disagree. What do I use for a decision? > > For my data set I have p.value of 0.05496421 returned from the > shapiro.test and 0.882027 returned from the jarque.bera.test. I have > included the data set below. >The explanation is quite simple - different tests for normality concentrate on different aspects of nonnormality and so are sensitive to different kinds of departures from it. Your sample is distinctly non-normal looking, but in a way that the jarque-bera test is insensitive to (the sample skewness and kurtosis are entirely unremarkable, being well within the typical range of values from a normal sample). However, the shapiro-wilk test is quite sensitive to this form of departure form normality. To see what's going on, do: (i) a density plot of your data via plot(density(x)). In particular note that second mode in the upper tail. (ii) a qq-plot. The departure from linearity evident in the upper right is caused by that mode. shapiro test sees this nonlinearity (the shapiro-francia, an approximation to the shapiro wilk, is a function of the squared correlation of the two variables in this plot). (iii) compute sample skewness and kurtosis and their standard errors (the JB combines these into a single measure); formulas easily found via google search. They're both close to the middle of their distribution for sampling from a normal with n=59. Glen -- View this message in context: http://n4.nabble.com/Normal-tests-disagree-tp932615p947452.html Sent from the R help mailing list archive at Nabble.com.