Hi there, I would like to draw a scatter plot and fit a smooth line by loess. Below is the data. However, the curve line started from 0, which my "resid" list doesn't consist of 0 value. It returned some warnings which I don't know if this is the reason affecting such problem. Here I also attached the warning messages. Please let me know if there is a solution to fix this. Thank you very much! YMRS_Sum<-c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 2, 2, 0, 0, 0, 1, 1, 4, 0, 0, 2, 0, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 3, 1, 4, 2, 1, 2, 0, 1, 0, 0, 0, 0, 2, 0, 0, 3, 0, 0, 0, 0, 2, 0, 4, 0, 2, 0, 0) resid<-c(80.1150, 84.0279, 88.3736, 90.1557, 55.1979, 78.4293, 87.4367, 74.0271, 80.8871, 91.5685, 82.4154, 73.3080, 66.7786, 70.2486, 82.4971, 77.3792, 70.7731, 66.9593, 85.5515 81.4071, 68.6646, 89.8271, 91.6041, 85.1980, 80.6071, 86.4362, 86.2915, 86.4493, 87.8664, 84.5150, 64.4975, 79.1246, 84.9350, 89.1608, 92.7546, 70.0253, 81.4146, 73.2755, 82.5200, 79.7164, 92.0786, 82.5633, 84.4336, 84.0193, 64.8029, 87.4864, 86.3338, 75.6758, 86.8567, 85.1077, 88.9533, 81.7240, 84.1713, 80.0400, 77.6050, 81.4436, 83.8379, 72.5050, 80.5423, 83.2564, 84.1436, 90.0662, 84.5293, 81.6771, 90.6425, 90.3285, 76.2371, 87.3625, 70.7917, 77.0993, 88.3608, 89.7200, 79.1031, 79.7421, 84.2469, 83.9371, 73.8800, 89.3921, 89.3900, 86.8921, 85.7036, 85.2664, 83.8700, 90.5493)> scatter.smooth(YMRS_Sum,resid)Warning messages: 1: at -0.02 2: radius 0.0004 3: all data on boundary of neighborhood. make span bigger 4: pseudoinverse used at -0.02 5: neighborhood radius 0.02 6: reciprocal condition number nan 7: zero-width neighborhood. make span bigger 8: There are other near singularities as well. 1 9: at -0.02 10: radius 0.0004 11: all data on boundary of neighborhood. make span bigger 12: pseudoinverse used at -0.02 13: neighborhood radius 0.02 14: reciprocal condition number nan 15: zero-width neighborhood. make span bigger 16: There are other near singularities as well. 1 17: at -0.02 18: radius 0.0004 19: all data on boundary of neighborhood. make span bigger 20: pseudoinverse used at -0.02 21: neighborhood radius 0.02 22: reciprocal condition number nan 23: zero-width neighborhood. make span bigger 24: There are other near singularities as well. 1 25: at -0.02 26: radius 0.0004 27: all data on boundary of neighborhood. make span bigger 28: pseudoinverse used at -0.02 29: neighborhood radius 0.02 30: reciprocal condition number nan 31: zero-width neighborhood. make span bigger 32: There are other near singularities as well. 1 33: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize FALSE, ... : at -0.02 34: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize FALSE, ... : radius 0.0004 35: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize FALSE, ... : all data on boundary of neighborhood. make span bigger 36: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize FALSE, ... : pseudoinverse used at -0.02 37: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize FALSE, ... : neighborhood radius 0.02 38: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize FALSE, ... : reciprocal condition number nan 39: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize FALSE, ... : zero-width neighborhood. make span bigger 40: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize FALSE, ... : There are other near singularities as well. 1 Thanks, Phoebe
The fundamental problem is that you only have five distinct x values. lowess cannot work in this situation. Try side-by-side boxplots: boxplot(resid.value ~ YMRS_Sum) -Ista On Tue, Oct 19, 2010 at 5:43 PM, phoebe kong <sityeekong at gmail.com> wrote:> Hi there, > > I would like to draw a scatter plot and fit a smooth line by loess. > Below is the data. > However, the curve line started from 0, which my "resid" list doesn't > consist of 0 value. > It returned some warnings which I don't know if this is the reason > affecting such problem. Here I also attached the warning messages. > Please let me know if there is a solution to fix this. Thank you very > much! > > YMRS_Sum<-c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, > 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 2, 2, 0, 0, 0, 1, 1, > 4, 0, 0, 2, 0, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 3, 1, 4, 2, 1, 2, 0, 1, > 0, 0, 0, 0, 2, 0, 0, 3, 0, 0, 0, 0, 2, 0, 4, 0, 2, 0, 0) > > resid<-c(80.1150, 84.0279, 88.3736, 90.1557, 55.1979, 78.4293, > 87.4367, 74.0271, 80.8871, 91.5685, 82.4154, 73.3080, 66.7786, > 70.2486, 82.4971, 77.3792, 70.7731, 66.9593, 85.5515 81.4071, 68.6646, > 89.8271, 91.6041, 85.1980, 80.6071, 86.4362, 86.2915, 86.4493, > 87.8664, 84.5150, 64.4975, 79.1246, 84.9350, 89.1608, 92.7546, > 70.0253, 81.4146, 73.2755, 82.5200, 79.7164, 92.0786, 82.5633, > 84.4336, 84.0193, 64.8029, 87.4864, 86.3338, 75.6758, 86.8567, > 85.1077, 88.9533, 81.7240, 84.1713, 80.0400, 77.6050, 81.4436, > 83.8379, 72.5050, 80.5423, 83.2564, 84.1436, 90.0662, 84.5293, > 81.6771, 90.6425, 90.3285, 76.2371, 87.3625, 70.7917, 77.0993, > 88.3608, 89.7200, 79.1031, 79.7421, 84.2469, 83.9371, 73.8800, > 89.3921, 89.3900, 86.8921, 85.7036, 85.2664, 83.8700, 90.5493) > >> scatter.smooth(YMRS_Sum,resid) > > Warning messages: > 1: at ?-0.02 > 2: radius ?0.0004 > 3: all data on boundary of neighborhood. make span bigger > 4: pseudoinverse used at -0.02 > 5: neighborhood radius 0.02 > 6: reciprocal condition number ?nan > 7: zero-width neighborhood. make span bigger > 8: There are other near singularities as well. 1 > 9: at ?-0.02 > 10: radius ?0.0004 > 11: all data on boundary of neighborhood. make span bigger > 12: pseudoinverse used at -0.02 > 13: neighborhood radius 0.02 > 14: reciprocal condition number ?nan > 15: zero-width neighborhood. make span bigger > 16: There are other near singularities as well. 1 > 17: at ?-0.02 > 18: radius ?0.0004 > 19: all data on boundary of neighborhood. make span bigger > 20: pseudoinverse used at -0.02 > 21: neighborhood radius 0.02 > 22: reciprocal condition number ?nan > 23: zero-width neighborhood. make span bigger > 24: There are other near singularities as well. 1 > 25: at ?-0.02 > 26: radius ?0.0004 > 27: all data on boundary of neighborhood. make span bigger > 28: pseudoinverse used at -0.02 > 29: neighborhood radius 0.02 > 30: reciprocal condition number ?nan > 31: zero-width neighborhood. make span bigger > 32: There are other near singularities as well. 1 > 33: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ?... : > ?at ?-0.02 > 34: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ?... : > ?radius ?0.0004 > 35: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ?... : > ?all data on boundary of neighborhood. make span bigger > 36: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ?... : > ?pseudoinverse used at -0.02 > 37: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ?... : > ?neighborhood radius 0.02 > 38: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ?... : > ?reciprocal condition number ?nan > 39: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ?... : > ?zero-width neighborhood. make span bigger > 40: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ?... : > ?There are other near singularities as well. 1 > > > Thanks, > Phoebe > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. >-- Ista Zahn Graduate student University of Rochester Department of Clinical and Social Psychology http://yourpsyche.org
On Oct 19, 2010, at 5:43 PM, phoebe kong wrote:> Hi there, > > I would like to draw a scatter plot and fit a smooth line by loess. > Below is the data. > However, the curve line started from 0, which my "resid" list doesn't > consist of 0 value. > It returned some warnings which I don't know if this is the reason > affecting such problem. Here I also attached the warning messages. > Please let me know if there is a solution to fix this. Thank you very > much!Yes, Increase the span parameter value. (And replace the missing comma in line 3 of the offered example.) -- David> > YMRS_Sum<-c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, > 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 2, 2, 0, 0, 0, 1, 1, > 4, 0, 0, 2, 0, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 3, 1, 4, 2, 1, 2, 0, 1, > 0, 0, 0, 0, 2, 0, 0, 3, 0, 0, 0, 0, 2, 0, 4, 0, 2, 0, 0) > > resid<-c(80.1150, 84.0279, 88.3736, 90.1557, 55.1979, 78.4293, > 87.4367, 74.0271, 80.8871, 91.5685, 82.4154, 73.3080, 66.7786, > 70.2486, 82.4971, 77.3792, 70.7731, 66.9593, 85.5515 81.4071, 68.6646, > 89.8271, 91.6041, 85.1980, 80.6071, 86.4362, 86.2915, 86.4493, > 87.8664, 84.5150, 64.4975, 79.1246, 84.9350, 89.1608, 92.7546, > 70.0253, 81.4146, 73.2755, 82.5200, 79.7164, 92.0786, 82.5633, > 84.4336, 84.0193, 64.8029, 87.4864, 86.3338, 75.6758, 86.8567, > 85.1077, 88.9533, 81.7240, 84.1713, 80.0400, 77.6050, 81.4436, > 83.8379, 72.5050, 80.5423, 83.2564, 84.1436, 90.0662, 84.5293, > 81.6771, 90.6425, 90.3285, 76.2371, 87.3625, 70.7917, 77.0993, > 88.3608, 89.7200, 79.1031, 79.7421, 84.2469, 83.9371, 73.8800, > 89.3921, 89.3900, 86.8921, 85.7036, 85.2664, 83.8700, 90.5493) > >> scatter.smooth(YMRS_Sum,resid) > > Warning messages: > 1: at -0.02 > 2: radius 0.0004 > 3: all data on boundary of neighborhood. make span bigger > 4: pseudoinverse used at -0.02 > 5: neighborhood radius 0.02 > 6: reciprocal condition number nan > 7: zero-width neighborhood. make span bigger > 8: There are other near singularities as well. 1 > 9: at -0.02 > 10: radius 0.0004 > 11: all data on boundary of neighborhood. make span bigger > 12: pseudoinverse used at -0.02 > 13: neighborhood radius 0.02 > 14: reciprocal condition number nan > 15: zero-width neighborhood. make span bigger > 16: There are other near singularities as well. 1 > 17: at -0.02 > 18: radius 0.0004 > 19: all data on boundary of neighborhood. make span bigger > 20: pseudoinverse used at -0.02 > 21: neighborhood radius 0.02 > 22: reciprocal condition number nan > 23: zero-width neighborhood. make span bigger > 24: There are other near singularities as well. 1 > 25: at -0.02 > 26: radius 0.0004 > 27: all data on boundary of neighborhood. make span bigger > 28: pseudoinverse used at -0.02 > 29: neighborhood radius 0.02 > 30: reciprocal condition number nan > 31: zero-width neighborhood. make span bigger > 32: There are other near singularities as well. 1 > 33: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ... : > at -0.02 > 34: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ... : > radius 0.0004 > 35: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ... : > all data on boundary of neighborhood. make span bigger > 36: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ... : > pseudoinverse used at -0.02 > 37: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ... : > neighborhood radius 0.02 > 38: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ... : > reciprocal condition number nan > 39: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ... : > zero-width neighborhood. make span bigger > 40: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize > FALSE, ... : > There are other near singularities as well. 1 > > > Thanks, > Phoebe > > ______________________________________________ > R-help at r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code.David Winsemius, MD West Hartford, CT
Hi:
I agree with Ista's point that you shouldn't be doing loess with these
data
(x and y both need to be continuous for loess, but your x is discrete), but
you shouldn't be computing boxplots at each YMRS_Sum value either because
you don't have enough resid observations at Sum = 3 and 4. A combination of
stripplot and average line would seem to be a reasonable thing to do,
though, as long as you don't try to overinterpret it - the standard errors
across Sum values are likely to be quite variable due to the severe
imbalance in sample size. Even error bars don't make sense because there are
two observations at Sum = 3 and three at Sum = 4, so the local standard
deviation/standard error estimates will be quite uncertain.
The lattice version of the plot is rather simple:
library(lattice)
xyplot(resid ~ YMRS_Sum, type = c('p', 'a'), pch = 16, col = 1,
col.line 'red', lwd = 2)
# Add grid and least squares line (spoiler alert - it's essentially flat):
xyplot(resid ~ YMRS_Sum, type = c('g', 'p', 'a',
'r'), pch = 16, col = 1,
lwd = 2)
# To get separate colors for the lines, a panel function is required:
xyplot(resid ~ YMRS_Sum, type = c('g', 'p', 'a'), pch =
16, col= 1, lwd = 2,
panel = function(x, y, ...) {
panel.xyplot(x, y, ...)
panel.lmline(x, y, col.line = 'red', ...) } )
The ggplot2 version is a little more complicated because the means are
computed and plotted by ggplot2:
library(ggplot2)
h <- ggplot(d, aes(x = YMRS_Sum, y = resid))
h + geom_point() + theme_bw() +
stat_summary(fun.y='mean', colour='red',
geom='line', size = 1)
# Add least squares line with confidence bands
h + geom_point() + theme_bw() +
stat_summary(fun.y='mean', colour='red',
geom='line', size = 1) +
geom_smooth(method = 'lm')
I like the confidence bands here because it points up the uncertainty in the
line where the data are sparse, but to get rid of it, use the option se FALSE.
(Zero slope falls well within the confidence bands...)
HTH,
Dennis
On Tue, Oct 19, 2010 at 2:43 PM, phoebe kong <sityeekong@gmail.com> wrote:
> Hi there,
>
> I would like to draw a scatter plot and fit a smooth line by loess.
> Below is the data.
> However, the curve line started from 0, which my "resid" list
doesn't
> consist of 0 value.
> It returned some warnings which I don't know if this is the reason
> affecting such problem. Here I also attached the warning messages.
> Please let me know if there is a solution to fix this. Thank you very
> much!
>
> YMRS_Sum<-c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0,
> 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 2, 0, 0, 2, 2, 0, 0, 0, 1, 1,
> 4, 0, 0, 2, 0, 1, 0, 2, 0, 0, 0, 0, 0, 0, 0, 3, 1, 4, 2, 1, 2, 0, 1,
> 0, 0, 0, 0, 2, 0, 0, 3, 0, 0, 0, 0, 2, 0, 4, 0, 2, 0, 0)
>
> resid<-c(80.1150, 84.0279, 88.3736, 90.1557, 55.1979, 78.4293,
> 87.4367, 74.0271, 80.8871, 91.5685, 82.4154, 73.3080, 66.7786,
> 70.2486, 82.4971, 77.3792, 70.7731, 66.9593, 85.5515 81.4071, 68.6646,
> 89.8271, 91.6041, 85.1980, 80.6071, 86.4362, 86.2915, 86.4493,
> 87.8664, 84.5150, 64.4975, 79.1246, 84.9350, 89.1608, 92.7546,
> 70.0253, 81.4146, 73.2755, 82.5200, 79.7164, 92.0786, 82.5633,
> 84.4336, 84.0193, 64.8029, 87.4864, 86.3338, 75.6758, 86.8567,
> 85.1077, 88.9533, 81.7240, 84.1713, 80.0400, 77.6050, 81.4436,
> 83.8379, 72.5050, 80.5423, 83.2564, 84.1436, 90.0662, 84.5293,
> 81.6771, 90.6425, 90.3285, 76.2371, 87.3625, 70.7917, 77.0993,
> 88.3608, 89.7200, 79.1031, 79.7421, 84.2469, 83.9371, 73.8800,
> 89.3921, 89.3900, 86.8921, 85.7036, 85.2664, 83.8700, 90.5493)
>
> > scatter.smooth(YMRS_Sum,resid)
>
> Warning messages:
> 1: at -0.02
> 2: radius 0.0004
> 3: all data on boundary of neighborhood. make span bigger
> 4: pseudoinverse used at -0.02
> 5: neighborhood radius 0.02
> 6: reciprocal condition number nan
> 7: zero-width neighborhood. make span bigger
> 8: There are other near singularities as well. 1
> 9: at -0.02
> 10: radius 0.0004
> 11: all data on boundary of neighborhood. make span bigger
> 12: pseudoinverse used at -0.02
> 13: neighborhood radius 0.02
> 14: reciprocal condition number nan
> 15: zero-width neighborhood. make span bigger
> 16: There are other near singularities as well. 1
> 17: at -0.02
> 18: radius 0.0004
> 19: all data on boundary of neighborhood. make span bigger
> 20: pseudoinverse used at -0.02
> 21: neighborhood radius 0.02
> 22: reciprocal condition number nan
> 23: zero-width neighborhood. make span bigger
> 24: There are other near singularities as well. 1
> 25: at -0.02
> 26: radius 0.0004
> 27: all data on boundary of neighborhood. make span bigger
> 28: pseudoinverse used at -0.02
> 29: neighborhood radius 0.02
> 30: reciprocal condition number nan
> 31: zero-width neighborhood. make span bigger
> 32: There are other near singularities as well. 1
> 33: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize >
FALSE, ... :
> at -0.02
> 34: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize >
FALSE, ... :
> radius 0.0004
> 35: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize >
FALSE, ... :
> all data on boundary of neighborhood. make span bigger
> 36: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize >
FALSE, ... :
> pseudoinverse used at -0.02
> 37: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize >
FALSE, ... :
> neighborhood radius 0.02
> 38: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize >
FALSE, ... :
> reciprocal condition number nan
> 39: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize >
FALSE, ... :
> zero-width neighborhood. make span bigger
> 40: In simpleLoess(y, x, w, span, degree, FALSE, FALSE, normalize >
FALSE, ... :
> There are other near singularities as well. 1
>
>
> Thanks,
> Phoebe
>
> ______________________________________________
> R-help@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
[[alternative HTML version deleted]]