Hi I am using the quantreg package for median regression for a large series of subsets of data. It works fabulously for all but one subset. When it reaches this subset, R takes the command and never responds. I end up having to kill R and restart it. It appears to be something with the particular data subset, but I can't pinpoint the problem. Here are some details Operating system: Windows 7 R version: 2.12.1 Here is the data and the rq command that gives me trouble: library(quantreg) x <- c(-0.340778085786686,-0.573639751645382,-0.663932762810308,-0.438591328531796,0.302202380883637,-0.675558868120683,-0.764547425063882,-0.751796238115147,-0.481835451050657,-0.588287304540034,-0.622315341312595,-0.542777491991884,-0.552343921339062,-0.587743299883,-0.758233854317935,-0.783134744819092,-0.97774093234124,0.859832969267456,0.69037126308323,0.185409334523753,-0.432951955490942,-0.988120972598647,0.243223425575187) y<- c(2.35531739878456,-1,2.26484142532915,-1,2.86895579641295,2.6655997506336,-1,1.33021078457153,-1,-1,-1,1.82633400562222,1.60831204269733,-1,2.45313479655685,-1,-1,-1,-1,-1,-1,-1,-1) fit1 <- rq(y ~ x, tau = .5) Any feedback would be greatly appreciated. Thanks Steve -- ==============================================Steven R. Corsi Phone: (608) 821-3835 Research Hydrologist email: srcorsi at usgs.gov U.S. Geological Survey Wisconsin Water Science Center 8505 Research Way Middleton, WI 53562
Steve: I would guess that the problem relates to the large number of tied values of -1 in your dependent y variable. You could randomly jitter these y = -1 by adding a random uniform number between, say, [ -0.01, 0.01] and see if the rq() converges to a solution. Then you would know that was the numeric computing issue. Then the question would be what to do next? Seems like a funny data problem with a point mass of responses at -1. Perhaps only higher quantiles, say 0.80, are going to give usable estimates. Perhaps the jittered responses can yield a reasonably interpreted estimate. Brian Brian S. Cade, PhD U. S. Geological Survey Fort Collins Science Center 2150 Centre Ave., Bldg. C Fort Collins, CO 80526-8818 email: brian_cade@usgs.gov tel: 970 226-9326 From: Steven R Corsi <srcorsi@usgs.gov> To: r-help@r-project.org Date: 07/21/2011 04:04 PM Subject: [R] Quantreg-rq crashing trouble Sent by: r-help-bounces@r-project.org Hi I am using the quantreg package for median regression for a large series of subsets of data. It works fabulously for all but one subset. When it reaches this subset, R takes the command and never responds. I end up having to kill R and restart it. It appears to be something with the particular data subset, but I can't pinpoint the problem. Here are some details Operating system: Windows 7 R version: 2.12.1 Here is the data and the rq command that gives me trouble: library(quantreg) x <- c(-0.340778085786686,-0.573639751645382,-0.663932762810308,-0.438591328531796,0.302202380883637,-0.675558868120683,-0.764547425063882,-0.751796238115147,-0.481835451050657,-0.588287304540034,-0.622315341312595,-0.542777491991884,-0.552343921339062,-0.587743299883,-0.758233854317935,-0.783134744819092,-0.97774093234124,0.859832969267456,0.69037126308323,0.185409334523753,-0.432951955490942,-0.988120972598647,0.243223425575187) y<- c(2.35531739878456,-1,2.26484142532915,-1,2.86895579641295,2.6655997506336,-1,1.33021078457153,-1,-1,-1,1.82633400562222,1.60831204269733,-1,2.45313479655685,-1,-1,-1,-1,-1,-1,-1,-1) fit1 <- rq(y ~ x, tau = .5) Any feedback would be greatly appreciated. Thanks Steve -- ==============================================Steven R. Corsi Phone: (608) 821-3835 Research Hydrologist email: srcorsi@usgs.gov U.S. Geological Survey Wisconsin Water Science Center 8505 Research Way Middleton, WI 53562 ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]]
Brian Thanks for the quick response! You have found a solution. I tried this and it worked well. For my current purpose it is perfectly fine to use jitter and the result is just as useful. For the one problem subset (of 160 subsets) the actual regression won't be super useful, but at least it won't crash the rest of the process to run all subsets. Those -1 values are actually log-transformed non-detects, and I am just running a visualization routine to get a series of panel scatters for a first look at the data. Thanks for the help... Steve ==============================================Steven R. Corsi Phone: (608) 821-3835 Research Hydrologist email: srcorsi@usgs.gov U.S. Geological Survey Wisconsin Water Science Center 8505 Research Way Middleton, WI 53562 ============================================== On 7/21/2011 5:37 PM, Brian S Cade wrote:> Steve: I would guess that the problem relates to the large number of > tied values of -1 in your dependent y variable. You could randomly > jitter these y = -1 by adding a random uniform number between, say, [ > -0.01, 0.01] and see if the rq() converges to a solution. Then you > would know that was the numeric computing issue. Then the question > would be what to do next? Seems like a funny data problem with a > point mass of responses at -1. Perhaps only higher quantiles, say > 0.80, are going to give usable estimates. Perhaps the jittered > responses can yield a reasonably interpreted estimate. > > Brian > > Brian S. Cade, PhD > > U. S. Geological Survey > Fort Collins Science Center > 2150 Centre Ave., Bldg. C > Fort Collins, CO 80526-8818 > > email: brian_cade@usgs.gov > tel: 970 226-9326 > > > From: Steven R Corsi <srcorsi@usgs.gov> > To: r-help@r-project.org > Date: 07/21/2011 04:04 PM > Subject: [R] Quantreg-rq crashing trouble > Sent by: r-help-bounces@r-project.org > > > ------------------------------------------------------------------------ > > > > Hi > > I am using the quantreg package for median regression for a large series > of subsets of data. It works fabulously for all but one subset. When it > reaches this subset, R takes the command and never responds. I end up > having to kill R and restart it. > > It appears to be something with the particular data subset, but I can't > pinpoint the problem. > > Here are some details > Operating system: Windows 7 > R version: 2.12.1 > > Here is the data and the rq command that gives me trouble: > > library(quantreg) > > x <- > c(-0.340778085786686,-0.573639751645382,-0.663932762810308,-0.438591328531796,0.302202380883637,-0.675558868120683,-0.764547425063882,-0.751796238115147,-0.481835451050657,-0.588287304540034,-0.622315341312595,-0.542777491991884,-0.552343921339062,-0.587743299883,-0.758233854317935,-0.783134744819092,-0.97774093234124,0.859832969267456,0.69037126308323,0.185409334523753,-0.432951955490942,-0.988120972598647,0.243223425575187) > > y<- > > c(2.35531739878456,-1,2.26484142532915,-1,2.86895579641295,2.6655997506336,-1,1.33021078457153,-1,-1,-1,1.82633400562222,1.60831204269733,-1,2.45313479655685,-1,-1,-1,-1,-1,-1,-1,-1) > > > fit1 <- rq(y ~ x, tau = .5) > > > Any feedback would be greatly appreciated. > Thanks > Steve > > -- > ==============================================> Steven R. Corsi Phone: (608) 821-3835 > Research Hydrologist email: srcorsi@usgs.gov > U.S. Geological Survey > Wisconsin Water Science Center > 8505 Research Way > Middleton, WI 53562 > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > <http://www.r-project.org/posting-guide.html> > and provide commented, minimal, self-contained, reproducible code. > >[[alternative HTML version deleted]]