Dear all, When calling rlm with the following data, I get an error. (R v.1.8.1, WinXP Pro 2002 with service pack 1.)> d <- na.omit(data.frame(CPRATIO, HEIGHTZ, FAMILYID)) > c <- tapply(d$CPRATIO, d$FAMILYID, mean) > h <- tapply(d$HEIGHTZ, d$FAMILYID, mean) > c1 2 3 6 7 9 10 11 6.000000 2.500000 3.250000 4.000000 2.333333 2.400000 2.750000 2.000000 12 13 14 15 16 18 21 22 4.000000 5.000000 4.000000 7.000000 4.500000 3.500000 4.000000 5.000000 23 25 30 31 35 39 40 44 6.000000 3.000000 9.000000 10.000000 8.000000 10.000000 4.000000 6.000000 45 47 50 4.500000 5.500000 5.000000> h1 2 3 6 7 9 10 -2.7916667 -1.0246838 -2.0681170 -1.3256766 -3.1122708 -2.1444948 -2.5806538 11 12 13 14 15 16 18 -0.6791551 -0.1278583 -3.7737900 -3.3077561 -2.4615233 -3.4016225 -2.2484673 21 22 23 25 30 31 35 -0.8938528 -2.2521325 -2.3461999 -2.1405470 -2.7657994 -3.3588822 -2.8144453 39 40 44 45 47 50 -3.8521439 -2.1882352 -1.7531826 -3.6464482 -2.2158240 -2.7162693> out <- rlm(h ~ c)Error in model.matrix.default(mt, mf, contrasts) : cannot allocate vector of length 1077237505 If I make the same rlm call a second or third time, R usually crashes. Any help or suggestions would be greatly appreciated. These commands were run after a much larger script had seemingly run successfully. But, I had no problem calling, e.g., rlm(HEIGHTZ ~ CPRATIO), after the same script had run. Although I have made some minor changes in the script, I also didn't have this problem when using R 1.7. Many thanks in advance, Ed. -- Edward H. Hagen Institute for Theoretical Biology phone: +49/30 2093-8649 Humboldt-Universit??t zu Berlin fax: +49/30 2093-8801 Invalidenstra??e 43 http://itb.biologie.hu-berlin.de/~hagen 10115 Berlin, Germany
You have not given us nearly enough info. On my WinXP laptop with 512MB RAM, R-1.8.1, I get:> system.time(out <- rlm(h ~ c))[1] 0.03 0.00 0.04 NA NA> outCall: rlm(formula = h ~ c) Converged in 8 iterations Coefficients: (Intercept) c -1.4720930 -0.1843265 Degrees of freedom: 27 total; 25 residual Scale estimate: 0.541> h1 2 3 6 7 9 10 -2.7916667 -1.0246838 -2.0681170 -1.3256766 -3.1122708 -2.1444948 -2.5806538 11 12 13 14 15 16 18 -0.6791551 -0.1278583 -3.7737900 -3.3077561 -2.4615233 -3.4016225 -2.2484673 21 22 23 25 30 31 35 -0.8938528 -2.2521325 -2.3461999 -2.1405470 -2.7657994 -3.3588822 -2.8144453 39 40 44 45 47 50 -3.8521439 -2.1882352 -1.7531826 -3.6464482 -2.2158240 -2.7162693> c1 2 3 6 7 9 10 11 6.000000 2.500000 3.250000 4.000000 2.333333 2.400000 2.750000 2.000000 12 13 14 15 16 18 21 22 4.000000 5.000000 4.000000 7.000000 4.500000 3.500000 4.000000 5.000000 23 25 30 31 35 39 40 44 6.000000 3.000000 9.000000 10.000000 8.000000 10.000000 4.000000 6.000000 45 47 50 4.500000 5.500000 5.000000 What OS / R version / MASS version (you are using MASS, no?) are you using? What's your hardware? Do you have lots of other stuff in the workspace when you tried to run this? What does gc() tell you? Andy> From: Ed Hagen > > Dear all, > > When calling rlm with the following data, I get an error. (R > v.1.8.1, > WinXP Pro 2002 with service pack 1.) > > > d <- na.omit(data.frame(CPRATIO, HEIGHTZ, FAMILYID)) > > c <- tapply(d$CPRATIO, d$FAMILYID, mean) > > h <- tapply(d$HEIGHTZ, d$FAMILYID, mean) > > c > 1 2 3 6 7 9 10 > 11 > 6.000000 2.500000 3.250000 4.000000 2.333333 2.400000 2.750000 > 2.000000 > 12 13 14 15 16 18 21 > 22 > 4.000000 5.000000 4.000000 7.000000 4.500000 3.500000 4.000000 > 5.000000 > 23 25 30 31 35 39 40 > 44 > 6.000000 3.000000 9.000000 10.000000 8.000000 10.000000 4.000000 > 6.000000 > 45 47 50 > 4.500000 5.500000 5.000000 > > h > 1 2 3 6 7 9 > 10 > -2.7916667 -1.0246838 -2.0681170 -1.3256766 -3.1122708 -2.1444948 > -2.5806538 > 11 12 13 14 15 16 > 18 > -0.6791551 -0.1278583 -3.7737900 -3.3077561 -2.4615233 -3.4016225 > -2.2484673 > 21 22 23 25 30 31 > 35 > -0.8938528 -2.2521325 -2.3461999 -2.1405470 -2.7657994 -3.3588822 > -2.8144453 > 39 40 44 45 47 50 > -3.8521439 -2.1882352 -1.7531826 -3.6464482 -2.2158240 -2.7162693 > > out <- rlm(h ~ c) > Error in model.matrix.default(mt, mf, contrasts) : > cannot allocate vector of length 1077237505 > > > If I make the same rlm call a second or third time, R usually crashes. > > Any help or suggestions would be greatly appreciated. These commands > were run after a much larger script had seemingly run > successfully. But, > I had no problem calling, e.g., rlm(HEIGHTZ ~ CPRATIO), after the same > script had run. Although I have made some minor changes in > the script, > I also didn't have this problem when using R 1.7. > > Many thanks in advance, > > Ed. > > > -- > Edward H. Hagen Institute for Theoretical Biology > phone: +49/30 2093-8649 Humboldt-Universit??t zu Berlin > fax: +49/30 2093-8801 Invalidenstra??e 43 > http://itb.biologie.hu-berlin.de/~hagen 10115 Berlin, Germany > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > >
My apologies for not reading:> When calling rlm with the following data, I get an error. (R v.1.8.1, > WinXP Pro 2002 with service pack 1.)If you get the error toward the end of a long script, you might have too many things taking up space in the workspace. Try rm() as many objects as you can in places within the script, and perhaps add a few calls to gc() as well and see if that helps. Andy> From: Liaw, Andy > > You have not given us nearly enough info. On my WinXP laptop > with 512MB > RAM, R-1.8.1, I get: > > > system.time(out <- rlm(h ~ c)) > [1] 0.03 0.00 0.04 NA NA > > out > Call: > rlm(formula = h ~ c) > Converged in 8 iterations > > Coefficients: > (Intercept) c > -1.4720930 -0.1843265 > > Degrees of freedom: 27 total; 25 residual > Scale estimate: 0.541 > > h > 1 2 3 6 7 > 9 10 > > -2.7916667 -1.0246838 -2.0681170 -1.3256766 -3.1122708 > -2.1444948 -2.5806538 > > 11 12 13 14 15 > 16 18 > > -0.6791551 -0.1278583 -3.7737900 -3.3077561 -2.4615233 > -3.4016225 -2.2484673 > > 21 22 23 25 30 > 31 35 > > -0.8938528 -2.2521325 -2.3461999 -2.1405470 -2.7657994 > -3.3588822 -2.8144453 > > 39 40 44 45 47 50 > -3.8521439 -2.1882352 -1.7531826 -3.6464482 -2.2158240 -2.7162693 > > c > 1 2 3 6 7 9 10 > 11 > 6.000000 2.500000 3.250000 4.000000 2.333333 2.400000 2.750000 > 2.000000 > 12 13 14 15 16 18 21 > 22 > 4.000000 5.000000 4.000000 7.000000 4.500000 3.500000 4.000000 > 5.000000 > 23 25 30 31 35 39 40 > 44 > 6.000000 3.000000 9.000000 10.000000 8.000000 10.000000 4.000000 > 6.000000 > 45 47 50 > 4.500000 5.500000 5.000000 > > What OS / R version / MASS version (you are using MASS, no?) > are you using? > What's your hardware? Do you have lots of other stuff in the > workspace when > you tried to run this? What does gc() tell you? > > Andy > > > From: Ed Hagen > > > > Dear all, > > > > When calling rlm with the following data, I get an error. (R > > v.1.8.1, > > WinXP Pro 2002 with service pack 1.) > > > > > d <- na.omit(data.frame(CPRATIO, HEIGHTZ, FAMILYID)) > > > c <- tapply(d$CPRATIO, d$FAMILYID, mean) > > > h <- tapply(d$HEIGHTZ, d$FAMILYID, mean) > > > c > > 1 2 3 6 7 > 9 10 > > 11 > > 6.000000 2.500000 3.250000 4.000000 2.333333 > 2.400000 2.750000 > > 2.000000 > > 12 13 14 15 16 > 18 21 > > 22 > > 4.000000 5.000000 4.000000 7.000000 4.500000 > 3.500000 4.000000 > > 5.000000 > > 23 25 30 31 35 > 39 40 > > 44 > > 6.000000 3.000000 9.000000 10.000000 8.000000 > 10.000000 4.000000 > > 6.000000 > > 45 47 50 > > 4.500000 5.500000 5.000000 > > > h > > 1 2 3 6 7 9 > > 10 > > -2.7916667 -1.0246838 -2.0681170 -1.3256766 -3.1122708 -2.1444948 > > -2.5806538 > > 11 12 13 14 15 16 > > 18 > > -0.6791551 -0.1278583 -3.7737900 -3.3077561 -2.4615233 -3.4016225 > > -2.2484673 > > 21 22 23 25 30 31 > > 35 > > -0.8938528 -2.2521325 -2.3461999 -2.1405470 -2.7657994 -3.3588822 > > -2.8144453 > > 39 40 44 45 47 50 > > -3.8521439 -2.1882352 -1.7531826 -3.6464482 -2.2158240 -2.7162693 > > > out <- rlm(h ~ c) > > Error in model.matrix.default(mt, mf, contrasts) : > > cannot allocate vector of length 1077237505 > > > > > > If I make the same rlm call a second or third time, R > usually crashes. > > > > Any help or suggestions would be greatly appreciated. > These commands > > were run after a much larger script had seemingly run > > successfully. But, > > I had no problem calling, e.g., rlm(HEIGHTZ ~ CPRATIO), > after the same > > script had run. Although I have made some minor changes in > > the script, > > I also didn't have this problem when using R 1.7. > > > > Many thanks in advance, > > > > Ed. > > > > > > -- > > Edward H. Hagen Institute for Theoretical Biology > > phone: +49/30 2093-8649 Humboldt-Universit??t zu Berlin > > fax: +49/30 2093-8801 Invalidenstra??e 43 > > http://itb.biologie.hu-berlin.de/~hagen 10115 Berlin, Germany > > > > ______________________________________________ > > R-help at stat.math.ethz.ch mailing list > > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide! > > http://www.R-project.org/posting-guide.html > > > > > > ______________________________________________ > R-help at stat.math.ethz.ch mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > > > -------------------------------------------------------------- > ---------------- > Notice: This e-mail message, together with any attachments, > contains information of Merck & Co., Inc. (One Merck Drive, > Whitehouse Station, New Jersey, USA 08889), and/or its > affiliates (which may be known outside the United States as > Merck Frosst, Merck Sharp & Dohme or MSD and in Japan, as > Banyu) that may be confidential, proprietary copyrighted > and/or legally privileged. It is intended solely for the use > of the individual or entity named on this message. If you > are not the intended recipient, and have received this > message in error, please notify us immediately by reply > e-mail and then delete it from your system. > -------------------------------------------------------------- > ---------------- >------------------------------------------------------------------------------ Notice: This e-mail message, together with any attachments,...{{dropped}}
Erin Hodgess wrote:> Please don't use c as a vector name. There is a function c and > that function gets overwritten.Thanks for that tip. I was actually using different variable names in my script, but simplified them for the posting. I just tried it using the names 'cpavg' and 'havg'. The first time I called rlm it worked, but I got the following errors the second and third time I ran it: > out <- rlm(havg ~ cpavg) Error: cannot allocate vector of size 4178828 Kb > out <- rlm(havg ~ cpavg) Error in model.matrix.default(mt, mf, contrasts) : negative length vectors are not allowed Liaw, Andy wrote: > You have not given us nearly enough info. > What OS / R version / MASS version (you are using MASS, no?) are you > using? WinXP Pro 2002. Service Pack 1. 512K RAM. R v. 1.8.1. MASS v. 7.1-11. AMD Athlon XP 2600+, 1.91. GHz. > Do you have lots of other stuff in the workspace when > you tried to run this? Yes. A lot of stuff is needed to produce the variables CPRATIO and HEIGHTZ > What does gc() tell you? After running the script, but before running the code in question: used (Mb) gc trigger (Mb) Ncells 720521 19.3 1073225 28.7 Vcells 209563 1.6 786432 6.0 gc() called during and after running the code in question: > d2 <- na.omit(data.frame(CPRATIO, HEIGHTZ, FAMILYID)) > cavg <- tapply(d2$CPRATIO, d2$FAMILYID, mean) > havg <- tapply(d2$HEIGHTZ, d2$FAMILYID, mean) > gc() used (Mb) gc trigger (Mb) Ncells 720802 19.3 1166886 31.2 Vcells 210286 1.7 786432 6.0 > out <- rlm(havg ~ cavg) Error in model.matrix.default(mt, mf, contrasts) : cannot allocate vector of length 2146959361 > gc() used (Mb) gc trigger (Mb) Ncells 720819 19.3 1166886 31.2 Vcells 210334 1.7 786432 6.0 As I noted in my first posting, rlm works fine with CPRATIO and HEIGHTZ, which are much larger datasets, if that's any clue (though even CPRATIO and HEIGHTZ are still quite small: 85 cases each). After running the main script, I ran > out <- rlm(HEIGHTZ ~ CPRATIO) about 20 consecutive times with no problems. Yet running rlm with the smaller havg and cavg immediately produces errors, and R will usually crash after the second or third attempt. Many thanks for the quick responses, Ed. -- Edward H. Hagen Institute for Theoretical Biology phone: +49/30 2093-8649 Humboldt-Universit??t zu Berlin fax: +49/30 2093-8801 Invalidenstra??e 43 http://itb.biologie.hu-berlin.de/~hagen 10115 Berlin, Germany
> Given what you have tried, I wonder whether the problem is > that 'cavg' and 'havg', produced by tapply, are not the > simple vectors that 'rlm' expects. A simple way to check: > > cavg <- as.vector(cavg) > havg <- as.vector(havg) > > then run rlm.That worked! Many thanks. Has there been a change in either MASS or R in this regard since R 1.7? It was only after updating to R 1.8.1 that I had this problem. Thanks again to all for your rapid replies, Ed. -- Edward H. Hagen Institute for Theoretical Biology phone: +49/30 2093-8649 Humboldt-Universit??t zu Berlin fax: +49/30 2093-8801 Invalidenstra??e 43 http://itb.biologie.hu-berlin.de/~hagen 10115 Berlin, Germany