Dear all, ? the following example somehow uses the "old data" (X) to make the predictions, but not the new data Xnew as intended. ? y <- rnorm(100) X <- matrix(rnorm(100*10), ncol=10) lm <- lm(y~X) Xnew <- matrix(rnorm(100*20), ncol=10) ynew <- predict(lm, newdata=as.data.frame(Xnew)) #prediction in not made for Xnew ? How can I foce predict.lm to use use the new data? ? Thank you very much for your efforts in advance! ? Best, ? Martin
Since you passed a matrix to lm() and then a data.frame to predict(), predict can't match up what variables to use for the prediction so it falls back on the original data. This seems to work:> set.seed(42) > y <- rnorm(100) > X <- matrix(rnorm(100*10), ncol=10) > Xd <- data.frame(X) > lm <- lm(y~., Xd) > Xnew <- matrix(rnorm(100*20), ncol=10) > Xnewd <- data.frame(Xnew) > ynew <- predict(lm, newdata=Xnewd) > head(ynew)1 2 3 4 5 6 0.35404067 0.14073495 -0.45442499 0.31065562 -0.02091366 0.25358175> head(predict(lm))1 2 3 4 5 6 0.75474817 0.06024122 -0.27221466 -0.20344713 0.20218135 -0.24045859>------------------------------------- David L Carlson Department of Anthropology Texas A&M University College Station, TX 77840-4352 -----Original Message----- From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of Martin Spindler Sent: Wednesday, April 29, 2015 9:21 AM To: r-help at r-project.org Subject: [R] Problem with predict.lm() Dear all, ? the following example somehow uses the "old data" (X) to make the predictions, but not the new data Xnew as intended. ? y <- rnorm(100) X <- matrix(rnorm(100*10), ncol=10) lm <- lm(y~X) Xnew <- matrix(rnorm(100*20), ncol=10) ynew <- predict(lm, newdata=as.data.frame(Xnew)) #prediction in not made for Xnew ? How can I foce predict.lm to use use the new data? ? Thank you very much for your efforts in advance! ? Best, ? Martin ______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Hi, It seems to be working in my R. Although it is throwing the warning message Warning message:'newdata' had 200 rows but variables found have 100 rows? ?y? [1] -1.071307580 ?0.102414204 -0.965046207 ?1.386057875 ?0.726835339? [6] -0.186549950 -0.777144258 ?1.137210314 -1.069446945 -0.696084338?[11] -0.467686285 ?0.997316781 ?0.776265490 -1.385720997 -0.007438381?[16] ?0.302821728 ?0.024075173 -0.590401970 ?0.877104292 ?0.652724314?[21] ?2.142135078 ?2.023051454 -0.547221960 ?0.342600702 ?0.080848203?[26] ?0.074609232 ?0.255946197 -0.191242759 ?1.036445108 ?0.895068954?[31] ?0.589477883 ?0.123230750 ?1.467210735 -1.636657283 -0.131504288?[36] -0.665000122 ?0.390977868 ?0.546802014 ?0.445498091 ?1.063872749?[41] ?1.406788635 -0.037147550 -1.047190960 -0.189105987 ?0.069617165?[46] -0.049760285 -1.454279226 ?0.358351554 ?0.246587937 -0.060735329?[51] ?1.664530111 -0.475931484 ?0.405480604 ?1.560446941 -0.030537155?[56] -1.060319583 -1.828624216 -0.429391165 ?0.301697744 -0.029593593?[61] -1.696307754 ?0.342678986 -0.433965195 -0.947338037 ?0.318186677?[66] ?0.539630789 -1.354555193 ?0.086168702 ?0.002950100 ?1.783486665?[71] -1.182419158 -0.930524123 ?0.376579158 -1.085035387 ?1.186125702?[76] ?0.719738391 -0.486692820 -2.105396602 ?0.531238276 ?1.302812739?[81] ?0.347851244 ?0.016452693 ?0.417535566 ?0.277705766 ?2.286275977?[86] ?1.610183518 ?2.032037030 ?1.319074179 ?1.129375593 ?0.176684807?[91] -0.630517144 ?1.302785450 ?0.994275267 -0.060116993 -0.655966924?[96] ?1.628197169 ?1.935532651 -1.635783346 -1.172511179 ?1.238336597> ynew? ? ? ? ? ?1 ? ? ? ? ? ?2 ? ? ? ? ? ?3 ? ? ? ? ? ?4 ? ? ? ? ? ?5 ? ? ? ? ? ?6?-0.270916637 ?0.169149841 ?0.191348061 -0.009541999 ?0.112027155 ?0.016242323?? ? ? ? ? ?7 ? ? ? ? ? ?8 ? ? ? ? ? ?9 ? ? ? ? ? 10 ? ? ? ? ? 11 ? ? ? ? ? 12?-0.062178365 ?0.275322344 ?0.397030485 ?0.565078468 ?0.301230303 ?0.305405674?? ? ? ? ? 13 ? ? ? ? ? 14 ? ? ? ? ? 15 ? ? ? ? ? 16 ? ? ? ? ? 17 ? ? ? ? ? 18??0.552136794 -0.151275710 ?0.470280882 ?0.349631748 ?0.022005869 ?0.181384646?? ? ? ? ? 19 ? ? ? ? ? 20 ? ? ? ? ? 21 ? ? ? ? ? 22 ? ? ? ? ? 23 ? ? ? ? ? 24??0.143719339 ?0.478791323 ?0.518731127 ?0.229860133 -0.199433324 ?0.310576455?? ? ? ? ? 25 ? ? ? ? ? 26 ? ? ? ? ? 27 ? ? ? ? ? 28 ? ? ? ? ? 29 ? ? ? ? ? 30??0.127612633 -0.157347145 ?0.413807523 ?0.007961485 -0.288867750 ?0.208759771?? ? ? ? ? 31 ? ? ? ? ? 32 ? ? ? ? ? 33 ? ? ? ? ? 34 ? ? ? ? ? 35 ? ? ? ? ? 36??0.286165027 ?0.299492579 ?0.197312294 ?0.135601904 ?0.452828662 ?0.187191405?? ? ? ? ? 37 ? ? ? ? ? 38 ? ? ? ? ? 39 ? ? ? ? ? 40 ? ? ? ? ? 41 ? ? ? ? ? 42??0.335596502 -0.109960231 -0.303770506 -0.276385255 ?0.429700474 ?0.003930969?? ? ? ? ? 43 ? ? ? ? ? 44 ? ? ? ? ? 45 ? ? ? ? ? 46 ? ? ? ? ? 47 ? ? ? ? ? 48??0.184186301 ?0.140858190 ?0.479882236 ?0.182523553 -0.133845870 ?0.443940376?? ? ? ? ? 49 ? ? ? ? ? 50 ? ? ? ? ? 51 ? ? ? ? ? 52 ? ? ? ? ? 53 ? ? ? ? ? 54??0.070571673 -0.383780163 ?0.362153269 ?0.202527841 ?0.164299813 ?0.327998904?? ? ? ? ? 55 ? ? ? ? ? 56 ? ? ? ? ? 57 ? ? ? ? ? 58 ? ? ? ? ? 59 ? ? ? ? ? 60??0.047612361 -0.032167295 ?0.060976285 ?0.231929803 -0.449532973 ?0.109925656?? ? ? ? ? 61 ? ? ? ? ? 62 ? ? ? ? ? 63 ? ? ? ? ? 64 ? ? ? ? ? 65 ? ? ? ? ? 66??0.468842330 ?0.108507841 ?0.158697337 -0.125813680 ?0.501159861 ?0.101646132?? ? ? ? ? 67 ? ? ? ? ? 68 ? ? ? ? ? 69 ? ? ? ? ? 70 ? ? ? ? ? 71 ? ? ? ? ? 72??0.194383106 -0.006185569 ?0.354467348 ?0.340013811 ?0.088757961 ?0.439984356?? ? ? ? ? 73 ? ? ? ? ? 74 ? ? ? ? ? 75 ? ? ? ? ? 76 ? ? ? ? ? 77 ? ? ? ? ? 78??0.330976669 ?0.449337326 ?0.081841142 -0.190123754 ?0.337794560 -0.111895039?? ? ? ? ? 79 ? ? ? ? ? 80 ? ? ? ? ? 81 ? ? ? ? ? 82 ? ? ? ? ? 83 ? ? ? ? ? 84??0.598231564 ?0.444399789 ?0.388313945 ?0.244270482 ?0.200026237 ?0.009025077?? ? ? ? ? 85 ? ? ? ? ? 86 ? ? ? ? ? 87 ? ? ? ? ? 88 ? ? ? ? ? 89 ? ? ? ? ? 90??0.341093767 -0.164196034 ?0.825849472 ?0.325975911 ?0.494473323 ?0.270037159?? ? ? ? ? 91 ? ? ? ? ? 92 ? ? ? ? ? 93 ? ? ? ? ? 94 ? ? ? ? ? 95 ? ? ? ? ? 96??0.369787280 ?0.247455471 ?0.282701738 -0.541688411 -0.145796547 ?0.073172268?? ? ? ? ? 97 ? ? ? ? ? 98 ? ? ? ? ? 99 ? ? ? ? ?100??0.685833173 -0.079174316 -0.193161949 -0.137517175? ?Arnab Kumar Maity Graduate Teaching Assistant Division of Statistics Northern Illinois University DeKalb, Illinois 60115 U.S.A From: Martin Spindler <Martin.Spindler at gmx.de> To: r-help at r-project.org Sent: Wednesday, April 29, 2015 9:21 AM Subject: [R] Problem with predict.lm() Dear all, ? the following example somehow uses the "old data" (X) to make the predictions, but not the new data Xnew as intended. ? y <- rnorm(100) X <- matrix(rnorm(100*10), ncol=10) lm <- lm(y~X) Xnew <- matrix(rnorm(100*20), ncol=10) ynew <- predict(lm, newdata=as.data.frame(Xnew)) #prediction in not made for Xnew ? How can I foce predict.lm to use use the new data? ? Thank you very much for your efforts in advance! ? Best, ? Martin ______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]]
On Apr 29, 2015, at 7:21 AM, Martin Spindler wrote:> Dear all, > > the following example somehow uses the "old data" (X) to make the predictions, but not the new data Xnew as intended. > > y <- rnorm(100) > X <- matrix(rnorm(100*10), ncol=10) > lm <- lm(y~X) > Xnew <- matrix(rnorm(100*20), ncol=10) > ynew <- predict(lm, newdata=as.data.frame(Xnew)) #prediction in not made for Xnew > > How can I foce predict.lm to use use the new data?If you look at what you are passing to `predict`, it should be apparent why it does not find a proper 'newdata' argument: str( as.data.frame(Xnew) ) 'data.frame': 200 obs. of 10 variables: $ V1 : num 1.2 -0.319 -0.175 -1.009 0.197 ... $ V2 : num 1.529 -0.107 -1.013 -0.869 1.166 ... $ V3 : num -0.417 -0.34 -0.101 -0.018 -2.237 ... $ V4 : num -2.2274 -1.15 0.0252 1.014 1.9455 ... $ V5 : num -0.207 1.628 -0.24 -0.194 -0.722 ... $ V6 : num -1.176 0.935 -0.862 -1.152 0.815 ... $ V7 : num 0.967 -1.464 -1.554 0.065 0.205 ... $ V8 : num -0.282 1.699 -0.267 -0.8 -0.643 ... $ V9 : num -0.34833 -0.24907 -0.84185 -0.0518 -0.00216 ... $ V10: num -0.37 -0.227 -2.949 0.899 -0.586 ... You need the newdata argument to be named exactly as lm would have coerced into being when given a single X predictor that was a matrix. Try instead: ynew <- predict(lm, newdata=list(X=Xnew) ) -- David Winsemius Alameda, CA, USA
Thank you! I think I now understand where the problem was. Best, Martin ? ? Gesendet:?Mittwoch, 29. April 2015 um 16:50 Uhr Von:?"David L Carlson" <dcarlson at tamu.edu> An:?"Martin Spindler" <Martin.Spindler at gmx.de>, "r-help at r-project.org" <r-help at r-project.org> Betreff:?RE: [R] Problem with predict.lm() Since you passed a matrix to lm() and then a data.frame to predict(), predict can't match up what variables to use for the prediction so it falls back on the original data. This seems to work:> set.seed(42) > y <- rnorm(100) > X <- matrix(rnorm(100*10), ncol=10) > Xd <- data.frame(X) > lm <- lm(y~., Xd) > Xnew <- matrix(rnorm(100*20), ncol=10) > Xnewd <- data.frame(Xnew) > ynew <- predict(lm, newdata=Xnewd) > head(ynew)1 2 3 4 5 6 0.35404067 0.14073495 -0.45442499 0.31065562 -0.02091366 0.25358175> head(predict(lm))1 2 3 4 5 6 0.75474817 0.06024122 -0.27221466 -0.20344713 0.20218135 -0.24045859>------------------------------------- David L Carlson Department of Anthropology Texas A&M University College Station, TX 77840-4352 -----Original Message----- From: R-help [mailto:r-help-bounces at r-project.org] On Behalf Of Martin Spindler Sent: Wednesday, April 29, 2015 9:21 AM To: r-help at r-project.org Subject: [R] Problem with predict.lm() Dear all, ? the following example somehow uses the "old data" (X) to make the predictions, but not the new data Xnew as intended. ? y <- rnorm(100) X <- matrix(rnorm(100*10), ncol=10) lm <- lm(y~X) Xnew <- matrix(rnorm(100*20), ncol=10) ynew <- predict(lm, newdata=as.data.frame(Xnew)) #prediction in not made for Xnew ? How can I foce predict.lm to use use the new data? ? Thank you very much for your efforts in advance! ? Best, ? Martin ______________________________________________ R-help at r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html[http://www.R-project.org/posting-guide.html] and provide commented, minimal, self-contained, reproducible code.