jude.ryan at ubs.com
2009-May-14 21:37 UTC
[R] mathematical model/equations for neural network in library(nnet)
Hi All, I am trying to manually extract the scoring equations for a neural network so that I can score clients on a system that does not have R (mainframe using COBOL). Using the example in Modern Applied Statistics with S (MASS), by Venables and Ripley (VR), pages 246 and 247, I ran the following neural network. The code is the same as in VR pages 246 and 247 except I have skip = F. The equation will have 3 more terms if skip = T. library(nnet) attach(rock) area1 <- area/10000; peri1 <- peri/10000 rock1 <- data.frame(perm, area = area1, peri = peri1, shape) # skip = F rock2.nn <- nnet(log(perm) ~ area + peri + shape, rock1, size=3, decay=1e-3, linout=T, skip=F, maxit=1000, Hess=T) # weights: 16 initial value 1420.968942 iter 10 value 96.823665 iter 20 value 32.177295 iter 30 value 25.012430 iter 40 value 23.109650 iter 50 value 20.981236 iter 60 value 15.019016 iter 70 value 14.082190 iter 80 value 14.042717 iter 90 value 13.931124 iter 100 value 13.883691 iter 110 value 13.877307 iter 120 value 13.875051 iter 130 value 13.873667 final value 13.873634 converged summary(rock2.nn) The output from summary(rock2.nn) is: a 3-3-1 network with 16 weights options were - linear output units decay=0.001 b->h1 i1->h1 i2->h1 i3->h1 10.65 -8.90 -14.63 6.17 b->h2 i1->h2 i2->h2 i3->h2 -0.72 11.76 -17.17 -1.56 b->h3 i1->h3 i2->h3 i3->h3 2.96 -9.03 -8.07 -2.54 b->o h1->o h2->o h3->o -6.91 2.45 11.53 9.22 Following the mathematical model / equations shown in VR (pages 243 to 247) and another book on neural networks, I extracted the neural network equations manually, and scored the dataset rock1, and compared the manual scores I obtained with the scores from predict(). They were totally different, and I am not sure what I am doing wrong. If anyone can give me some pointers I would appreciate it. The mathematical model/equations I come up with from the weights are: # manual calculate neural network predictions based on neural network equations rock1$h1 <- 10.65 - 8.9 * rock1$area - 14.63 * rock1$peri + 6.17 * rock1$shape rock1$logistic_h1 <- exp(rock1$h1) / (1 + exp(rock1$h1)) rock1$h2 <- -0.72 + 11.76 * rock1$area - 11.17 * rock1$peri - 1.56 * rock1$shape rock1$logistic_h2 <- exp(rock1$h2) / (1 + exp(rock1$h2)) rock1$h3 <- 2.96 - 9.03 * rock1$area - 8.07 * rock1$peri - 2.54 * rock1$shape rock1$logistic_h3 <- exp(rock1$h3) / (1 + exp(rock1$h3)) # predictions based on manual scoring rock1$pred_perm <- -6.91 + 2.45 * rock1$logistic_h1 + 11.53 * rock1$logistic_h2 + 9.22 * rock1$logistic_h3 # predictions using predict() and object that has the output of the neural network rock1$nn_pred_perm <- predict(rock2.nn) rock1$log_perm <- log(rock1$perm) head(rock1) perm area peri shape h1 logistic_h1 h2 logistic_h2 h3 logistic_h3 pred_perm nn_pred_perm log_perm 1 6.3 0.4990 0.279190 0.0903296 2.6816839 0.9359372 1.888774 0.8686156 -4.028470 0.0174901847 5.559444 1.920348 1.840550 2 6.3 0.7002 0.389260 0.1486220 -0.3596561 0.4110428 2.934467 0.9495242 -6.881634 0.0010254128 5.054524 1.546815 1.840550 3 6.3 0.7558 0.393066 0.1833120 -0.6961405 0.3326685 3.491694 0.9704505 -7.502529 0.0005513831 5.099416 2.630932 1.840550 4 6.3 0.7352 0.386932 0.1170630 -0.8318165 0.3032611 3.421303 0.9683637 -7.098737 0.0008254655 5.005834 2.489565 1.840550 5 17.1 0.7943 0.394854 0.1224170 -1.4406711 0.1914414 4.019478 0.9823546 -7.709940 0.0004481475 4.889712 3.235397 2.839078 6 17.1 0.7979 0.401015 0.1670450 -1.2874918 0.2162777 3.923376 0.9806092 -7.905522 0.0003685659 4.929703 3.078584 2.839078 sum((log(perm) - rock1$nn_pred_perm)^2) [1] 12.55929 sum((log(perm) - rock1$pred_perm)^2) [1] 82.63254 Thanks in advance, Jude ___________________________________________ Jude Ryan Director, Client Analytical Services Strategy & Business Development UBS Financial Services Inc. 1200 Harbor Boulevard, 4th Floor Weehawken, NJ 07086-6791 Tel. 201-352-1935 Fax 201-272-2914 Email: jude.ryan at ubs.com -------------- next part -------------- Please do not transmit orders or instructions regarding a UBS account electronically, including but not limited to e-mail, fax, text or instant messaging. The information provided in this e-mail or any attachments is not an official transaction confirmation or account statement. For your protection, do not include account numbers, Social Security numbers, credit card numbers, passwords or other non-public information in your e-mail. Because the information contained in this message may be privileged, confidential, proprietary or otherwise protected from disclosure, please notify us immediately by replying to this message and deleting it from your computer if you have received this communication in error. Thank you. UBS Financial Services Inc. UBS International Inc. UBS Financial Services Incorporated of Puerto Rico UBS AG UBS reserves the right to retain all messages. Messages are protected and accessed only in legally justified cases.