The package AMORE appears to be more flexible, but I got very poor
results using it when I tried to improve the predictive accuracy of a
regression model. I don't understand all the options well enough to be
able to fine tune it to get better predictions. However, using the
nnet() function in package VR gave me decent results and is pretty easy
to use (see the Venables and Ripley book, Modern Applied Statistics with
S, pages 243 to 249, for more details). I tried using package neuralnet
as well but the neural net failed to converge. I could not figure out
how to set the threshold option (or other options) to get the neural net
to converge. I explored package neural as well. Of all these 4 packages,
the nnet() function in package VR worked the best for me.
As another R user commented as well, you have too many hidden layers and
too many neurons. In general you do not need more than 1 hidden layer.
One hidden layer is sufficient for the "universal approximator"
property
of neural networks to hold true. As you keep adding neurons to the one
hidden layer, the problem becomes more and more non-linear. If you add
too many neurons you will overfit. In general, you do not need to add
more than 10 neurons. The activation function in the hidden layer of
Venables and Ripley's nnet() function is logistic, and you can specify
the activation function in the output layer to be linear using linout T in
nnet(). Using one hidden layer, and starting with one hidden neuron
and working up to 10 hidden neurons, I built several neural nets (4,000
records) and computed the training MSE. I also computed the validation
MSE on a holdout sample of over 1,000 records. I also started with 2
variables and worked up to 15 variables in a "for" loop, so in all, I
built 140 neural nets using 2 "for" loops, and stored the results in
lists. I arranged my variables in the data frame based on correlations
and partial correlations so that I could easily add variables in a
"for"
loop. This was my "crude" attempt to simulate variable selection
since,
from what I have seen, neural networks do not have variable selection
methods. In my particular case, neural networks gave me marginally
better results than regression. It all depends on the problem. If the
data has non-linear patterns, neural networks will be better than linear
regression.
My code is below. You can modify it to suit your needs if you find it
useful. There are probably lines in the code that are redundant which
can be deleted.
HTH.
Jude Ryan
My code:
# set order in data frame train2 based on correlations and partial
correlations
train2 <- train[, c(5,27,19,20,25,26,4,9,3,10,16,6,2,14,21,28)]
dim(train2)
names(train2)
library(nnet)
# skip = T
# train 10 neural networks in a loop and find the one with the minimum
test and validation error
# create various lists to store the results of the neural network
running in two for loops
# The Column List is for the outer for loop, which loops over variables
# The Row List is for the inner for loop, which loops over number of
neurons in the hidden layer
col_nn <- list() # stores the results of nnet() over variables - outer
loop
row_nn <- list() # stores the results of nnet() over neurons - inner
loop
col_mse <- list()
# row_mse <- list() # not needed because nn.mse is a data frame with
rows
col_sum <- list()
row_sum <- list()
col_vars <- list()
row_vars <- list()
col_wts <- list()
row_wts <- list()
df_dim <- dim(train2)
df_dim[2] # number of variables
df_dim[2] - 1
num_of_neurons <- 10
# build data frame to store results of neural net for each run
nn.mse <- data.frame(Train_MSE=seq(1:num_of_neurons),
Valid_MSE=seq(1:num_of_neurons))
# open log file and redirect output to log file
sink("D:\\XXX\\YYY\\ Programs\\Neural_Network_v8_VR_log.txt")
# outer loop - loop over variables
for (i in 3:df_dim[2]) { # df_dim[2]
# inner loop - loop over number of hidden neurons
for (j in 1:num_of_neurons) { # upto 10 neurons in the hidden layer
# need to create a new data frame with just the predictor/input
variables needed
train3 <- train2[,c(1:i)]
coreaff.nn <- nnet(dep_var ~ ., train3, size = j, decay = 1e-3,
linout = T, skip = T, maxit = 1000, Hess = T)
# row_vars[[j]] <- coreaff.nn$call # not what we want
# row_vars[[j]] <- names(train3)[c(2:i)] # not needed in inner loop
- same number of variables for all neurons
row_sum[[j]] <- summary(coreaff.nn)
row_wts[[j]] <- coreaff.nn$wts
rownames(nn.mse)[j] <- paste("H", j, sep="")
nn.mse[j, "Train_MSE"] <- mean((train3$dep_var -
predict(coreaff.nn))^2)
nn.mse[j, "Valid_MSE"] <- mean((valid$dep_var -
predict(coreaff.nn,
valid))^2)
}
col_vars[[i-2]] <- names(train3)[c(2:i)]
col_sum[[i-2]] <- row_sum
col_wts[[i-2]] <- row_wts
col_mse[[i-2]] <- nn.mse
}
# cbind(col_vars[1],col_vars[2])
col_vars
col_sum
col_wts
sink()
cbind(col_mse[[1]],col_mse[[2]],col_mse[[3]],col_mse[[4]],col_mse[[5]],c
ol_mse[[6]],col_mse[[7]],
col_mse[[8]],col_mse[[9]],col_mse[[10]],col_mse[[11]],col_mse[[12]],col_
mse[[13]],col_mse[[14]])
# build a neural network with the same 9 variables as in regression
pred <- list()
summaries <- list()
wts <- list()
Indrajit wrote:
You are right there is a pdf file which describes the function. But let
tell you where I am coming from.
Just to test if a neural network will work better than a ordinary least
square regression, I created a dataset with one dependent variable and 6
other independent variables. Now I had deliberately created the dataset
in such manner that we have an excellent regression model. Eg: Y = b0 +
b1*x1 + b2*x2 + b3*x3.. + b6*x6 + e
where e is normal random variable. Naturally any statistical analysis
system running regression would easily predict the values of b1, b2, b3,
..., b6 with around 30-40 observations.
I fed this data into a Neural network (3 hidden layers with 6 neurons in
each layer) and trained the network. When I passed the input dataset and
tried to get the predictions, all the predicted values were identical!
This confused me a bit and was wondering whether my understanding of the
Neural Network was wrong.
Have you ever faced anything like it?
Regards,
Indrajit
___________________________________________
Jude Ryan
Director, Client Analytical Services
Strategy & Business Development
UBS Financial Services Inc.
1200 Harbor Boulevard, 4th Floor
Weehawken, NJ 07086-6791
Tel. 201-352-1935
Fax 201-272-2914
Email: jude.ryan at ubs.com
-------------- next part --------------
Please do not transmit orders or instructions regarding a UBS
account electronically, including but not limited to e-mail,
fax, text or instant messaging. The information provided in
this e-mail or any attachments is not an official transaction
confirmation or account statement. For your protection, do not
include account numbers, Social Security numbers, credit card
numbers, passwords or other non-public information in your e-mail.
Because the information contained in this message may be privileged,
confidential, proprietary or otherwise protected from disclosure,
please notify us immediately by replying to this message and
deleting it from your computer if you have received this
communication in error. Thank you.
UBS Financial Services Inc.
UBS International Inc.
UBS Financial Services Incorporated of Puerto Rico
UBS AG
UBS reserves the right to retain all messages. Messages are protected
and accessed only in legally justified cases.