hi all
i'm new to the area of neural networks. i've been reading some
references and seem to understand some of the learning algorithms. i am
very familiar with regression and would just like to see how neural nets
handle this problem so i've been using the nnet package.
i simply want to use a 3 layer neural net, ie 1 input, 1 hidden layer
(where the hidden layer is linear, since i want to basically perform
regression analysis by means of neural nets) and 1 output layer.
the x and y vector was simulated as follows:
x<-1:100
x
[1] 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
17 18
[19] 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
35 36
[37] 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52
53 54
[55] 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70
71 72
[73] 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88
89 90
[91] 91 92 93 94 95 96 97 98 99 100
y<-2+5*x+rnorm(100)*5
y
[1] 8.789605 11.151109 14.622276 30.381379 19.328647 29.317038
33.793720 39.557390 51.939294 45.045965
[11] 58.783991 63.191745 72.882202 79.184778 85.034551 94.446000
89.243004 88.223547 106.327683 104.424668
[21] 103.057648 112.855778 111.777823 108.359485 128.956152 127.369102
128.784481 139.760279 151.959887 152.014623
[31] 158.869586 167.030970 166.711160 177.415680 173.542293 182.484224
179.767128 192.284343 196.173830 202.353030
[41] 220.449623 213.410307 216.746041 219.812526 230.440402 230.759429
239.311279 244.151390 248.637023 254.648298
[51] 262.694237 253.619539 276.975714 280.395284 280.173787 286.813617
284.766870 296.705692 295.110064 304.709464
[61] 305.650793 310.128992 314.035624 314.649213 322.958865 333.640203
342.538307 340.546359 342.433629 344.720633
[71] 354.115051 363.631246 371.479886 367.066764 377.184512 386.634677
392.310577 386.151325 400.345393 408.831710
[81] 413.999148 405.009358 418.679828 418.388427 419.282955 432.329471
433.448313 444.166060 447.773185 455.103503
[91] 448.588598 464.410358 465.565875 478.677403 478.306390 479.565728
487.681689 491.422090 502.468491 500.385458
i then went about to use the nnet function:
> a.net<-nnet(y~x,linout=T,size=1)
# weights: 4
initial value 8587786.130082
iter 10 value 2086583.979646
final value 2079743.529111
converged
NOTE: the function said that four weights were estimated. This is
correct as shown below. the model can be represented as:
input ---(w1)---> hidden ---(w2)---> output
x ----------> a1+w1*x ----------> a2+w2*(a1+w1*x)
where:
wi are the weights, i=1,2
x is an input pattern
further results were:
summary(a.net)
a 1-1-1 network with 4 weights
options were - linear output units
b->h1 i1->h1
-276.48 -295.11
b->o h1->o
254.92 764.72
is the following statement correct? (i think that it is!)
a1= "b->h1"
w1= "i1->h1"
a2= "b->o"
w2= "h1->o"
If the hidden layer and the output layers are both LINEAR then the
following should be true:
1. 2= a2+a1*w2
2. 5= w1*w2
THIS IS NOT THE CASE, see the results. The only thing that i can think
of thats happening is that the activation function from the hidden layer
is not linear. Is this correct since i used the linout=T arguement? are
we able to change the activation function from the hidden layer?
two other questions:
1. with regard to the size arguement, how does one know how many nodes
are in the hidden layer? (this might be a silly question.) e.g we might
have 2 hidden layers both with 3 nodes.
2. are we able to plot the drop in the error function as a function of
the epochs?
hope someone can help
thanking you!!!