R help - Jun 2012 - Softmax Action Selection

R Community -

I'm attempting to apply a softmax action selection to a probability
generated by a hidden Markov model.  I'm having difficulties in how to
apply the softmax temperature parameter (beta).  Here is my code thus far.
 I'm thinking the sigmoid function will work but I need this function to
return the total error instead of the probability so I can't optimize it
with optim().

      calc.probs <- function(delta,beta)
          {
            # initial starting probabilities
            thList = 0; block = 0
            for (i in 1:length(dat$choice))
              {
              if (dat$RepNum[i] != block)
                {
                  pL = 0.5; pR = 0.5; block = dat$RepNum[i];
                }
            # Markov Transitions
              pL <- pL*(1-delta) + pR*delta
              pR <- 1-pL
            # Apply feedback
              pflc <- ifelse(dat$choice[i] == dat$reward[i], .8, .2)
              pfrc <- 1 - pflc
              denom <- pflc * pL + pfrc * pR
            # What's the new belief given observation
              posteriorL <- pflc * pL/denom
              posteriorR <- 1-posteriorL
              pL <- posteriorL; pR <- posteriorR
            # Insert softmax here
              pLlist[i] <- pL

              }

          return(pLlist)
          }


-- 
Edward H. Patzelt
Research Assistant – TRiCAM Lab
University of Minnesota – Psychology/Psychiatry
VA Medical Center
S355 Elliot Hall: 612-626-0072
www.psych.umn.edu/research/tricam

	[[alternative HTML version deleted]]