R help - Nov 2003 - svm in e1071 package: polynomial vs linear kernel

I am trying to understand what is the difference between linear and 
polynomial kernel:

          linear: u'*v

          polynomial: (gamma*u'*v + coef0)^degree

It would seem that polynomial kernel with gamma = 1; coef0 = 0 and degree 
= 1
should be identical to linear kernel, however it gives me significantly 
different results  for very simple
data set, with linear kernel significantly outperforming polynomial 
kernel.

*** mse, r2 = 0.5, 0.9 for linear
*** mse, r2 = 1.8, 0.1 for polynomial

What am I missing ?

Ryszard

P.S.

Here are my results:

# simple cross validation function
cv.svm <- function(formula, data, ntry = 3, kernel = "linear",
scale =
FALSE, cross = 3,
                   gamma = 1/(dim(data)-1), degree = 3) {
   mse <- 0; r2 <- 0
   for (n in 1:ntry) {
      svm.model <- svm(formula , data = data, scale = scale, kernel = 
kernel,
                       cross = cross)
      mse <- mse + svm.model$tot.MSE
      r2  <- r2 + svm.model$scorrcoeff
   }
   mse <- mse/ntry; r2 <- r2/ntry; result <- c(mse, r2)
   cat(sprintf("cv.svm> mse, r2 = %5.3f %5.3f\n", mse, r2))
   return (result)
}

# define data set

x1 <- rnorm(9); x2 <- rnorm(9)
df <- data.frame(y = 2*x1 + x2, x1, x2)

#  invoke cv.svm() for linear and polynomial kernels few times
> r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1,
degree = 1,
ntry = 32)
cv.svm> mse, r2 = 1.888 0.162
> r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1,
degree = 1,
ntry = 32)
cv.svm> mse, r2 = 1.867 0.146
> r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1,
degree = 1,
ntry = 32)
cv.svm> mse, r2 = 1.818 0.105
> r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree =
1, ntry =
32)
cv.svm> mse, r2 = 0.525 0.912
> r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree =
1, ntry =
32)
cv.svm> mse, r2 = 0.537 0.878
> r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree =
1, ntry =
32)
cv.svm> mse, r2 = 0.528 0.913


	[[alternative HTML version deleted]]

On Mon, 3 Nov 2003 ryszard.czerminski at pharma.novartis.com wrote:
> I am trying to understand what is the difference between linear and 
> polynomial kernel:
> 
>           linear: u'*v
> 
>           polynomial: (gamma*u'*v + coef0)^degree
> 
> It would seem that polynomial kernel with gamma = 1; coef0 = 0 and degree 
> = 1
> should be identical to linear kernel, however it gives me significantly 
> different results  for very simple
> data set, with linear kernel significantly outperforming polynomial 
> kernel.
> 
> *** mse, r2 = 0.5, 0.9 for linear
> *** mse, r2 = 1.8, 0.1 for polynomial
> 
> What am I missing ?
Well: perhaps, that you should pass *all* parameters from your cv.svm
function to the call of svm()?

g.,
-d
> 
> Ryszard
> 
> P.S.
> 
> Here are my results:
> 
> # simple cross validation function
> cv.svm <- function(formula, data, ntry = 3, kernel = "linear",
scale =
> FALSE, cross = 3,
>                    gamma = 1/(dim(data)-1), degree = 3) {
>    mse <- 0; r2 <- 0
>    for (n in 1:ntry) {
>       svm.model <- svm(formula , data = data, scale = scale, kernel = 
> kernel,
>                        cross = cross)
>       mse <- mse + svm.model$tot.MSE
>       r2  <- r2 + svm.model$scorrcoeff
>    }
>    mse <- mse/ntry; r2 <- r2/ntry; result <- c(mse, r2)
>    cat(sprintf("cv.svm> mse, r2 = %5.3f %5.3f\n", mse, r2))
>    return (result)
> }
> 
> # define data set
> 
> x1 <- rnorm(9); x2 <- rnorm(9)
> df <- data.frame(y = 2*x1 + x2, x1, x2)
> 
> #  invoke cv.svm() for linear and polynomial kernels few times
> 
> > r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1,
degree = 1,
> ntry = 32)
> cv.svm> mse, r2 = 1.888 0.162
> > r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1,
degree = 1,
> ntry = 32)
> cv.svm> mse, r2 = 1.867 0.146
> > r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1,
degree = 1,
> ntry = 32)
> cv.svm> mse, r2 = 1.818 0.105
> > r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1,
degree = 1, ntry =
> 32)
> cv.svm> mse, r2 = 0.525 0.912
> > r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1,
degree = 1, ntry =
> 32)
> cv.svm> mse, r2 = 0.537 0.878
> > r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1,
degree = 1, ntry =
> 32)
> cv.svm> mse, r2 = 0.528 0.913
> 
> 
> 	[[alternative HTML version deleted]]
> 
> ______________________________________________
> R-help at stat.math.ethz.ch mailing list
> https://www.stat.math.ethz.ch/mailman/listinfo/r-help
>