R help - May 2005 - SVM linear kernel and SV

Dear all,

I'm a trainee statistician in a company and we'd like to understand svm 
mechanism, at first with simple examples.

I use e1071 package and I have several questions. I'm working with data 
extracted from cats data (from R). My dataset corresponds to a completely 
separable case with a binary response variable ( Sex with 2 levels: F and 
M), two  explanatory variables (Bwt and Hwt) and the classes are balanced.

 I've used  svm() with a linear kernel and I'd like to plot the linear 
hyperplane and the support vectors. I use plot.svm() and, according to me, 
I would have found aligned support vectors (because the hyperplane is 
linear) for each class but it wasn't the case. Could you explain me why ?

In addition, when I change the option 'scale' (from TRUE to FALSE) the 
results change. Could you explain me why ? the option 'scale' of svm() 
acts on the dataset or on the weight vector w and threshold b  ?

Thank you very much,
Gladys.

My dataset 'catseq' is following:

n  Sex  Bwt Hwt
1     F 2.0  7.0
2     F 2.0  7.4
3     F 2.0  9.5
4     F 2.1  7.2
5     F 2.1  7.3
6     F 2.1  7.6
7     F 2.1  8.1
8     F 2.1  8.2
9     F 2.1  8.3
10    F 2.1  8.5
11    F 2.1  8.7
12    F 2.1  9.8
13    F 2.2  7.1
14    F 2.2  8.7
15    F 2.2  9.1
16    F 2.2  9.7
17    F 2.2 10.9
18    F 2.2 11.0
19    F 2.3  7.3
20    F 2.3  7.9
21    F 2.3  8.4
22    F 2.3  9.0
23    F 2.3  9.0
24    F 2.3  9.5
25    F 2.3  9.6
26    F 2.3  9.7
27    F 2.3 10.1
28    F 2.3 10.1
29    F 2.3 10.6
30    F 2.3 11.2
31    F 2.4  6.3
32    F 2.4  8.7
33    F 2.4  8.8
34    F 2.4 10.2
35    F 2.5  9.0
36    F 2.5 10.9
37    F 2.6  8.7
38    F 2.6 10.1
39    F 2.6 10.1
40    F 2.7  8.5
42    F 2.7 10.8
43    F 2.9  9.9
91    M 2.8 11.4
92    M 2.8 12.0
93    M 2.8 13.3
94    M 2.8 13.5
98    M 2.9 11.3
99    M 2.9 11.8
103   M 3.0 11.6
104   M 3.0 12.2
105   M 3.0 12.4
106   M 3.0 12.7
107   M 3.0 13.3
108   M 3.0 13.8
110   M 3.1 11.5
111   M 3.1 12.1
112   M 3.1 12.5
113   M 3.1 13.0
114   M 3.1 14.3
115   M 3.2 11.6
116   M 3.2 11.9
117   M 3.2 12.3
118   M 3.2 13.0
119   M 3.2 13.5
120   M 3.2 13.6
121   M 3.3 11.5
122   M 3.3 12.0
123   M 3.3 14.1
124   M 3.3 14.9
125   M 3.3 15.4
126   M 3.4 11.2
127   M 3.4 12.2
128   M 3.4 12.4
129   M 3.4 12.8
130   M 3.4 14.4
131   M 3.5 11.7
132   M 3.5 12.9
133   M 3.5 15.6
134   M 3.5 15.7
135   M 3.5 17.2
136   M 3.6 11.8
137   M 3.6 13.3
138   M 3.6 14.8
139   M 3.6 15.0


My program is following:

library(e1071)
library(MASS)

catseq <- read.table('P:/catsredeq.txt',header=T,sep="")
plot(catseq$Hwt,catseq$Bwt,pch=as.integer(catseq[,2]),col=as.integer(catseq[,2]),xlab="poids
du coeur",ylab="poids du corps")
title(main="Données 'chats' (cas séparable)")

catseq <- catseq[,2:4]
attach(catseq)


svm12 <- svm(Sex~.,data=catseq,kernel="linear",scale=T)
svm12
x11()
plot(svm12,catseq,grid=200)

x11()
svm22 <- svm(Sex~.,data=catseq,kernel="linear",scale=F)
svm22
plot(svm22,catseq,grid=200)
	[[alternative HTML version deleted]]

Gladys,
> 
>  I've used  svm() with a linear kernel and I'd like to plot the
linear
>  
> hyperplane and the support vectors. I use plot.svm() and, according to
> me, I would have found aligned support vectors (because the hyperplane
> is linear) for each class but it wasn't the case. Could you explain me
> why ?
In how far does the plot give you the impression is wouldn't? The two
classes look pretty separated to me.
> 
> In addition, when I change the option 'scale' (from TRUE to FALSE)
the
> 
> results change. 
(Which results?) The plot is, of course, slightly different since the
model is based on different data, but the class predictions (on the
training data) are the same. Why does this surprise you?

Could you explain me why ? the option 'scale' of svm()
> acts on the dataset or on the weight vector w and threshold b  ?
On the data set, and therefore also on w and b.

Best,
David


-- 
Dr. David Meyer
Department of Information Systems and Process Management

Vienna University of Economics and Business Administration
Augasse 2-6, A-1090 Wien, Austria, Europe
Fax: +43-1-313 36x746 
Tel: +43-1-313 36x4393
HP:  http://wi.wu-wien.ac.at/~meyer/

> Thank you for your answer,
> but my problem concerns the support vectors. Indeed the two classes
> are well separated and the hyperplane is linear but the support
> vectors aren't aligned in parallel to the hyperplane. And according to
> me,  the support vectors (for each class) should be aligned along the
> linear hyperplane and form the marge (by definition). But it's not the
> case. In fact,  I'd like to understand why they are not aligned. 
Remember the `cost'-penalty controlling for overlapping classes. It has
some effect even in the linearly separable case causing more SVs than
would actually be needed. Try adding e.g. `cost=1000' and you will
obtain a result with only 2 SVs (why not 3? Because 2 SVs here solve the
optimization problem. So in fact the hyperplane in this case is not
uniquely defined, although only in a small range.)

Best,

David