# Material

The practical session is done using R. For a quick tutorial, follow this link.

# Practical Session

## Necessary support:

• The lecture slides
• The practical session R and the datasets file

### Part A: Linear dataset.

A.1/A.2) Visualization of the first dataset:

``````load('linear1.RData')
``````

3 new variables are in the environment now (type `ls()` to check):

• linear1.train : a matrix of size n1x3 (first column is x1, second column is x2 and third column is the output y).
• linear1.test.input : a matrix of size n2 x 2 (same as the train but without the output y).
• linear1.data : the combined dataset (for visualization).

Then visualize the dataset:

``````require( 'ggplot2' )
qplot( data=linear1.data, x.1, x.2, colour=factor(y),
shape=factor(train) )
``````

Question 1: How can you characterise this dataset ?

Given what we learned during the lecture, we train a linear SVM on the training set which will be our predictor for the testing set.

A.3/A.4/A.5) Training the SVM:

``````### A.3) Train a linear SVM
require( 'kernlab' )
linear1.svm <- ksvm( y ~ ., data=linear1.train, type='C-svc',
C=100, scale=c() )

### A.4) Plot the model
plot( linear1.svm, data=linear1.train )

### A.5) Adding points of test on the graph
points( linear1.test.input[ sample.int(nrow(linear1.test.input),10), ], pch=4 )
``````

Question 2: What are the black points in the figure ?

A.6/A.7 Testing on another dataset:

``````### A.6) Prediction
linear1.prediction <- predict( linear1.svm, linear1.test.input )

### A.7) Look at accuracy
# contains linear1.test.output

print(paste0('Accuracy: ', floor(100*sum( linear1.prediction == linear1.test.output )/length(linear1.test.output)), '%'))
``````

Let's consider a dataset a bit more complex:

A.9 to A.14) This is the exact same thing as the first part except that the dataset is non-separable.

Question 3: Is the accuracy a sufficient method to assess the performance of a model?

Let's discuss a few ways to improve the assessment of the performance:

A.15) Separate positive and negative examples :

``````### A.15) A confusion matrix gives more information than just accuracy
print('Confusion Matrix: ');print(table( linear2.prediction, linear2.test.output, dnn= c("prediction","reality") ))
``````

A.16) ROC Curves (See wikipedia):

``````linear2.prediction.score <- predict( linear2.svm, linear2.test.input, type='decision' )

require( 'ROCR' )

## ROC
linear2.roc.curve <- performance( prediction( linear2.prediction.score, linear2.test.output ), measure='tpr', x.measure='fpr' )
plot( linear2.roc.curve )
``````

### Part B: Non-Linear dataset.

B.1/B.2/B.3) Trying linear SVM on a dataset where it is not appropriate.

Let's try a different kernel

B.4) RBF Kernel:

``````nonlinear.svm <- ksvm( y ~ ., data=nonlinear.train, type='C-svc',
kernel='rbf', kpar=list(sigma=1),
C=100, scale=c() )
plot( nonlinear.svm, data=nonlinear.train )
``````

Question 4: Recall what is the parameter `C` in the svm. In the following we will see what happens when we vary `C`.

B.5) Impact of `C`

``````require('manipulate')
manipulate( plot( ksvm( y ~ ., data=nonlinear.train, type='C-svc',
kernel=k, C=2^c.exponent, scale=c() ),
data=nonlinear.train ),
c.exponent=slider(-10,10),
'Hyperbolic'='tanhdot','Spline'='splinedot',
'Laplacian'='laplacedot') )
``````

B.6) Visualization of the impact of `C` on the prediction accuracy (Bias-Variance Tradeoff):

``````### B.6) Bias-Variance Tradeoff
BiasVarianceTradeoff <- function( dataset, cross=10, c.seq=2^seq(-10, 10), ... ) {
err <- sapply( c.seq, function( c )
{
cross( ksvm( y ~ ., data=dataset, C=c, cross=cross, ...) )
})
return(data.frame( c=c.seq, error=err ))
}

qplot( c, error, data=BiasVarianceTradeoff( nonlinear.train, type='C-svc', kernel='rbfdot' ), geom='line', log='x' )
``````

Question 5: How to choose `C` ?

### Part C: Acute lymphoblastic leukemia dataset.

In this part, we work on a real public dataset of Acute lymphoblastic leukemia (ALL) patients.

C.1) We are interested in classifying leukemia patients into two classes: B-cell ALL vs T-cell ALL because this classification can have an impact on the patient's prognosis or its response to a given treatment.

C.2) When we add supplementary clinical information, we have more than 2 classes (B1, B2,...).

Question 6: How can you derive from what you learned a SVM classifier that can predict more than 2 classes?