Page 103 - Data Science Algorithms in a Week

Evolutionary Optimization of Support Vector Machines … 87

We consider two mutation operators: a simple mutation with fixed mutation
probability and a more complex mutation operator with dynamic rate of mutation that
depends on the generation according to the equation:

 n  2   1
p   2  t   .
 t T  1 

In addition, the simple mutation operator was also modified to experiment with other
techniques for varying the mutation rate: a self-adaptation method and a feedback
mechanism based on the genetic diversity.
The self-adaptation method consists on adding 16 bits to each individual in order to
obtain a probability p . From this value the mutation rate is obtained according to the
following equation (Bäck and Schütz, 1996):

 1 p   1

' p   1 e    N  (0,1) 
 p 

Where  is the rate that controls the adaptation speed and N (0,1) is a random normal

number with mean 0 and standard deviation 1. The normal random variable is generated
according to the Box and Muller method (see, for example, Law and Kelton 2000 p 465).
The feedback mechanism was based on calculating the genetic diversity of the
population by the ratio between the average and the best fitness (
AvgFitness / BestFitness ). If the genetic diversity falls below a particular level the

mutation rate is increased and the crossover rate is reduced. The contrary happens if the
genetic diversity becomes bigger than a given value.
For the selection operator we only considered the deterministic k Tournament
selection.

Comparison between Variations of GAs

To select the operators with the best performance (e.g., faster convergence of the
GA) from the different possibilities, we repeated the runs 30 times with different random
initial solutions. With each replication, we obtain an independent estimate for the best
generalization ability at each generation.
At the start of each replication, the dataset is randomly split in the ten subsets
required by the 10-fold crossvalidation. Using the same split during the whole run allows
us to study the effect of the different variations without being affected by randomness,