Page 178 - Data Science Algorithms in a Week

162 Djordje Cica and Davorin Kramar

PSO-Based Artificial Neural Networks

PSO algorithm is a relatively new optimization technique originally introduced by
Eberhart and Kennedy (1995). It was inspired by the social interaction and
communication of bird flocking or fish schooling. PSO algorithm is population-based
stochastic optimization method, where the population is referred to as a swarm. The
optimization process of a PSO algorithm begins with an initial population of random
candidate solutions called particles. These particles change their positions by moving
around in the multidimensional search space through many iterations to search an optimal
solution for the problem by updating various properties of the individuals in each
generation. Each particle in the swarm is represented by the following characteristics: the
position vector of the particle, the velocity vector of the particle and the personal best
position of the particle. During the search process, the position of each particle is guided
by two factors: the best position visited by itself, and the global best position discovered
so far by any of the particles in the swarm. In this way, the trajectory of each particle is
influenced by the flight experience of the particle itself as well as the trajectory of
neighborhood particles of the whole swarm. This means that all the particles fly through
search space toward personal and global best position a navigated way, at the same time
exploring new areas by the stochastic mechanism in order to escape from local optima.
The performance of particles are evaluated using a fitness function that varies depending
on the optimization problem.
Position of the i-th particle in the d-dimension solution space at iteration k is denoted
as

k 
k
k
x k
x i    ,1 i  , x ,2 i  , ..., x , i d   (6)

while

ˆ k 
ˆ
ˆ
k
  ˆ , y

k
y i    y k ,2 i  , ..., y , i d   (7)
,1 i

denote the best position found by particle i up to iteration k, and

y    k  y ,1 i  , k y ,2 i  , ..., k y , i d   (8)
k

denote the best position found by any of the particles in the neighborhood of xi up to
iteration k.
The new position of particle i in iteration k + 1, xi(k + 1), is computed by adding a
velocity