Page 174 - Data Science Algorithms in a Week

158 Djordje Cica and Davorin Kramar

Although many quite efficient bio-inspired algorithms have been developed for the
optimization of ANN, in this study two of them, namely, genetic algorithm (GA) and
particle swarm optimization (PSO), were utilized to train a feed forward ANN with a
fixed architecture. Therefore, numerical weights of neuron connections and biases
represent the solution components of the optimization problem.

GA-Based Artificial Neural Networks

Genetic algorithms belong to the larger class of evolutionary algorithms (EA) in
which a population of candidate solutions to a problem evolves over a sequence of
generations. GA has been successfully used in a wide variety of problem domains that are
not suitable for standard optimization algorithms, including problems in which the
objective function is highly nonlinear, stochastic, nondifferentiable or discontinuous. An
implementation of a GA begins with a randomly generated population of individuals, in
which each individual is represented by a binary string (called chromosomes) for one
possible solution. These strings encode candidate solutions (called individuals) to an
optimization problem, evolves toward better solutions. The evolution happens in
generations and during each generation a measure of the fitness with respect to an
objective function is evaluated. Based on fitness value, a new population is then created
based on the evaluation of the previous one, which becomes current in the next iteration
of the algorithm. Individuals with a higher fitness have a higher probability of being
selected for further reproduction. Thus, on average, the new generation will possess a
higher fitness value than the older population. Commonly, the algorithm continues until
one or more of the pre-established criteria, such as maximum number of generations or a
satisfactory fitness level, has been reached for the population.
Following are the steps involved in the working principle of GA: (i) chromosome
representation, (ii) creation of the initial population, (iii) selection, (iv) reproduction, (v)
termination criteria and (vi) the evaluation function.
Chromosome representation. The basic element of the genetic algorithm is the
chromosome which contain the variable information for each individual solution to the
problem. The most common coding method is to represent each variable with a binary
string of digits with a specific length. Each chromosome has one binary string and each
bit in this string can represent some characteristic of the solution. Another possibility is
that the whole string can represent a number. Therefore, every bit string is a solution, but
not necessarily the best solution. This representation method is very simple; strings of
ones and zeroes would be randomly generated, e.g., 1101001, 0101100, etc., and these
would form the initial population. The strings may be of fixed length or, more rarely, be
of variable length. Apart from binary encoding, octal encoding, hexadecimal encoding,