Page 85 - Data Science Algorithms in a Week

Machine Learning Applied to Autonomous Vehicles 69

̂
θt ← θt−1 − α · /( √ ̂ + ∈) (Update parameters)

end while
return θt (Resulting parameters for the Deep Neural Network)

 Batch size: This hyper-parameter defines the number of examples that are going
to be propagated in a forward/backward iteration. A fine tuned batch size can
support schemes of less memory and faster training. However, it can reduce the
accuracy of the gradient estimation.
 Epochs: One epoch is a forward pass and one backward pass of all the training
examples of the training data set. The analyst monitors each epoch and analyzes
how the training process is evolving. Note that in each epoch the training and
validation data should be shuffled to improve the generalization of the neural
network.
 Hyperparameters: Depending on the algorithm and the framework used there
exist values that should be tuned. Learning rates of the optimizer are usually an
important hyper-parameters to find. CNNs may involve other hyperparameters
such as filter windows, dropout rates, and the size of the mini-batches. These
hyper-parameters can be different for each layer. For example, the following
hyper-parameters can be relevant for a CNN: Number of Filters (K), F: filter size
(FxF), Stride (S), and the amount of padding (P). Techniques can be used in
order to optimize the tuning process and avoid trial and error efforts. These
techniques can involve models from Operations Research, evolutionary
algorithms, Bayesian schemes, and heuristic searches.

The training process will end when one finds a good accuracy between the model
outputs and the known output classes. In this project, an accuracy over 98% for a CNN
developed for Colombian traffic signals was achieved. This is a very good value taking
into consideration that humans are in the 98.32% accuracy range. The training process
requires sophisticated computational power. It is essential to have access to high-level
computing resources or cloud services providers like Amazon (https://aws.amazon.com/),
IBM Bluemix (https://www.ibm.com/cloud-computing/bluemix/) or Microsoft Azure
(https://azure.microsoft.com/).

Testing the Model

The last step is to prove that the neural network model works in other situations
which are different than the data that was used to train and validate the model. It is very
important to use data that has not been used in the process of training and validation. For
example, we developed a CNN with Colombian Traffic signals and obtained a moderate