Page 76 - Data Science Algorithms in a Week

60 Olmer Garcia and Cesar Diaz

et al. (2012) states that dropout “consists of setting to zero the output of each hidden
neuron with probability 0.5. The neurons which are “dropped out” in this way do not
contribute to the forward pass and do not participate in back- propagation. So every time
an input is presented, the neural network samples a different architecture, but all these
architectures share weights. This technique reduces complex co-adaptations of neurons
since a neuron cannot rely on the presence of particular other neurons. It is, therefore,
forced to learn more robust features that are useful in conjunction with many different
random subsets of the other neurons. At test time, we use all the neurons but multiply
their outputs by 0.5, which is a reasonable approximation to taking the geometric mean of
the predictive distributions produced by the exponentially-many dropout networks.”

Transfer Learning

Transfer learning is the process of taking a pre-trained model (the weights and
parameters of a network that has been trained on a large quantity of data by others) and
“fine-tuning” the model with your own dataset (Yosinski, Clune, Bengio, & Lipson,
2014). The idea is that this pre-trained model will act as a feature extractor. You will
remove the last layer of the network and replace it with your own classifier or regression.
The Algorithm blocks the change of the weights of all the other layers and trains the
network normally. Transfer learning led to the introduction of the deep learning principle.
Reusing architecture and learning work is possible in CNNs. Therefore, one must review
the most successful architectures used before such as ImageNet by Krizhevsky et al.
(2012), ZF Net by Zeiler and Fergus (2014), VGG Net by Simonyan and Zisserman
(2014), GoogLeNet by Szegedy et al. (2015), and Microsoft ResNet (residual network)
by He et al. (2016).

ARCHITECTURE OF AUTONOMOUS VEHICLES

A typical architecture of mobile robots as described by Siegwart et al. (2011) is an
intelligent and autonomous system consisting of three main layers: perception, planning
and motion control. Each layer seeks to answer specific questions related to the
respective tasks as performed by the autonomous system (Figure 3).
The perception layer consists of the process to keep an internal description of the
external environment. The external environment is that part of the universe, which is
accessible to the proprioceptive sensors of an agent. In theory, it is also possible to use
the environment itself as the internal model. However, this requires a complete and
timeless sensing ability. It is easier to build a local description from a set sources and to
exploit the relative continuity of the universe to combine/fuse individual observations.