Page 20 - Monocle Quarterly Journal Vol 3 Issue 2 Spring

MONOCLE QUARTERLY JOURNAL | DEEP LEARNING
battled through the AI Winter without much support and often under much criticism.
Unfortunately, the inventor of the Perceptron, Frank Rosenblatt, would not live to see the revival of his field – having died in a boating accident not long after the release of Perceptrons – but the late Marvin Minsky would, living long enough at least to swallow his words and completely change his mind. Minsky, who for many years doubted the capability of neural networks, would even later go on to co-found the Massachusetts Institute of Technology’s (MIT) AI laboratory, becoming one of the foremost experts in the field and a great believer in the massive potential of the “learning machine.”
A New Dawn
What Marvin Minsky and Seymour Papert did not account for in their industry-altering book was that neural networks would become multi-layered – a breakthrough that, along with the significant developments in the processing power of computers, would eventually end the AI Winter and open up a world of possibilities for the implementation of artificial intelligence and machine learning.
Like many of mankind’s greatest technological triumphs, the most significant technique propelling artificial intelligence into the future is inspired by nature. The concept of the artificial neural network (ANN), as perhaps the most advanced system in the realm of machine learning at present, is loosely based on the neural circuits that occur naturally in the brain. In a biological neural network, chemical and electrical synapses connect unimaginably intricate circuits of neurons that link together to make up the central nervous system. Each of these neurons has dendrites (receptors) and axons (transmitters) that respectively receive and send signals across a network of neurons, each of which translates various signals and stimuli into meaningful information for use in the brain.
Whilst much simpler in design compared to their biological counterparts, artificial neural networks work in much the same way. At the most conceptual level, neural networks can “learn” through considering many inputs via their neurons and adjusting the translation or processing of the data, based on the relevance of the output to the desired result – which may or may not
be dictated by the user. This is what makes this system of machine learning so powerful – the ability to learn and self-correct without the need for continuous manual intervention by the programmer. And critical to the evolution of neural networks in the quest for true self- actualising artificial intelligence has been the advent of deep learning.
Deep Learning
Deep learning involves the training of artificial neural networks that are several layers of neurons deep – known as deep nets. Instead of one node processing all incoming data and producing a final result, deep nets rely on sequentially filtering data through multiple layers to refine the output. One can think of these layers of neurons as a stack of sieves or nets, each with a different sized mesh, allowing some particles through whilst blocking others. In natural neural networks this
Unfortunately, the inventor
of the Perceptron, Frank
Rosenblatt, would not live to see the revival of his field – having died in a boating accident
not long after the release of Perceptrons ...
filtering process is similar – individual neurons decide which stimuli are most relevant, and which are not, in determining whether or not the synaptic connection will fire to pass on the signal to the next layer of neurons. Crucially, however, this filtering process is not a binary yes-no system, but rather relies on the calibration of a finely-tuned weighting mechanism for each neuron. The adjustment of these weights on the inputs to the neuron is the key capability that allows a neural network to “learn”.
To better understand this concept, let us imagine a common use for neural networks in the real world – image recognition. To narrow this down even further, let us just focus on a system that can recognise hand-written numbers. In this example, as in all deep learning neural networks, there are multiple layers of neurons making
18