Page 92 - Deep Learning

The Production of Novelty 75

generate-and-test cycles, but with the twist that the generation is not random
because the failure of one action informs the selection of future actions.
Thorndike’s interest in aftereffects, or reinforcement in the behaviorist
terminology foreshadowed a major intellectual development at mid-century.
During World War ii, psychologists and technologists worked shoulder to
shoulder on many difficult and unfamiliar problems, including how to aim (or
train someone to aim) an anti-aircraft gun. if the gunner aims at a moving
42
target but misses, how should he correct his aim? interdisciplinary work on
this and other problems was the origin of cybernetics, the science of control
systems – the human mind among them – that was launched by the math-
43
ematician Norbert Wiener and his associates in the 1940s. although the
cybernetic approach faded with the postwar spread of digital computers, it
left behind a three-part legacy. First, the cybernetic term feedback was gen-
erally adopted to describe information about the outcome of an action that
can be used by the acting agent – man or machine – to guide future behavior.
Second, in the transition from “reinforcement” to “feedback,” the modifiers
“positive” and “negative” lost their motivational flavor. Machines know naught
of pleasure or pain, so feedback is neither reward nor punishment but merely
information. technically, feedback is positive when it implies that the agent
should repeat the action that caused it, negative when it implies that the action
should cease.
The third advance was a shift to a more fine-grained analysis. Wiener’s
feedback systems continuously compared the effects of their actions with their
intended effects. an attempt to perform a task – a single trial in the trial-and-error
schema – might require multiple elementary actions, each of which can generate
feedback. This shift played a central role in Plans and the Structure of Behavior,
the 1960 attempt by G. a. Miller, e. Galanter and K. H. Pribram to specify what
we now would call a cognitive architecture by replacing reflex arcs with feed-
back circles. a similar project was undertaken by W. t. Powers in Behavior: The
44
45
Control of Perception. When cybernetics faded, so did these theories, but the con-
cept of feedback, the use of outcome information to guide future action, remains
as Norbert Wiener’s permanent contribution to the cognitive sciences.
Both behaviorism and cybernetics focused on overt behavior. Navigating
a maze, tracking a moving target like an airplane and hammering a nail were
prototypical examples of the kinds of behaviors these theories attempted to
explain, unpromising starting points for the study of creative thinking. in the
1960s, Nobel Prize–winning economist H. a. Simon and computer scientist
a. Newell put thinking and problem solving front and center by moving the
idea of a succession of choices between alternative actions, guided by action