Page 223 - Deep Learning

206 Adaptation

Every action is a probe that bounces off the environment like a sonar signal,
returning with the outlines of otherwise unseen causes, objects and processes.
Deviations between expected and observed returns tell us that the world is not
as we assumed and hence provide an opportunity to correct our assumptions.
The prevalence of error must have been one of the factors that exerted selec-
tive pressure on early humans once they set out on the unique evolutionary
pathway of relying more on acquired than innate skills. As the hunter-gatherer
bands moved through habitat after habitat on their great migration across the
3
globe, their survival strategies were forever becoming maladaptive. It is plausi-
ble that they evolved a special-purpose cognitive mechanism for making use of
the information that resides in errors, failures and other undesirable outcomes
to improve the fit between their strategies and their environments. If so, errors
are not merely eliminated as a side effect of successful adaptation. Errors play an
active role in their own elimination; we unlearn errors by learning from them.
The theoretical questions are these: What information resides in erroneous out-
comes? How, by what cognitive processes, can that information be extracted and
utilized? What behavioral implications follow from those processes?

FRAMING THE PROBLEM

The old proverbs burnt child dreads the fire and once bitten, twice shy suggest
that learning from error is straightforward: The cure is to refrain from perform-
ing whatever action produced the bad outcome. Edward Thorndike codified
4
this idea in the second half of his Law of Effect: What he called an “annoy-
ing aftereffect” (i.e., an undesirable outcome) lowers the probability that the
5
learner – adult, animal or child– will perform that same action in the future.
Repeated negative outcomes cause the erroneous action to disappear from the
learner’s behavior. In Thorndike’s colorful terminology, the “futile impulse” is
6
“stamped out.” In the terminology introduced in Chapter 6, this effect can be
modeled by reducing – weakening – the strength of the rule that produced the
offending action. That rule will then lose to competing rules during conflict res-
olution and hence apply less often and therefore generate fewer errors. Learning
a skill is to a considerable extent a matter of learning what not to do.
This don’t do it again response to error works with respect to sticking
fingers into flames, but it does not work as a general explanation. Unlike
burning one’s fingers, most actions are not intrinsically correct or incorrect.
For example, if an absent-minded professor tries to open the door to his
home with the key to his office, the lesson cannot be never use the office key,
lest he lock himself out of his office forever. The more plausible lesson is not