Page 406 - Deep Learning

12

The Recursion Curse

… all inferences from experience suppose, as their foundation, that the future
will resemble the past. … It is impossible, therefore that any arguments from
experience can prove this resemblance of the past to the future, since all such
arguments are founded on the supposition of that resemblance.
David Hume 1

… predictions based on some regularities are valid while predictions based
on other regularities are not. … To say that valid predictions are those based
on past regularities, without being able to say which regularities, is thus quite
pointless.
Nelson Goodman 2

If the deep learning thesis is at least approximately correct, our cognitive
processes not only endow us with cognitive mechanisms for learning new
knowledge and skills but also allow us to override what we have learned. The
three micro-theories of creativity, adaptation and conversion describe cogni-
tive mechanisms that accomplish this. It would be foolish to claim that the
micro-theories are exactly accurate. Further research will no doubt force revi-
sions. However, there is no reason to expect the fundamental principle that
knowledge is, and must be, defeasible to be contradicted by future research.
This feature of cognition is grounded in the ceaseless turbulence of the mate-
rial and social worlds and in our evolutionary strategy of relying more on
acquired skills than on innate behaviors.
The consequence is that we sometimes succeed by projecting the past
onto the current situation, but at other times we do better if we override prior
experience and generate a novel response to the situation at hand. The choice
between extrapolation and drawing back to leap is, in part, under voluntary
control and it seems possible that we could become more disposed to override

389