Page 148 - Deep Learning

Creative Insight Writ Large 131

system levels? The traditional strategy for how to build this kind of bridge is ster-
ile, but the complex systems perspective provides an alternative.

GENERALIZATION VERSUS SCALING
Philosophers and psychologists often discuss the relevance of results from lab-
oratory studies in terms of a hypothetical process called generalization. When
generalizing, a person supposedly takes a statement that turned out to be true
in one or a handful of situations and asserts that it is also true of other, similar
situations. A simple example is to conclude that golden retrievers are friendly
after meeting a few and finding them friendly. The notion of generalization has
its roots in empiricist philosophy and it is deeply embedded in the culture of
academic psychology.
Like other types of projections from past experience, generalization car-
ries epistemic risk: How can we know the set of situations in which a par-
ticular assertion holds? The fact that a handful of golden retrievers turns
out to be friendly does not, after all, guarantee that all golden retrievers
are friendly; there could exist, somewhere, a retriever that is miserable and
3
snaps at babies. This much is common sense. The problem is that not even
10,000 friendly retrievers warrant the conclusion that all golden retrievers
are friendly; a miserable one might nevertheless be lurking somewhere.
N. Goodman and other philosophers have tried to pinpoint the conditions
under which a projection from the observed to the not-yet-observed is
warranted, without having produced a satisfying explanation of how such
projections are supposed to work, and without producing a technique for
generating valid generalizations in practical contexts. The root of the prob-
lem is that generalization is defined in terms of the size of the reference set;
the concept “dog” is said to be more general than “retriever” because there
are more dogs than retrievers. But this is only true at a single moment in
time. Over infinite time, all reference sets are infinitely large, so this concept
of “more general than” is incoherent.
Systems theory implies that the central concept in projecting knowledge
gained from the study of simple situations onto more complex ones is not gen-
erality but scale. Everyday situations extend further in space, last longer and
have more components than simplified laboratory situations. The question is
how the processes that operate at small scale influence systems that are orders
of magnitude greater along one or more dimensions.
Different sciences scale along different dimensions. One remarkable
aspect of Newton’s theory of mechanical motion is that it applies across