Page 149 - Deep Learning

132 Creativity

levels of size and mass. It is equally applicable to the stone I throw in a
creek and to a planet orbiting a star; indeed, to entire galaxies. Today this
is so well established that we tend to forget how surprising it is. Before
Newton there was no good reason to expect the physics of small objects to
scale to the physics of the heavens, but it did. Chemical regularities, on the
4
other hand, do not scale with respect to temperature: A substance changes
radically at its melting and boiling points, creating new chemical phenom-
ena and hence forcing a division of chemistry into the study of solids, liq-
uids and gases. The General Gas Law is precisely what the label claims, a
law about gases; it does not apply to liquids. Scaling moves along different
5
dimensions in different types of systems. For each dimension some proper-
ties scale, others do not.
Scale is important because complex systems must be understood at dif-
ferent levels of description. But the properties of one level seldom matter
at higher levels. Only gross properties of the units and processes of level N
impact level N+1 directly. Consider the limitation on our working memory
6
capacity. It does not matter exactly how working memory capacity is limited
(amount of activation? interference? storage space?), but it matters greatly that
it is limited. This feature of our cognitive system constrains which cognitive
strategies we can reliably execute without external memory aids and hence
directly impacts higher-level thought processes.
As spelled out in Chapter 1, direct impact is one of four flavors of scaling.
7
In amplified, cascading causation – popularly known as butterfly effects – a
property at system level N propagates upward to have a greater effect on level
N+1, which in turn propagates so as to have an even greater effect on level N+2,
and so on. A contrasting flavor of scaling is called self-organization. This con-
cept applies to a system that consists of a very large number of similar compo-
nents that interact according to rules that apply to each pairwise interaction.
The components fall together into structures that are stable and exhibit prop-
erties that differ from the properties of their components. Self-organization
is presumably what brain cells do in response to experience. A fourth flavor
of scaling is level-invariance. The determinants of some patterns are inde-
pendent of the material constitution of the relevant system, so the patterns
recur at multiple system levels. Sometimes this is referred to as self-similarity;
the system looks the same at each level of scale. Sometimes level-invariance
comes about because the units at system level N exhibit some property P such
that when multiple units are combined into a larger-scale unit, that unit also
exhibits P. Direct impact, cascading causation, self-organization and level-
invariance are not competing or mutually exclusive principles. The task is