Page 388 - Deep Learning

Elements of a Unified Theory 371

space of possibilities at the latter layer, but N is itself created by projection from
the preceding layer N­1. Changes in the latter create a different layer N and hence
a different possibility space at layer N+1. The linking of at least three layers creates,
not a single space of possibilities, but a space of possibility spaces.
A philosopher or a lawyer – specialists in counterarguments – might object
that the three­layer model only pushes the problem of a fixed boundary on our
cognitive powers one step backward without eliminating it. if the elements and
the combination rules at layer N­1 are constant, then the space of possibility spaces
is itself finite and bounded. A space of possibility spaces is an awesome construct,
providing what for practical purposes might be called infinite variability, espe­
cially if we recall that in most cases there are likely to be more than three layers.
By the time we work through the combinatorics of, for example, 10 layers, the
total set of possibilities might transcend what can be explored in a lifetime, and
perhaps in all of human history. nevertheless, it might seem as if the process of
reaching down into the next layer must bottom out at the lowest layer, so, in prin­
ciple, the total space of possible mental representations is finite and bounded. if
the human species survives indefinitely, will we eventually reach a state in which
we have formulated every idea that the human brain can conceive?
The conclusion does not follow. First, higher­order layers are feeding back
information to lower layers. This means that the elements at the lowest layer,
the sense data of positivist epistemology, are not fixed but partially shaped by
the higher layers that are built out of them. i know of no reason that a system
built on the basis of such circular interactions could not continue to change
forever due to its internal dynamics. second, the processing units at the so­
called lowest layer are not fixed but are the result of self­organization among
neurons in response to environmental stimulation. Which sensory features
we can distinguish in the world depends on which world we live in. As long as
the material and social worlds remain turbulent, there will be no fixed lowest
layer and hence no limit to change, no principled boundary on our mental
representations.

layered, selective, Capacity­limited, Feed­Forward Processing
A processing layer consists of processing units, and each unit in layer N receives
inputs from some of the units in layer N­1. Processing layers need not be thought
of in physical or spatial terms. They are defined by processing relations: A unit
that takes input from another unit is at a higher level than the latter. Choices
are made within each unit regarding which operations to execute on its inputs
and which computational results to forward to units in the next layer. The key