Page 387 - Deep Learning

370 Conclusion

structured, unbounded Representations

17
Mental representations exhibit what psycholinguists call constituent structure.
A representation consists of components, parts, which themselves are mental
representations. Every mental representation at layer N is both a combination
of constituents from layer N­1 and itself a constituent of some representation
of greater complexity or scope at layer N+1. The number of layers cannot be
determined with any certainty or precision and is likely to vary from one type
of representation to another. in perception, many psychological accounts oper­
ate with only two layers, visual features and objects, but vision scientists have
identified perhaps a dozen distinct layers: features, contours, objects and scenes,
with further divisions within each. in the study of discourse comprehension,
18
researchers distinguish between concepts, propositions, text bases and situation
models. in the study of cognitive skills, researchers distinguish between actions,
19
21
individual rules and strategies, and in the study of declarative knowledge, it
20
is necessary to distinguish between concepts, beliefs and belief systems. it mat­
ters how the components of a representation are combined. The structure of
relations among the components is not imposed on the representation, but is an
integral part of the representation. A representation at layer N can be changed by
selecting different elements at layer N­1 or by combining them differently.
A repertoire of elements at layer N in conjunction with the rules for how
they combine to generate a possibility space at level N+1, and the astronomi­
cal sizes of such combinatorial spaces guarantee that surprises will never
cease. As an example from the material world, the number of different carbon
22
compounds runs into the millions. With respect to the living world, biolo­
gists report that there are more ways of making a beetle than one would have
thought; taxonomists recognize some 350,000 different species, and that num­
ber can only increase over time as evolution throws new species down and
23
naturalists pick them up. in the domain of human creativity, we can ask how
many different ways there are to put paint on a canvas: Beetles and painters
might be neck and neck with respect to the sizes of their respective possibil­
ity spaces. The concept of combination goes a long way toward explaining the
diversity and richness of mental representations.
By itself, the concept of a combinatorial space commits us to the counterin­
tuitive conclusion that cognition is bounded in the sense that there is a large but
finite and predetermined space of possibilities that we cannot transcend. To reach
deeper, we have to recall that the edge of a possibility space is movable, because
changes in the underlying layer might generate a different set of elements or new
ways of combining them. The projection from layer N to layer N+1 creates the