Page 397 - Deep Learning

380 Conclusion

Table 11.2. Summary of the deep learning principles.

Spontaneous Activity:
The cognitive system is continuously and spontaneously manipulating mental
representations; activity per se needs no explanation.
Structured, Unbounded Representations:
Representations consist of (less complex) representations. The simplest representations
are not fixed but continuously re­shaped by feedback and environmental turbulence.
Layered, Feed-Forward Processing:
Representations are created through a succession of layers, the units in each layer
performing certain computations on their inputs and passing their results forward.
Selective, Capacity-Limited Processing:
Each processing unit passes its partial results forward selectively, along some of its
outward bound links but not others, due to limits on cognitive processing capacity.
Ubiquitous Monotonic Learning:
The cognitive system is continuously creating new representations in the course of
processing and some of those representations are stored in long­term memory.
Local Coherence and Latent Conflict:
The creation of mental representations is not subject to any global coherence check;
coherence is only maintained locally. Cognitive conflicts can remain undetected.
Feedback and Point Changes:
Higher processing units feed outcomes of behavior down through the processing layers,
possibly tipping the balance among options at a processing unit in some lower layer.
Amplified Propagation of Point Changes:
A change at a single point in the processing system might amplify as it propagates upward
through the processing layers, and hence create a new top­level representation.
Interpretation and Manifest Conflict:
Activation of a representation R A for a domain A in the context of some domain B might
reveal that R A applies to B as well, and thereby makes the R A ­R B conflict manifest.
Competitive Evaluation and Cognitive Utility:
Conflicts among representations are resolved on the basis of quantitative properties that
reflect the past ability of the competing representations to produce successful outcomes.

Are the deep learning principles also collectively sufficient? That is, will
a cognitive system that exhibits these properties inevitably undergo non­
monotonic change? if such a system lives in a clockwork world in which past
experience always and accurately predicts the future, there is no need for non­
monotonic change and hence such changes are unlikely to occur. in a turbu­
lent world, on the other hand, the deep learning principles are sufficient to
guarantee that a non­monotonic change will occur sooner or later. if negative