Page 389 - Deep Learning

372 Conclusion

concept is that forward propagation is selective: The results of an operation are
only transmitted along some subset of the unit’s outbound links. Choices are
made on the basis of activation levels that reflect past experience.
in the micro­theory of insight, these concepts are instantiated by the per­
ceptual system. The construction of percepts proceeds through a succession of
layers, each layer consisting of processing units – feature detectors, gradient
identifiers, contour recognizers, shape extrapolators, movement recorders and
so on – which take input from the layer below, manipulate it and, if sufficiently
active, pass the result of the manipulation forward to the next layer. The process­
ing units are linked through both excitatory and inhibitory links within each
layer. A processing unit is a choice point: A unit is typically linked to more than
one successor in the next layer and when a result is fed forward, it might travel
along some links but others. The choice in each unit is a function of the activa­
tion received from the lower­layer units as well as from surrounding units in
the same layer. Each choice point is subject to a capacity limitation; it can only
receive so much information per unit of time, it can only perform computations
of a certain maximum complexity and it only has so much activation to allocate
across its outbound links. Activation is allocated over outgoing links in propor­
tion to their strengths, and when the activation that is allocated to one outgoing
link is reduced, that amount is redistributed to the other links, once again pro­
portional to strengths. For the result of a processing unit to be forwarded along
an outgoing link, the activation of that link has to be above a certain threshold.
in the theory of cognitive skills, the processing units are goal­situation­
action associations, also known as rules. Rules match in parallel, and each rule
performs some transformation on the current contents of working memory.
The processing is selective because rules compete to fire. The results of exe­
cuting a rule are fed forward to the next layer of rules by placing those results
in working memory. in the theory of declarative knowledge, the elementary
unit is a single belief, and the larger structures are arguments, descriptions or
explanations. single beliefs serve as premises for inferences that generate new
conclusions, which in turn serve as premises for further inferences.

ubiquitous Monotonic learning

A person’s knowledge base is continuously extended as a side effect of routine
cognitive processing. Perception of an object lays down a trace that makes that
very object seem familiar when observed a second time. Repeated performance
of a particular task will gradually generate a stock of goal­situation­ action
associations that encode a habitual response to that task. in routine belief