Page 126 - Deep Learning

Creative Insight: The Redistribution Theory 109

excitatory or inhibitory activation passed sideways from them to other units
within the relevant layer. Unlike most network theories, the present theory
assumes that forward propagation is selective.
Fourth, when an option is executed, its activation level is affected by sub-
sequent events. A negative outcome can be either an internal evaluation of a
problem state in the mind’s eye (this is not going to work) or the perception of
a physical or undesirable outcome (that didn’t work). This outcome is fed back
to all choice points that were instrumental in producing it. The effect of the
negative feedback is to subtract a certain amount of activation from the rele-
vant option or options. Presumably, the amount of activation lost is a function
of multiple variables, including the importance of the problem and the severity
of the negative feedback. We would expect a painful outcome in a matter of
great importance to have a greater effect than a minor annoyance with respect
to something trivial. The structure of the present argument is not dependent
on the exact function that determines the activation decrement. It is sufficient
that the activation of an unsuccessful option is lowered by some amount.
Fifth, the activation subtracted from one option is redistributed over all
other options associated with that option. The redistribution process can
operate in different ways, but the simplest assumption is that the activation
subtracted from an unsuccessful option is redistributed in proportion to the
relative strengths of the other options at the same choice point. The result is
that the option that was already tried now has a lower level of activation and
the others a higher, by some amount.
Sixth, a processing unit that receives negative feedback passes that feed-
back down to the layer that precedes it. Its antecedents might thereby be
pushed below threshold, in which case the unit loses its own inputs. The effect
is that the unit turns itself off. This restricts the solution space, because fewer
options are considered. But cognitive processes and structures are always com-
peting against their alternatives. A processing unit represents options, but it
also places constraints on its rivals via inhibitory, within-layer links. If a unit
becomes dormant, its inhibitory activity ceases. Turning off a choice point
relaxes whatever constraints it imposed on the alternatives to the options it
represents. This gives other, competing choice points a chance to become
active and propagate activation and information along their outbound links,
so the overall effect can be to widen the search space.
A processing unit that implements the six redistribution principles
will exhibit alterations in mode and tempo like those that define the insight
sequence. To demonstrate this, I embedded the redistribution principles in
a computer model of a single processing unit with two input links and three