Page 243 - Deep Learning

226 Adaptation

An unexpected or undesirable outcome is sufficient to show that the respon-
sible rule needs to be revised, but it is not by itself sufficient to determine the
direction of the change, and the number of logically possible changes is large. The
rule specialization process extracts the information that resides in the particular
way in which the outcome of an action deviates from the outcome specified by
the constraint for the domain and uses that information to decide how the faulty
rule should be revised. The content of the constraint and the manner in which
the observed outcome violates it – that is, the nature of the error – contains infor-
mation that can be used to compute exactly which restrictions should be placed
on the rule. In effect, we say to ourselves, I guess it is only okay to do A when
such-and-such is the case; for example, I guess it is only okay to switch to the left
lane when that lane is empty, or I guess it is only okay to turn on the projector when
it is already hooked up to the computer. The added condition elements ensure
that the revised version of the rule will not violate the particular constraint that
triggered the revision, but there is no guarantee that the revised rule is correct in
other respects. It might violate other constraints and so have to be revised again.
Eventually, it will only apply in situations in which it does not cause errors.
Are there alternative mechanisms for learning from error that are equally
consistent with standard cognitive concepts but that make significantly differ-
ent assumptions about the error-correcting process? Although the unlearning
of errors is both interesting theoretically and of considerable practical interest,
the majority of learning mechanisms incorporated into cognitive simulation
models capitalize on successes. For example, analogy and generalization both
require a correct step as input. There is little point in building an analogy with
a past problem that one failed to solve, or to generalize over one’s mistakes. The
same point applies to most of the mechanisms proposed in theories of skill
acquisition.
The core problem in learning from error is to distinguish the set of
situations in which an action or a rule has positive outcomes from those in
which it has negative outcomes. One long-standing line of thought proposes
that learners accomplish this by comparing particular situations of either type.
A rat might be trained in a laboratory to jump to the left when it sees a star-
shaped pattern and to the right when it sees a circular pattern, thus proving
that it can discriminate between the two shapes. Computational mechanisms
that operate on the comparison principle have been explored by P. Langley and
other theorists.
Langley’s version of discrimination utilizes the information in the execu-
33
tion history of a rule to compute the appropriate revision of that rule. Suppose
that action A is recommended by some rule R,