Page 290 - Deep Learning

Error Correction in Context 273

ERROR REDUCTION IN COLLECTIVES

Collectives – groups, teams, organizations, populations – are no less error
prone than individuals, so a theory of learning from error is potentially appli-
cable at the collective level. In the context of social systems, the study of errors
is typically driven by concerns other than those that drive the study in errors in
education and training. In the latter contexts, error rates are indices of mastery
or lack thereof, clues to how the student or trainee thinks about a subject mat-
ter or a target skill. In the context of collectives, errors take on a different and
more sinister aspect: They are all too often among the root causes of accidents
18
and disasters. The consequences of an erroneous action can cascade through
the collective with disastrous results. The near-meltdown of the Three Mile
Island nuclear power plant, the explosion of the Challenger space shuttle, the
burning of the Hindenburg airship and the spectacular capsizing of the Swedish
17th-century warship Wasa on its maiden voyage in 1628 have become iconic
instances of disasters and failures in sociotechnical systems. The collapse of the
walkways in the Kansas City Hyatt Regency Hotel and the failure of the Tacoma
Narrows Bridge are equally familiar, at least among engineers. The failure of the
Xerox Corporation to exploit its lead in developing the personal computer, and
the Allied losses during Operation Market Garden in World War II illustrate
the potential for disaster in economic and military affairs.
Since the pioneering works of Jens Rasmussen, Donald Norman and
James Reason in the 1980s, safety scientists have pondered the nature of the
relation between errors on the part of individuals and accidents, disasters and
failures at the collective level. A variety of principles have been proposed. In
19
particular, there have been multiple attempts to classify errors into types as a
preliminary step to understanding the origins of errors. Some researchers have
focused on the cognitive mechanisms of individual errors, others on operator
actions when faced with opaque interactions among multiple simultaneous
breakdowns and yet others on the decision-making processes that let a space
shuttle take off with leaking O-rings or a commander to launch a battle in the
face of intelligence reports that indicate superior enemy forces. It would be
rash to claim that any one of these factors, or types of factors, is less than essen-
tial for the understanding of collective failure.
However, the list of relevant factors does not provide a theory of the
dynamics of collective error rates over time. Do error rates remain constant or
fall? If the latter, what is the shape of change? The observation that hierarchical
decision-making structures can override counterindicators to an action – the
Stupid Boss Theory of collective failure – makes no prediction other than that