Page 297 - Deep Learning

280 Adaptation

further errors and hence might attract less attention. Everyone knows that the
Challenger space shuttle exploded due to a leaking O-ring, but knowledge of
how NASA made sure that this will never happen again is less widespread.
In some cases, there is no principled system response. Detected pre-failure
errors are passed over in the rush to get work done, and there is no systemwide
action. For example, safety scientists who study errors in hospitals report that
adverse drug events are quite frequent, in some clinics and hospitals as fre-
24
quent as one or more such error per patient. But they are handled through
local responses and immediate repairs because the nurses who detect them
work under severe time pressure and they have to find local solutions that
allow them to care for their patients. Amy C. Edmondson, a safety scientist
who specializes in the study of adverse drug events, writes that “… healthcare
organizations that systematically and effectively learn from failures occurring
25
in the care delivery process are rare.” If a collective does not try to learn from
its errors, then there is no response to describe. I know of no general descrip-
tion of how collectives respond to errors that is as well grounded in actual
cases as the descriptions of the errors themselves.
Common sense suggests that the natural reaction to an accident or a
near miss in a manufacturing plant or some other organization is to try to
“tighten” – make more explicit and precise – the relevant safety standards. This
typically means to formulate more specific prescriptions for how to perform
the shared task. In cases of this sort, the specialization principle applies.

The shape of change in collectives
If collectives learn in the same manner as individuals, what patterns or regu-
larities should we expect in the dynamics of error reduction in collectives? To
investigate this, we need to separate populations from organizations. Although
the term “population” already has several established usages, it will be used
here to refer to a set of more or less independent individuals who communi-
cate with other members of the set but who nevertheless go about their work
in an independent manner. There is no division of labor and no organizing
center. The set of all pilots that fly commercial airplanes and the population of
scientists in a discipline like chemistry or physics are examples.
In general, a collective S can be said to be a population, if the N individuals
that make up S are independent, or nearly independent, in the sense that an error
on the part of one individual does not affect the probability of error on the part
of the other individuals. However, correct behavior on the part of every indi-
vidual is necessary for S itself to function correctly. Differently put, it is sufficient
for a single individual to commit an error for the overall system to have made