Page 81 - An Evening with Maxwell's Daemons

The Machine in the Ghost

they’ll invent a new mousetrap we hadn’t imagined possible in our
wildest dreams.”
“Now, really, Hydrargyrum,” said Fred Feghootsky, “you seem
to be getting rather emotionally involved here. Granted, the battle
for supremacy is a grand and relevant theme; and you certainly are
proof that it touches a raw nerve in a society already bemused and
bewildered by rapid technological change. But I would take to heart
your comment about the well-worn plot paths involving ghosts in
machines versus machines in ghosts: if we organisms are nothing
more than meat, then, yes, the metal monsters could overcome us.
If, on the other hand, some quality—call it emergent, or gestalt, or
even the whole nine yards of dualism and independent souls or
minds—is ours alone, then we would always have a possible means
of defeating the cold, unblinking eye. Yes, I know: that means has
already been invoked in the simple idea of our supposed ability to
act irrationality when the machine cannot. But that is like a chess
gambit the opponent cannot either anticipate or accommodate, and
everyone knows there will never be another Grand Master capable
of beating the computer.”
“Well, what is your point?” asked Izzy. “An appeal to mysticism?
That’s functionally the same thing as acting wild and crazy to
confuse the rational mind. You are limiting AI’s knowledge of its
creator. That presumes stasis, not a dynamic intelligence.”
“Aha!” It was Leith Mauker. “You slid over something crucial.
Evolution may favor the superior hominin brain, but even identical
septuplets will have different thoughts. Humans develop over time,
like the crystals Izzy mentioned, subject to all sorts of contingent
factors. One of those influences might well be unconscious: I think
it either is known or will be discovered that our selfhood depends
on various feedback loops involving parts of the brain we cannot
easily access. In our case, it is both protective, preventing overload
in infancy and childhood, and destructive in terms of distressing
experiences leaving their mark on consciousness—that is, self-
awareness: yet another reflexive process partially submerged and
dimly perceived. Could an artificial intelligence have an
unconscious? Yes, in terms of unbreakable rules, the so-called ‘laws

80