Page 114 - An Evening with Maxwell's Daemons

“It’s a no-brainer.”

“Certainly, AI has its critics and enemies: they see humanity up
against the wall, on the verge of irrelevance and replacement. The
highest-level ruling intelligence may have contradictory directives:
save the planet and preserve humanity. No human can be certain
which priority is being served by any specific instance of the
system’s decisions. It is smart enough to be duplicitous; after all, we
taught it that the ends justify the means. And we have left this
super-brain to develop its own algorithms for evaluating courses of
action, confident that it would do a better job than we have of
running the world. Ethics and morality? We supplied them as input,
maybe even gave them high values in the initial programming—but
now we have no idea of their weight in calculations of such abstruse
complexity that no human could unwind them. All that the
opponents of the system can do is yell, ‘It’s a no-brainer!’ whenever
they judge that their affairs have been handled remotely by a clever
computer instead of a low-wage, poorly-education human—or even
a highly-trained professional person, for that matter. Scream, rant
and rave, and hope others will realize that even the autonomy of the
autocrats has been lost.”
“My unresolved scenario, therefore, is the evolution of such a
popular movement and what the response of the system might be
to it. That, in turn, relates to whatever chance you think we have of
wresting control from universal AI once it has taken over. It seems
like we always are looking over our shoulder at prospective readers,
wondering if they are pessimists or optimists, and whether or not
we should let that influence the resolution of crises that—by
dramatic necessity—could go either way. Our own autonomy as
creative artists cannot help but be compromised by such
considerations, but we are involved in oxymoronic commercial art
and cannot ignore them. My tendency is to leave it unresolved, an
outcome perhaps as unlikely as the reassertion of human control
over its creations. That may be optimistic, or simply an opening for
sequels if the first story is successful. What do you think? How
would grass-roots resistance play out? It ultimately means finding a
way to be smarter when you are not as smart.”
Perversity Tinderstack wasted no time in responding.

113