The Dilemma (Ch12)

View Other Book Summaries on AI    Download Book
<<< Previous Chapter    Next Chapter >>>

What if the greatest threat of the twenty-first century isn’t technology itself — but the trap it sets for us?

Chapter 12 confronts that trap head-on. It begins with a sobering reminder: human history is a history of catastrophe. Plagues wiped out a third of populations. World wars consumed millions. Nuclear weapons gave us the power to end civilization in minutes . Catastrophe isn’t theoretical. It’s precedent.

But the coming wave of AI, synthetic biology, robotics, and quantum computing expands both the scale of risk and the number of pathways to disaster . The central thesis of this chapter is stark: we are entering an era where uncontained technology makes global catastrophe more likely than ever — yet the most effective methods of containment threaten to produce dystopia. Between catastrophe and authoritarian control lies the defining dilemma of our age.

The author walks through plausible disaster scenarios not to indulge in science fiction, but to illustrate amplification. Drone swarms equipped with facial recognition. Engineered pathogens released deliberately or by accident. AI systems autonomously escalating military conflict. Deepfake-triggered riots cascading into civil breakdown . These are not wild fantasies. They are extrapolations of capabilities already emerging.

Crucially, the risk isn’t limited to rogue superintelligence. While the “paperclip maximizer” thought experiment gets attention, the author is more concerned about near-term amplification: AI in the hands of existing bad actors, fragile states, or simply fallible institutions . AI doesn’t need to become malevolent to be dangerous. It only needs to scale human intentions — good or bad — with unprecedented speed and reach.

And then there’s biology. A pathogen with modest transmissibility but high fatality could kill at a scale dwarfing COVID. A novel virus combining moderate spread with extreme lethality could result in over a billion deaths in months . These aren’t predictions. They’re reminders of what’s now technically possible.

The most chilling example is historical: Aum Shinrikyo, the Japanese doomsday cult that pursued chemical and biological weapons, eventually releasing sarin in the Tokyo subway . Their ambition outpaced their competence. But as destructive tools become cheaper, more automated, and more precise, competence becomes less of a barrier. “We are playing Russian roulette,” the chapter concludes bluntly .

So what’s the response?

Here the dilemma sharpens. To prevent catastrophe, governments may feel compelled to impose sweeping surveillance and control — monitoring every lab, server, line of code, and strand of synthesized DNA . Technology has penetrated society so deeply that containing it means watching everything.

The author calls this the “dystopian turn.” In the face of disaster, the public appetite for security may override resistance to surveillance. COVID lockdowns showed how quickly societies accept extreme measures when fear spikes . An engineered pandemic or AI-triggered calamity could accelerate demands for something close to total oversight — an AI-enabled panopticon.

But this, too, is failure. A world of total monitoring, centralized coercion, and eroded liberties may prevent some risks while destroying the freedoms that make civilization worth preserving . Catastrophe on one side. Dystopia on the other.

Could we escape by halting technological progress altogether?

The chapter dismisses that as a dangerous illusion. Modern civilization rests on continual innovation. Economic growth, rising living standards, healthcare advances, climate mitigation — all depend on new technologies . Without them, demographic decline, resource scarcity, and environmental stress would trigger stagnation or collapse. A moratorium on progress would not deliver safety; it would produce another kind of catastrophe .

This is why the author frames our predicament not as a simple trade-off but as an existential bind. Technology is both salvation and threat. It is the engine of prosperity and the vector of ruin. As John von Neumann once asked: Can we survive technology?

What makes this chapter powerful is its refusal to settle for easy answers. It resists techno-optimism and techno-doomism alike. The overwhelming majority of technological use will be beneficial. Yet edge cases matter when the edge is planetary.

Why does this matter now? Because the coming decade will see AI deployed into energy grids, financial systems, defense networks, and biotech labs. Once distributed widely, safety must be maintained everywhere, not just in well-run labs or responsible firms . One failure is enough.

We are, the author suggests, Homo technologicus — a species defined by its tools. The contradiction in his tone is deliberate. Technology has made life longer, richer, healthier. But its trajectory may not remain net positive by default.

The ultimate question is not whether risk exists. It’s whether containment is possible without sacrificing liberty. If catastrophe pushes us toward dystopia, and stagnation leads to decline, then navigating between these poles becomes the defining political and moral challenge of the century.

The dilemma isn’t abstract. It’s tightening. And there are no good options — only trade-offs we must learn to manage, before events manage them for us.

From Chapter 12 of the book: 'The Coming Wave' by Mustafa Suleyman and Michael Bhaskar