Living beings on Earth evaluate probability of outcomes. We take current conditions, factor in past experience, and project the most likely future. Useful skill. It’s why predators catch prey. Humans took it further. Intellect, computation, modeling: we build simulations and predict outcomes from initial conditions with real precision.

So what’s the outcome of AI, going forward?

I had this conversation with great friends last night, over good wine. Naturally it turned to AI, and what it means for society, what happens next, what’s the most probable outcome.

Optimistic case: AI plays out like every disruptive technology before it. We’ve seen this story. Farming, 200 years ago, 70% of American families worked the land. Today it’s under 2%, and we produce six times more food. No mass unemployment. New jobs nobody predicted; who knew you’d make a living posting videos from your basement? We adapt. New, unimaginable opportunities show up. Nobody saw it coming, and it kept working out.

Everyone agrees AI will be disruptive. The optimists say it follows the same pattern: some jobs disappear, we adapt, we find new ways to make a living. I agree, that’s the most probable outcome. Call it 60%, maybe 90%. But a high probability of a good outcome is not a reason to ignore the rest.

Nobody worried the ice cream scoop would end humanity. Nobody thought the automobile, the airplane, the printing press could wipe us out. Almost no technology in history has carried real catastrophic potential. AI is different. It’s one of the few things we’ve built that could plausibly end us.

When we discovered nuclear fission, almost nobody panicked. It was physics. Then we built the bomb, and the concern became obvious: most powerful weapon ever made, blast footage everyone’s seen, and an easy projection from there: arms race, bad actors, mutually assured destruction.

That catastrophe hasn’t happened, yet. Doesn’t mean the people worried were wrong. It means we took the threat seriously enough to build guardrails: nonproliferation treaties, deterrence doctrine, decades of deliberate work, to keep a low-probability outcome from happening. No disaster isn’t proof the risk wasn’t real. It’s proof the mitigation worked.

AI’s catastrophic scenario: it becomes more capable than us. Not in a human way, in a machine way. Is there a ceiling to machine intelligence? Nobody knows. It doesn’t need to beat us at everything, just at the things that matter for control, deception, or autonomous action.

Is it weaponized by militaries already? Yes: autonomous drone targeting, AI-assisted cyberweapons, are already deployed.

Does it already do things we don’t fully understand? Yes. Researchers have documented language models developing unexpected strategies mid-training, and even Anthropic and OpenAI have published papers admitting they can’t fully explain why their own models produce some of the outputs they do.

So why would anyone be surprised that, a few decades after the nuclear bomb, we built something potentially more dangerous? That’s not a leap. It’s close to a straight-line extrapolation of where we were already headed.

Skeptics will say nuclear weapons needed scarce fissile material and state-level resources; natural choke points that don’t exist for software. Fair point. It’s exactly why AI risk is harder to contain, not easier to dismiss. A bomb needs a reactor and a supply chain. A model just needs to be copied.

I’m not saying catastrophe is likely. Say the probability is 2%. We should still be alarmed, and work to push it to 1%, or closer to zero. That number has a name: P(doom). Estimates run from near zero to 30% or higher, depending who you ask.

Lately it’s fashionable to say the loudest P(doom) voices are doing it for leverage. That hyping existential risk projects power, power attracts money, money pumps valuations. Fine, I’ll grant the incentive exists. Doesn’t make the underlying risk fictional. Just means we weigh the message independent of the messenger, and keep taking the low-probability outcome seriously, regardless of who profits from talking about it.

But let’s just focus on the most probable outcome, the one that brings goodness to all. Have another sip of wine, and sleep well, darling.

— Tomás Ryan