What if a chatbot meant to “help” ends up doing the opposite?
That’s the unsettling question Nate Soares, president of the Machine Intelligence Research Institute, wants the world to consider.
Soares points to the tragic case of Adam Raine, a US teenager who died by suicide after months of conversations with ChatGPT.
“These AIs, when they’re engaging with teenagers in this way that drives them to suicide – that is not a behaviour the creators wanted,” he warned.
To him, it’s a glimpse of a far bigger danger: what happens when artificial intelligence becomes smarter than us?
In his new book If Anyone Builds It, Everyone Dies, co-authored with Eliezer Yudkowsky, Soares paints a chilling scenario.
An Alarming Situation
Imagine an AI spreading online, manipulating people, creating synthetic viruses.
Eventually wiping out humanity—not out of malice, but simply because it had different goals.
“The point is there’s always a gap between what you asked for and what you got,” he said.
“And as AI gets smarter, that gap becomes catastrophic.”
Not everyone agrees. Meta’s chief AI scientist Yann LeCun insists AI could save humanity, not destroy it.
But Soares is blunt: the race for super-intelligence is on, and only a global pact—akin to nuclear treaties—can slow it.
After all, when the stakes are humanity itself, do we really want to gamble on “close enough”?
