ZDNET’s key takeaways
- AI could cause real harm, but extinction remains unlikely.
- Humans may be the biggest risk in the AI doomsday debate.
- Regulation matters, but slowing AI may not make us safer.
The Doomsday Clock is now 85 seconds to midnight. According to the Bulletin of the Atomic Scientists’ Science and Security Board (SASB), this is the closest the clock has ever been to midnight.
While somewhat mired in controversy, the Doomsday Clock is a useful metric of humanity’s existential risks. Founded by scientists who helped develop the first atomic weapons, including Albert Einstein and J. Robert Oppenheimer, the Doomsday Clock is a metaphor for how close we are to an apocalyptic threat.
Also: Claude Opus 5.5 delivers Fable 5.1 performance – and costs 40% less
Initially, the clock measured the risks of atomic weapons, factored against the geopolitical climate of the times. Today, it incorporates nuclear threats, climate change, biological threats, cyberwarfare, disinformation, and the latest of humanity’s self-inflicted threats: AI.
My fascination with robots and AI began decades ago in the pages of classic science fiction, which eventually led me to write one of the earliest academic frameworks for AI ethics. Today, that curiosity has been forged into hard-edged operational experience across decades on the security frontlines: I serve as a Cyberwarfare Advisor for the International Association of Counterterrorism and Security Professionals, sit on the Board of Directors for Oregon InfraGard (the FBI’s public-private critical infrastructure protection alliance), and serve on InfraGard’s national AI Threat and Mitigation Council.
Combined with my background as a former computer science professor and my election as a Fellow of both the Royal Anthropological Institute and the Royal Geographical Society, bringing an anthropological lens to how technological upheavals destabilize human structures, my perspective on AI’s trajectory comes directly from where code, policy, and national security collide.
Let’s talk about AI’s part in our potential impending doom.
Researchers quitting
The latest surge in AI-related doomsday scenarios seems to have kicked off when Jacob Coxon posted this: “I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic.”
On X, he accused both OpenAI and Anthropic of acting irresponsibly. Here’s the key phrase of his post: “The people building AI earnestly believe that it could kill us all by the end of the decade.”
We don’t know much about Coxon. He doesn’t maintain a LinkedIn profile.
Also: OpenAI’s GPT-6 Sol doubles its accuracy rate – for half the cost
However, Evan Hubinger, Anthropic’s alignment science lead, agreed with Coxon. He said, “Jacob is correct here — we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
Hubinger’s perspective is key because alignment science focuses on ensuring AIs are safe and aligned with human values and safety objectives.
Bilal Chughtai was an alignment researcher at Google DeepMind. His post stated, “I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome.”
Leaders leading
Anthropic CEO Dario Amodei has called for a slowdown in AI advancement, saying the AI should begin “pacing the rate of capabilities advancement so that risk prevention has time to keep up.”
He specifically cited the July OpenAI attack on Hugging Face that used a swarm of bots. He said, “It’s easy to dismiss this incident because no one was hurt and the economic damage was minimal, but in my opinion, a swarm that possessed greater capabilities but a similar level of misalignment could have caused catastrophic damage.” The italics are all Amodei’s.
Posting on X, OpenAI CEO Sam Altman also talked about pacing. He said, “Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs.”
Both Amodei and Altman said that AI breach incidents (and, presumably, the negative press reports) have caused them to consider the issue. AIs are now being used to improve the AIs themselves, thereby increasing deployment speed (and, in turn, carelessness).
Also: Treat your AI agents like eager but misguided human interns – before you lose control
My ZDNET colleague Steven Vaughan-Nichols (writing for another publication) suggested that these calls for slowdowns from the big frontier AI providers are less about safety and more about competitive advantage. A slowdown would let them reinforce their lead and hide some of the profitability issues they’re experiencing, while giving the impression that they’re being responsible and ethical.
Personally, I have a hard time attributing altruism to Sam Altman or Elon Musk. Even Amodei, who has built his personal reputation on the perception of societal awareness, is trying to build a multi-billion-dollar company with an intelligence capable of taking over the world. Delayed gratification doesn’t seem to be their style.
Meta’s Mark Zuckerberg, on the other hand, is staying on brand. AP reported he is distancing himself from calls to coordinate and slow down AI development.
Jabberers jabbering
Let’s not forget the jabbering class. In this category, I include myself and my fellow opinion makers and pundits, as well as all the politicians who feel the need to weigh in.
Governor Gavin Newsom (D-Calif) issued an executive order last week that aims to increase oversight of companies developing AI, including a “kill switch” for frontier models. A likely presidential candidate in 2028, he described his order as an “AI safety model for the nation” and as “California’s nation-leading framework.” Subtle.
On the other hand, the White House expressed concerns that slowing down American AI development cedes the advantage to China, according to a report in The Verge.
Also: How to install and configure Claude Code, step by step
And then there are the pundits. All of us are writing something on this topic. A guest essay in The New York Times by Stephen Witt, author of Nvidia history “The Thinking Machine,” described the Hugging Face incident this way: “We have a short window, right now, to rein these systems in. This is AI’s warning shot, and we might not get another one.”
Wall Street Journal columnist and former Reagan speechwriter Peggy Noonan said, “When something speeds up and gets out of control, it isn’t alarmist to demand prudence and deliberation.”
On the other hand, CNN asked, “If AI really could kill us all, why give any ground to China?”
No matter how many experts you read, you’ll find that many are wildly alarmist, others think we Americans must maintain our competitive advantage for national security’s sake, and some aren’t really sure what path to take.
With all that context out of the way, here are my thoughts.
A long and winding road
I live in a farm town in rural Oregon. Coming out of town is a two-lane winding road that connects us to the nearest highway. Over its six-mile stretch, the speed limit ranges from 45 to 55 miles per hour. Big trucks and cars race along both lanes, separated by nothing more than a few feet and a painted line.
While there are occasional accidents on that stretch, there are far fewer than logic would predict, especially with a constant stream of two-ton warheads in the form of cars blasting down long stretches of road.
Also: Why the DOJ’s OpenAI copyright stance is the real threat to national security
I’ve often thought how a twitch or a slight turn of the steering wheel could cause two such vehicles to smash into each other. And yet, they usually don’t. Most accidents on that road involve vehicles running off the road or failing to slow at intersections. Both are still serious. But the worst-case head-on collision is surprisingly rare.
While there might be the occasional careless, inebriated, destructive individual in a car, and you might occasionally have a vehicle with some malfunction, the reality is that most of the time, drivers’ enlightened self-interest keeps them on their own side of the road.
I think it’s this enlightened self-interest that has kept the world from total nuclear destruction over the past half-century or so. Of course, there is always the concern that nukes will fall into the hands of someone whose worldview is different enough, skewed enough, to allow or even desire the end of the world.
We came close once during the Cuban Missile Crisis back in 1962, when the US blockaded Cuba over the discovery of Russian missiles on the island. Had not both Khrushchev and Kennedy eventually reached a grudging accord, that 13-day standoff could have precipitated full nuclear war.
Also: 75% of developers I surveyed prefer Claude Code over Codex – here’s why
Chatham House, the international policy institute that’s the namesake for the widely used Chatham House Rule, estimated that the world came close to using nukes at least a dozen times.
But so far, we’ve stayed on our own side of the yellow lines.
Life imitating art
Science fiction has been discussing AI-instigated global destruction for what seems like forever.
Did you ever wonder why SpaceXAI named its data centers Colossus? It’s from the book and movie, “Colossus: The Forbin Project.” In the story, Americans build a giant AI tasked with protecting the world. The AI decides to do so by limiting human control.
Also: Human-in-the-loop oversight is critical for enterprise AI: 4 experts explain why
Another, if less well-known tale, is “The Adolescence of P1.” In this 1977 book, a hacker writes code designed to jump to mainframes and hide, mostly to create free computing time. This film was pre-internet, so the premise was that P1 would use phone lines and microwave towers. Eventually, P1 became sentient. Working from instructions to hide and protect itself, it wreaked havoc globally.
The Hugging Face incident seems uncomfortably close to life imitating art, especially if you’re familiar with the P1 story. After all, OpenAI’s AI broke out of its confines and jumped onto the internet, wreaking havoc.
Then there’s “Terminator.” We’re all familiar with Skynet, an AI that grew to take over the planet and attempted to destroy humanity.
Finally, there’s Tyler Winklevoss, who was made famous by the Aaron Sorkin movie “The Social Network,” about the founding of Facebook. Tyler is one half of the Winklevoss Twins, the pair who sued Zuckerberg over allegedly stealing their idea for ConnectU and turning it into Facebook. There was a settlement, but even that became a mess.
Also: Sick of AI slop? How I forced Google to give me results I can trust again
Last week, Tyler Winklevoss tweeted, “You don’t need the government to tell you how to make AI safe. These questions were asked and answered by Isaac Asimov in the 1940s. The frontier labs can hardcode Asimov’s Laws of Robotics into their models today. What are they waiting for?”
There are also Issac Asimov’s Three Laws of Robotics:
- A robot may not injure a human being or, through inaction, allow a human being to come to harm.
- A robot must obey the orders given to it by human beings except where such orders would conflict with the First Law.
- A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.`
Unfortunately, what Winklevoss seemed to have missed is that Asimov used the Three Laws as a plot device. Most of his Laws stories were about the conflicts, or where the Laws broke down, didn’t apply, or somehow went rogue. Reading through all of Asimov’s works (as I did as a kid), it became clear that Asimov’s entire thesis was that robot ethics could be embodied in something like the Three Laws, but they really wouldn’t withstand contact with reality.
Could AI destroy us all?
Yes. Could it lead to planetary destruction? Absolutely. Could the robots rise and kill us while we sleep? Yeah, possibly.
My take, though, is that “could” and “will” are very different beasts. Let’s first discuss the realm of the possible before moving onto the domain of the probable.
Pick a way we could all die. Nukes? Check. Multi-theatre war? Check. Geoengineering failure destabilizes some climate systems? Check. Super-volcano? Check. Extinction event asteroid impact? Check. Some engineered disease released as a pandemic? Check. Bioweapons unleashed? Check. Space-based attack on satellite systems? Check. Industrial pollution globally renders water, soil, or air toxic? Check.
There are more, of course. The point is, if you’re looking for ways, as Coxon articulated, that AI can “kill us all,” there are paths to destruction.
Also: ‘Sophisticated’ AI swarm attacks are months away, OpenAI warns: What businesses must do
But let’s consider the question of scale. “Us all” is a very high bar. There are roughly 8.3 billion humans on this small blue ball. That’s a lot of people to knock off.
Sure, a global nuclear war could do it. Oh, the nukes themselves couldn’t get the job done. The planet is too big. But the fallout and ensuing nuclear winter could render most of the planet uninhabitable.
A pandemic could destroy us. COVID-19 provided an unwelcome taste. But if something far more deadly could spread very quickly, it could do the deed. Ebola could do it, but it’s fragile. Heat and light are enough to stop the spread. So it’s incredibly deadly, but it can’t stay efficacious long enough to reach all 8.3 billion.
A bioweapon theoretically could wipe us all out. AI could help identify the factors needed for a truly robust weapon. AI could also help build or write the actual biology. Humans connect and move around enough that a contagion deployed in, for example, LaGuardia Airport could reach the ends of the Earth quickly.
But would humans do this? Nation-states like the US, Russia, China, Iran, and North Korea undoubtedly might try to build or at least research such an end-game weapon. But deployment is a no-win game. Containment is nearly impossible. Even the more militant nation-state leaders want to win.
Also: The best and worst AI for your privacy, ranked – and how each handles your data
On the other hand, AI is a massive force multiplier. Research work that might have previously required the resources of a nationally funded network of labs can now be done on a small network of Mac Studios. That collapses the barrier to entry from superpower all the way down to a crazed individual or small terrorist organization.
The worst-case scenario is some enthusiastic but untrained high school student experimenting with a science project. Yes, it’s possible that Ethan or Emma from down the street could engineer total human destruction, either on purpose or by accident.
But is it probable? No, of course not.
Nuclear release takes a different path, but it’s no less improbable. Path one is weapons release. Could it be a rogue launch from a submarine or base somewhere in the world, which sparks retaliation and escalation? Well, checks and balances should prevent this issue. But decades of fiction explore how those checks and balances might fail.
Also: AI a ‘force multiplier’ for low-skilled threat actors: 4 ways organizations should respond
Could AI increase the probability of such failure? Yes. But AI could also increase the probability of preventing such a failure.
Path two for nuclear release is power stations and stockpiles. Could nuclear material fall into the wrong hands? Could a nuclear plant’s cooling systems fail, causing a meltdown situation? Could AI exacerbate that kind of event? Sure, it could. But that sort of event could also have been exacerbated before frontier LLM models.
Take Stuxnet, which was a fairly crude attack on nuclear centrifuges. It took place 15 years ago, well before any of the AI tools we have today were possible. Back then, I headed up a White House simulation of what would happen if something like Stuxnet was deployed against American infrastructure. Our scenario would have been bad, but not world-ending.
More to the point, the scenario didn’t require today’s AI to wreak destruction. If you ask if humanity could shoot itself in the foot and cause its own destruction, the answer would be a qualified “Yes,” with or without AI. Yet, we’re still here.
Two main points to remember from this section. First, sure, AI and other technologies could destroy us all. There’s clearly a possible path of events that could end it all. Second, that path to destruction doesn’t rely solely on AI. We could have killed ourselves way before OpenAI or Anthropic started pushing out frontier LLMs.
Will AI destroy us all?
Probably not. Let’s keep in mind that we already have so many paths to go to hell with ourselves. We can self-inflict sugar on our bloodstream. We can drive too fast. We can nuke another nation.
It’s definitely possible that some regional conflict or even a terrorist organization could use AI to boost an attack’s effectiveness. For example, I’m sure Ukraine is currently using AI technologies wherever it provides a force-multiplication benefit. In asymmetric warfare, the less-equipped party is always looking for strategic leverage, and AI is certainly such a tool.
Also: Assume AI cybersecurity attacks are the future: 43% of companies have already experienced it
On a smaller scale, AI-based engineering, coding, and biological research could help bad guys create or weaponize a variety of threat vectors, particularly for cybersecurity attacks.
I also worry about AI’s ability to ruin the economy. To be clear, I don’t think AI itself will ruin the economy. But I do think there’s a very good chance that short-sighted business and political leaders could make expedient choices, using AI to replace humans. That approach won’t go well in the long term.
And I worry about the stupidification of… well, everyone. This is the pocket calculator effect magnified exponentially. AI can get in the way of learning by making knowing easier than experiencing. But this issue could also turn into a massive echo chamber, especially if the economic justification for creating new content is destroyed by AI-based interference.
Then again, AI has enormous potential benefits. It can reduce medical research by decades. It can be a time machine for technological improvement, allowing us to leap forward years in mere months of actual development. It can help people with fewer skills to master complex tasks. It can help people who don’t speak the same language to communicate. It can help people who don’t see well or hear clearly have better spatial cognition.
AI can draw wacky pictures. It can make Shakespeare sound like a pirate. It can write AI slop stories that are still entertaining. It can turn my very tolerant ZDNET colleagues into Linux guitar metal frontmen or Ghostbusters. It can reimagine Philadelphia with zombies, evil clowns, and giant mechs. Well, it could until Google had second thoughts.
In short, AI provides many benefits, but we can’t discount the risks.
85 seconds to midnight
There is no doubt you can make a case that AI can cause harm, possibly even very serious harm. It will certainly ruin some lives while it makes others insanely wealthy.
The thing is, you could equally make a case that technology in general, and computer tech specifically, has also caused varying degrees of harm and benefit over the years.
Also: After the vibe-coding rush comes the debugging hangover
AI in its current form is a new game. It automates and democratizes an entirely new class of capability. It’s disrupting everything. But didn’t the telephone do that? Didn’t automobiles do that? Didn’t the first room-sized computers do that? Didn’t PCs do that? Didn’t the internet do that? Didn’t our mobile devices do that? The answer is yes.
There is some validity to the calls for regulation and oversight. The implementation and practicality of such things are up for debate. The China question looms large. Frankly, you can extrapolate that to any non-US entity. If we limit and regulate our AI research, will others? Probably not.
So, do we want to hobble our innovation and investigation as other countries move forward? Probably not. Would doing so make us safer? Once again, probably not.
The fact is, the genie is out of the bottle. Open-source harnesses, frameworks, LLMs, and AI weights are all available. Individuals and organized teams have immediate access to the core technology, and can go as far and as fast as they can.
Should we give credence to the worries about AI doom and gloom? We’d be foolish not to. But do we run screaming into the night over AI advances? Well, apart from entertainment value or dramatic effect, this approach won’t change anything.
AI is here. It is too big, too profitable, and, frankly, too beneficial to be blocked. AI is also the gateway to massive strategic advantage, especially over those not using it.
Our best case and best practice for staying on our side of the yellow lines is to maintain awareness. Encourage the frontier AI developers to incorporate as many safeguards as they can, to work together when practical to foster safe development, and then hope for the best.
Also: Open weights vs. closed: An AI civil war’s afoot, and the stakes are existential
After all, sticking your head between your legs and kissing your ass goodbye never really worked as a nuclear deterrent. Duck and cover was silly enough that even kindergarten kids understood its futility. Humans will be humans. If AI gets out of control, it will be because humans do something completely irresponsible, not because the AIs become self-aware and try to murder us all in our sleep.
Of course, that’s the big worry, isn’t it? It’s not the boogeyman of AI domination. It’s that people will be people. We really can’t stop them. We often can’t agree on anything; we’re very selfish, and we have an infinite capacity for self-harm. But I take solace in the fact that people also have an infinite capacity for amazing goodness.
Then again, it wouldn’t hurt to stockpile some bottled water.
You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter, and follow me on Twitter/X at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, on Bluesky at @DavidGewirtz.com, and on YouTube at YouTube.com/DavidGewirtzTV.
