Children telling each other scary stories around the campfire is a long, long tradition. They know those stories aren’t true, but they (and even adults) sometimes scare themselves anyway. The thrill of the scary story is just too good. And they do offer some reminders that are useful. No, the hook murderer is not waiting in the backseat of your car, but yes, you should be vigilant of your surroundings because there are far more mundane things that could hurt you.

And so it goes with Silicon Valley AI apocalypse stories. This week, the tweet that launched a thousand hot doomer takes was from a corporate AI researcher. But it was energized when Anthropic’s head of alignment reposted it, saying “we really do earnestly believe AI could kill all humans!” (What a place for an exclamation point). 

While I do think (as I’ve unpacked elsewhere) this existential doomerism is almost always an attempt to hype tech or screen AI companies from taking real responsibility for the significant safety, security, and democratic problems they are causing, I believe this tweet. I DO think that there are many AI leaders who “earnestly believe” AI could kill all humans. The groupthink in Silicon Valley is incredibly strong and dramatic, compelling silly ideas spread like wildfire. Most of these tech elites also “earnestly believe” they are smarter than everyone else (so why listen to anyone outside the valley) and but also lack the kind of self reflection that would prevent them from wholeheartedly buying any kind of nonsense their more outrageous peers are selling.

Of course, Dario Amodei of Anthropic has weighed in with his detailed analysis and earnest ideas for a better way forward. That includes slowing the pace of development (which may just be a function of the costs these companies are running into) but also of encouraging third party assessors to vet the models (I should say they should vet the companies and their structures as well). Despite the silly idea that antitrust concerns would block responsible action, its an interesting essay. As always, I hope this time the “discovery” of good governance by Silicon Valley goes somewhere. In the meantime, here are some of the great takes, including by leaders in good tech governance and cybersecurity, delivering smart critiques and offering good ideas:

Getting started, you need to read Kevin Collier’s post on the need to actually define AI’s risks:

In Wired, Timnit Gebru presents a reality check, describing this existential risk concern as an attempt to draw focus away from realistic harms.

Avaro Bedoya effectively deflates the ridiculous claim that antitrust law is a bar to responsible AI: “Antitrust law does not prevent AI companies from coordinating to make sure AI does not hurt people.”

Christy Hoffman highlights the disconnect between AI company’s claims about their “earnest” beliefs and their lobbying efforts against safety

Henry Ajder also describes the dynamics of power and trust at play.

Of course, with cybersecurity and AI being one of the most critical actual risks we’re facing. The OGs of cyber have a lot of important stuff to say: 

Both Jen Easterly and Ciaran Martin highlight the errors in thinking among AI execs on cybersecurity but also point out that taking the underlying problems and proposals seriously is critical.

And Joe Levy brings up the “Quis custodiet ipsos custodes?” issue here. Good governance is all about watching over our systems even super powerful ones we are supposed to trust (iykyk…)

Ultimately, the fiction that AI will take over and there are no human decisions is just that, a fiction, a scary story. If the story keeps us vigilant about our responsibilities, then it’s served its purpose. If it distracts us from real-world risks and harms, then it’s far past time to move on. In any case, its human decisions that determine the course of AI.