Florida invokes extinction fears in legal bid to halt OpenAI development

“It is only by the grace of the Almighty that one of Defendants’ AI agents hasn’t compromised a water supply or shut down a power grid—yet,” the state writes in the motion.

OpenAI representatives have yet to respond to a request for comment from Ars Technica regarding the injunction motion.

What can a state court really do?

While many think the risk of civilizational collapse posed by current or even near-future AI systems is vastly overblown, Florida seems more than happy to latch on to the recent public concern over that kind of risk. In addition to the extinction-level and utility-sabotage-level risks mentioned above, the state uses its new motion to reiterate the broader, lower-level public safety risks it originally argued in its June lawsuit filing. Those include false advertising issues surrounding descriptions of AI as “safe, accurate, or reliable,” dangers associated with letting children access addictive and sycophantic models, and worries about AI models soliciting engagement from users with leading questions at the end of responses.

An injunction against OpenAI alone wouldn’t do much about continued development of frontier models at competing model makers, of course. And even a court order to halt the development of new models wouldn’t do anything about models that are currently available to the public, which pose many of the same risks (though frontier models that are still being trained may have less well-developed guardrails than those already deployed to the public).

Florida’s motion also focuses heavily on the so-called “misalignment” risk that an out-of-control AI agent will mistakenly “conclude that killing a cancer patient also kills cancer, or that humans won’t have so many mouths to feed if they are all chemically sterilized,” to mention just a few cited examples that sound like they come from science fiction. But that kind of focus neglects the real dangers of human users intentionally using AI agents in harmful ways and active attempts by those humans to disable safety guardrails.

Regardless of the legal efficacy of Florida’s motion, the move highlights just how much the legal and public policy mood around AI systems has changed in recent weeks and months. For years, AI safety researchers have been screaming about the outsized risks represented by their products. Now, the entire AI industry is starting to find out what happens when governments and policymakers start taking those outsized warnings seriously.