An OpenAI safety employee has quit and is sounding the alarm

David Robinson used to write the safety reports that accompanied every major model release at OpenAI. This week, he resigned from his position and is now speaking out in an editorial in The Atlantic.

It’s understandable if you’re feeling a bit cynical about everyone suddenly coming out of the woodwork to warn about how dangerous the thing they helped build is. They did, after all, make this mess. But that doesn’t mean we should discount their warnings.

Robinson says that the culture in industry is fundamentally broken. That this is a deeper issue than simply slapping a few new rules or regulations on how we handle training models. Silicon Valley has operated with “extreme confidence” and “perpetual sprints,” he says, building bigger and better models with “unimpeded optimism” that ignores or underestimates potential problems.

He says the time has come for AI companies to develop a sense of humility and look outside the insular, move-fast-and-break-things world of the tech industry. Specifically, he says AI needs nuclear-level safeguards:

Given today’s risks, frontier labs need to run like nuclear power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster.