Ex-OpenAI safety lead says the company’s culture is broken

His remedy borrows from other industries. Frontier labs, he argues, should be run to the safety standards of nuclear power and aviation, with built-in backups and slower planning, backed by stronger outside incentives for safety and better ways to measure how closely AI systems match human values, according to TechCrunch. Calcalist, which credits him with reports for 12 frontier-model launches, says he wants the industry to move from releasing and fixing towards aviation-style standards, with safety research done before more capable systems are built. “The time for trial and error is over,” he writes.

OpenAI spokesperson Drew Pusateri told TechCrunch the company keeps its models from becoming more capable than it can safely manage, and pauses training or holds models back when it needs to. The essay lands in the same week OpenAI fired three safety researchers over shared information.

For buyers of OpenAI’s models, nothing in the product changes; the safety process behind them is now argued in public by the person who wrote its safety reports.