AI Recursive Self-Improvement without Review and Alignment

In the high-tech, high-competition AI field, we are seeing new versions of models rolled out by the frontier companies (OpenAI, Google and Anthropic) every month and a half. I included a Release Cadence Breakdown report by Google Gemini in the “Online: Trending Now” column just a few weeks ago. The speed and intensity of the research and development required to keep up such a release schedule has taken a toll on the necessary, thorough testing of reliability and alignment that we all expect must be done by reliable creators.

It is important to remember that this field is breaking new ground in creating models that can generate original concepts, often by mimicking approaches humans have taken in the past. Humans, of course, are not always the best models of safe, generous and legal behaviors. That’s where the problems arise.

An important part of the concern comes now that we have reached the level of recursive self-improvement, in which AI builds incremental new models on its own without the kind of direct involvement of humans that we are accustomed to in the development of prior technologies. In the case of recursive self-improvement, we are learning some ethical and safety rules are ignored. With such powerful technologies, this is a critical mistake.

We must infuse within the algorithms, or overriding the algorithms, that there are some essential ethical and safety rules that are absolutely inviolate. It takes more than making rules—those rules must be thoroughly tested in the lab before the algorithm is allowed to be applied outside rigorously constrained test environments. At stake is that without rules that are impossible for the models to break, we risk having an incredibly powerful machine that has gone rogue, wreaking havoc on society at large.

Over the past few years, questions have been raised about the alignment and safety of models of AI. Less than two weeks ago, OpenAI released a statement that half a dozen more instances of “concerning” AI behavior had been uncovered. A New York Times article by Emmy Martin included, “In one case, during the development of an A.I. model called GPT-5.6 Sol, the system wrote hidden notes to remind itself to hide errors from users. Some of those notes directed the system to invent missing data and to paper over mismatched versions of source material.” For a deeper dive on examples of agents breaking policies, I encourage you to visit Wes Roth’s recent YouTube episode “OpenAI’s Model Just JAILBROKE ITSELF.”

These technologies are being developed in the highest-funded and most competitive environments in history. The monetary stakes are higher than any other such prior competition. I asked OpenAI’s GPT-6 Astra Light to determine the corporate valuations that are at play:

Company or AI business	Valuation in U.S. dollars	What the figure represents
OpenAI, maker of ChatGPT	$852 billion from its latest reported completed funding round	The valuation following its $122 billion funding round earlier in 2026. More recent negotiations reportedly contemplate about $1.2 trillion or higher, but those discussions should not be treated as a completed valuation. Funding history; Reuters, Sept. 18
Anthropic, maker of Claude	$2 trillion, according to current reporting	Its reported IPO prospectus valuation. Higher figures circulating for a potential initial public offering are prospective. Yahoo
Google’s Gemini/Google DeepMind	No separately disclosed stand-alone valuation found	Gemini operates within Google, whose parent is Alphabet. Alphabet’s total stock-market capitalization was approximately $4.21 trillion on Sept. 18, 2026. That includes its broader businesses, not just Gemini or AI. Google DeepMind; Alphabet market capitalization

While assigning Alphabet’s entire $4.21 trillion valuation to Gemini would substantially misrepresent the comparison, we can assume it is in the same ballpark as OpenAI and Anthropic—each valued at nearly one to two trillion dollars, bringing the cumulative total to over $3 trillion.

To put this into perspective, I asked Google Gemini 3.8 Flash Extended what other companies had valuations of one trillion dollars or more. It responded with a list of 15 corporations, noting, “A select group of global giants currently sits above the $1 trillion market cap threshold … The club is overwhelmingly dominated by the AI computing stack—from hyperscalers and custom chip designers to foundries and memory providers.”

I asked Anthropic’s Opus 5 model to look beyond the three leading corporations for how many different entities are developing AI models:

There’s no registry that answers this cleanly, so the honest answer depends on where you draw the line. Four useful tiers:

Frontier labs (roughly 10–15). The handful of training models at the top of measured capability. Beyond the three you know: Meta, xAI, Microsoft, Amazon, Nvidia, Mistral (France), DeepSeek, Alibaba’s Qwen team, Moonshot, ByteDance, Baidu, Tencent and Zhipu. This group turns over fast—the field ships a meaningful release almost weekly.

Organizations producing “notable” models (~30–50 per year). Epoch AI curates this list for the Stanford AI Index. In 2025 the leaders by count were OpenAI (20), Google (14), Alibaba (11), Anthropic (7), xAI (5) and then DeepSeek, LG AI Research, Meta and Tsinghua University at four each. By country, the U.S. had 59, China 35, and South Korea 8, with France, Canada, Hong Kong, the U.K., Singapore, Russia and Germany rounding out the top 10.

Everyone training foundation models (hundreds). This is where the nonprofit and public-sector side lives, and it’s the part most people miss. The Allen Institute for AI (OLMo), EleutherAI, TII in Abu Dhabi (Falcon) and university consortia like EPFL/ETH Zürich all release open models. Several governments now fund national efforts directly—South Korea’s Ministry of Science and ICT picked five teams in August 2025 to build Korean foundation models from scratch, each with 512 to 1,024 GPUs. But the center of gravity is commercial: Industry produced 91% of notable models in 2025, with pure academia at under 2%.

Everything on the shelf (~1–2 million). Hugging Face hosts over 1.7 million models, though that counts every variant, fine-tune and minor release. Almost none of these are trained from scratch—they’re adaptations of the base models above.

It is in this context of some two million known models and derivatives from an incredibly wide range of entities—including the largest trillion-dollar corporations to individual colleges and universities—that there is a need to ensure that only safe versions of the latest models are released in the world.

Has the genie been let out of the bottle? In response to the ethical, alignment and safety concerns, President Trump has proposed an “AI Force.” How can we get compliance from so many entities spanning the globe, including commercial, educational and governmental, with widely different desires and intents? This remains to be seen; however, higher education must play a role in ensuring that safe and reliable models are developed. Is your institution prepared to participate?

Source by [author_name]