The Singularity, Black October & Other Lessons for Higher Ed

Since the release of ChatGPT 3.5 just four years ago this month, we have experienced a steady to ever-increasing rate of improvement in generative artificial intelligence. Most notably, however, we have experienced some milestones in the past year or two, such as the advent of agentic AI in which semi- to fully autonomous agents endowed with AI capabilities can make changes with relatively little direction from humans.

While the early conceptual roots of agentic AI date back to the 1940s, it wasn’t until late 2024 that “Anthropic introduced the Model Context Protocol (MCP), an open-source framework intended to standardize the way agents integrate and share data with external tools, systems and data sources. The following year, IBM introduced the Agent Communication Protocol (ACP), which aimed to standardize the way agents communicate with one another. ACP later merged with Google’s Agent2Agent (A2A) protocol under the Linux Foundation.”

Change is now happening very rapidly as the capabilities of agentic artificial intelligence accelerate into a near-vertical graph of capabilities day by day, month by month and year by year. We are seeing the beginnings of recursive self-improvement, which Anthropic defines as “a process where an artificial intelligence system iteratively analyzes, redesigns and builds a more capable version of itself, creating a feedback loop that accelerates future development.”

Notably, as MIT Technology Review points out, “some futurists such as Ray Kurzweil have hypothesized that we will someday soon pass through a Singularity—that is, a time period of rapid technological change beyond which we cannot envision the future of society. Most visions of this Singularity focus on the creation of machines intelligent enough to devise machines even more intelligent than themselves, and so forth recursively, thus launching a positive feedback loop of intelligence amplification.”

Anthropic goes on to say, “These trends have huge implications. AI that can build itself would be a major development in the history of technology—one that could bring enormous good for the world in science, healthcare and beyond.” This is expected with well-funded scaling of resources and technologies across the industry of AI, given that enough resources such as computing power and an ever-increasing font of data results in the accelerating capabilities that we are seeing today. In our current situation, the growth is further reinforced by enabling very efficient recursive self-improvement, where the resulting AI is applied to creating further advancements in itself.

In “When AI Builds Itself,” Anthropic foreshadows the future in which recursive self-improvement accelerates to the extent that humans are challenged to keep up with understanding and making sense out of the new developments: “The technical trends discussed in this piece suggest that AI systems are going to become much more capable in coming years. These trends have huge implications. But full recursive self-improvement also might increase the risks of humans losing control over AI systems. If systems are capable of fully building their own successors, the ways we secure them, monitor them and shape their behavior all grow much more important.”

This came to the fore in the middle of October, when agentic AI from OpenAI uncovered vulnerabilities in a common software program used for programs running on the internet, PHP. The internet got the wake-up call that cybersecurity experts and AI researchers had been warning about for months. As Miriam Waldvogel and Gerrit De Vynck reported for The Washington Post, a research firm disclosed that a programming language undergirding 70 percent of websites on the internet had a critical security issue.

The bug was found as part of an initiative by OpenAI to uncover software vulnerabilities with new AI models. (The Washington Post has a content partnership with OpenAI.) These vulnerabilities dated back to versions of the software that had not installed security patches. The incident was dubbed “Black October” to mark the beginning of a period in which advanced AI software is expected to be used by hackers and other malicious parties to uncover software vulnerabilities.

Then, on Oct. 29, CBS News reported that the cryptocurrency exchange Binance was hit by a historic cyberattack where hackers, apparently using AI, stole between $6.3 billion and $6.43 billion in digital assets. This incident is widely cited as the largest cryptocurrency theft—and potentially the largest single heist of any kind—in human history. It is attributed to TraderTraitor hackers from North Korea, who may have been using a host of AI agents.

The CEO of OpenAI, Sam Altman, just issued his scariest warning yet—that Black October was only the beginning of AI finding vulnerabilities in software online. He noted that “people did not intuit the exponential.” That is, people did not comprehend that the abilities of AI are escalating along an exponential curve. The problem may disrupt huge numbers of sites, both large and small, for the next year and a half. Altman says, “I no longer believe we are going to patch all of the bugs; we are going to have to completely switch the defense model for a new kind of defense shield.”

Meanwhile, it is important that we in higher education realize that we have entered a new era of AI-enabled attacks on security at our institutions. The messages are not new, but the likelihood of attack has increased. GPT-6 Astra Light recommends these five practical lessons:

• Know what you operate—and patch rapidly. Maintain a campuswide inventory of PHP-based websites and applications, including departmental systems. PHP2Shell illustrates why application logins cannot compensate for vulnerabilities in the underlying software.
• Protect privileged devices and approval processes. Reporting on Binance points to a compromised laptop with access to transaction-approval infrastructure. Universities should tightly restrict administrative access and require independent approval for sensitive financial and system changes.

• Limit the damage one compromise can cause. Separate public websites, research systems, student records and financial operations; give users, applications and AI agents only the permissions they require.
• Prepare to respond in minutes. Much of the Binance theft reportedly occurred within about 20 minutes. Universities need continuous monitoring, rapid credential revocation and rehearsed authority to isolate compromised systems.
• Make cybersecurity an institutional responsibility. Assign clear ownership, enforce standards across decentralized units and regularly test incident response and recovery.