The “Nonexistent” Research on AI’s Benefits for Education

The introduction of artificial intelligence–powered learning tools is key to maintaining higher education’s value—at least according to the tech industry and a growing chorus of university leaders.

The companies jockeying for multimillion-dollar university contracts say giving students access to such tools is essential for fostering deep learning and preparing them for a labor market increasingly shaped by AI-induced disruptions. 

“To thrive in this Intelligence Age, students need to build agency: the ability to learn continuously, solve hard problems, and create new economic opportunities for themselves with AI,” reads a recent memo from OpenAI about why its ChatGPT Edu—which allows students, faculty, researchers and administrators to use its large language model within a closed, secure system—is one of the “necessary tools” for equipping institutions “to meet this moment.”

And many colleges and universities have taken OpenAI and other tech companies at their word.

OpenAI—which debuted ChatGPT, the first mainstream large language model, in late 2022—has sold at least 700,000 licenses to dozens of higher learning institutions, including the California State University system, Arizona State University and the University of Maine system. Numerous other universities have inked deals with Anthropic and Google, which market their education-specific LLMs as “a thinking partner,” time-saver and tutor. And many smaller education-technology companies have also given their products an AI makeover.

Scores of university leaders have also bought in to the idea that embracing these tools is the only path forward. “In a world that promises to be defined by AI, universities that fail to produce graduates who can use it productively will consign themselves to irrelevance,” Dartmouth College President Sian Leah Beilock wrote in The Atlantic this month, calling for “more AI at Dartmouth.”

But numerous experts are urging caution, arguing that there’s not enough independent research on the efficacy of AI as a teaching and learning tool to guarantee big returns on these increasingly common investments.

“The evidence base is almost nonexistent,” Justin Reich, a digital media professor and director of the Teaching Systems Lab at the Massachusetts Institute of Technology, told Inside Higher Ed. “Building products and testing them rigorously takes a really long time, and there’s hardly any funding to do it.”

Research Lag

A handful of small, scattered studies from around the globe show a mix of promising and concerning results regarding AI’s impact on teaching and learning. But a large-scale, randomized controlled trial would be the gold standard for determining how AI influences learning outcomes, and those aren’t the types of studies the federal government—which last year gutted the Institute for Education Sciences—or philanthropists typically fund. Even if they supported such an undertaking, “one well-designed randomized controlled trial could still be wrong, or it could be correct in its own context but not generalize well to other contexts,” Reich said, noting that researchers have only recently developed sound best practices to guide students’ internet research. “It might similarly take us decades to come up with a good plan for teaching students how to use LLMs.”

In the meantime, colleges and universities must weigh the implications of embracing AI in the classroom on their own.

“Schools need to understand that big science isn’t going to have good answers for a long time,” said Reich, whose research focuses on the limitation of technology to transform education. “Instead, they have to move forward with little science and realize that the claims from these companies shouldn’t be trusted. Any time they hear something that sounds like a best practice, it has to be treated as just a hypothesis.”

While higher education institutions have long taken chances on new innovations—the internet chief among them—the rapid evolution of AI-powered tools presents additional challenges to testing such hypotheses, said Stacey Alicea, executive director of the Research Partnership for Professional Learning.

“AI models were changing every six months; now they’re changing every one to three months. Baseline data in these LLMs are changing weekly, and traditional research models cannot keep up,” she told Inside Higher Ed. “By the time researchers have a finding, it’s no longer relevant or generalizable.”

That’s why Alicea and her colleagues are advocating for studying AI’s efficacy as a teaching and learning tool through smaller, faster studies that monitor and evaluate the tools as they are implemented, though, she admits, “Our systems are not set up to do that right now.”

Study AI Features, Not Tools

Another downside of the existing research on AI in the classroom is that most studies focus on specific tools. For instance, after the CSU system signed a $17 million contract with OpenAI in 2025, it surveyed close to 100,000 faculty and students about which tools they use most often and why; most of them said ChatGPT.

“These tools are being treated the way an intervention would be treated in research, asking if a specific tool works,” Alicea said. “But if we’re seeing over 1,000 tools on the market for students right now, there’s no way we’re going to run 1,000 studies on all of them at a time when education funding is getting cut and we have tons of privacy concerns about student data.” Instead of studying individual tools, “we should be studying the features across tools,” she said. “And it’s not just the tools themselves—it’s how they interact with all of the other tools teachers and students are using.”

That’s not the only barrier to assessing AI’s usefulness in the classroom.

“Can you get students to use AI tools consistently enough to even test whether or not they’re effective?” said Carly Robinson, director of research at Stanford University’s Systems Change Advancing Learning and Equity initiative, which tracks emerging research on AI and education. “The design has to be really deliberate just to get students to engage with these tools in the first place.”

Although high-quality causal research is slow going, the limited findings so far support a measured approach to AI adoption in higher education.

“There is increasing evidence that AI has the potential to benefit learning, but using it on its own without intentional design or guardrails probably reduces learning through cognitive offloading,” Robinson said. “There’s logic to the idea that AI can improve learning and potential for it to be transformational in education, but we’re a ways off from that.”

And when substantial research finally does materialize, it will only raise more nuanced questions about the best uses of AI, depending on the type of material students are trying to learn.

“We can ask questions about how AI impacts critical thinking or problem-solving, but the applications of those skills aren’t identical across disciplines, even if there are some commonalities,” said Claire Baytas, a program manager at Ithaka S+R whose work focuses on generative AI’s impact on teaching, learning and research. “Different disciplines have different cultures and attitudes toward AI, which will need to be dealt with,” she added. “There are instructors who are innovating, but there are also some who are still trying to gather basic AI literacy and others who are ethically and morally opposed to it. That varied landscape may also change the way the research shakes out.”

For now, though, “universities need to be very careful about what the research on AI is telling them,” Patrick O’Neill, associate professor of life and physical sciences at Ivy Tech Community College in Indiana, told Inside Higher Ed. “There’s such an incredible appetite for the message that AI is effective in education that a lot of people just assume that’s going to be true.”

But according to preliminary findings he published last month, several of the existing peer-reviewed studies that say AI is an effective learning tool are based on flawed methodologies and analyses that rely on misapplied, miscalculated or misinterpreted statistics. Meanwhile, a series of “methodologically effective” studies indicate that “having access to LLMs without guidance is counter-productive” to building critical thinking skills.

And that reality is something universities should keep in mind as they push students and faculty to embrace AI without robust data to steer it.

“Universities are spinning hard to make it sound like they have control of AI, but I don’t think they do,” said O’Neill, who’s heard from frustrated employers that some recent graduates can’t complete basic functions without leaning on an LLM. “Colleges need to do a better job of getting control of how assessments should work in a world with AI.”