AI ‘content creators’ are getting harder to spot

This is The Stepback, a weekly newsletter breaking down one essential story from the tech world. For more on AI confusion, follow Robert Hart. The Stepback arrives in our subscribers’ inboxes at 8AM ET. Opt in for The Stepback here.

At first, AI influencers were relatively easy to identify — and to ignore. Aside from the occasional bursts of hype, they didn’t seem to change much about the way social media worked. The earliest virtual influencers — Lil Miquela with her blunt fringe and freckles, Imma with her bubblegum pink bob, and Shudu Gram with her flawless complexion — were obviously digital productions. Collaborations were announced with fanfare. Posts required studios, money, coordination, and a lot of polish.

Over time, I’ve noticed that the fake people on my timeline have started looking more and more like everyone else on it. Characters like Emily Pellegrini and Aitana Lopez moved a bit closer to reality — or at least to the reality of that well-traveled, well-off friend from college you didn’t keep in touch with, forever posting from nice restaurants and beautiful places, or from Coachella and Wimbledon. Not exactly relatable, but, then again, most professional influencers aren’t either.

Even then, many of these accounts aren’t standard ones by any means. Lopez is the product of a Spanish creative agency called The Clueless, which manages a stable of AI influencers. Pellegrini’s creator, who goes by the pseudonym Professor EP, told me he used to manage OnlyFans creators. Now he sells courses teaching people how to make AI influencers of their own.

Which is exactly what people are starting to do. A lot of people.

The novelty has worn off. Early AI influencers stood out because there were so few of them. Now they are part of a much larger mess of AI-generated content inundating social media: low-quality drivel lazily copied from chatbots, slop images and videos, and that catchy Lord of the Rings disco song that took over my TikTok for a month.

The fake people are now everywhere. They’re upselling drop-ship junk, scamming men out of money with fake photos, pushing disinformation and racist talking points, and catering to an increasingly weird, often sexual niche. Of course, there are a lot of thirst traps. There’s also a lot of mundane content, with avatars simply copying whatever’s popular among human creators, often just putting their fake faces on it.

That makes the scale of AI content creator influence hard to gauge. Platforms do not publish figures on how many of their users are fake people, and most AI avatars don’t become popular or influential enough to justify the kind of media attention the earlier wave received. Databases like Virtual Humans track hundreds of popular avatars, but those are only the accounts strange, weird, or big enough to get noticed. Below them is an ocean of accounts flying totally under the radar.

Part of the reason these accounts are able to avoid detection is that the technology used to make them has improved massively. A still image of a fake person can now be good enough to pass as genuine at a glance, especially in a feed filled with real influencers making generous use of staging, filters, and editing effects. Video and audio are quickly catching up, giving virtual people voices and movements that could fool undiscerning scrollers. The tools are no longer niche or prohibitively expensive, either. Mainstream products from companies like Google and OpenAI sit alongside specialized services from firms like Higgsfield, HeyGen, and ElevenLabs. With a little effort, almost anyone can make an AI influencer — or stable of them — without needing a studio, specialized equipment, or (much) money.

All this leaves social media platforms with a problem they do not seem especially interested in solving head-on. After several years of grappling with AI-generated images, videos, and audio, most major platforms now have some kind of policy covering synthetic media. But beyond requiring labels for AI-generated content, such rules often amount to little more than shoehorning the material into existing categories covering things like scams, spam, impersonation, and graphic material. AI people, especially those designed to behave like real people, don’t fit neatly into any of these buckets. They are not necessarily running a scam, posting graphic content, or impersonating someone — who would they even impersonate? And if they disclose that their posts are AI-generated, it’s not obvious what rules they’d be breaking.

For now, platforms seem content to live in ambiguity, neither fully welcoming nor shunning AI creators. They have cultivated a contradictory position, promoting AI as a creative tool while also trying to stop a tidal wave of slop from overwhelming their services. YouTube, TikTok, Instagram, and other platforms have developed rules for labeling synthetic media, particularly the realistic kind, while also promoting their own suites of AI tools, including some that can clone or simulate users. But those rules tend to focus on individual posts rather than the accounts and personas behind them, leaving AI influencers in a gray area.

In that uncertainty, the AI influencer ecosystem is thriving. Some market research firms estimate the virtual influencer market could be worth more than $60 billion by 2030, up from around $12 billion this year. Cultural clout is growing too. There are AI influencer awards, beauty pageants, dedicated talent agencies representing synthetic creators, and a booming market of synthetic creators selling courses and tools promising to help people make and run fake creators of their own, often with the promise of faceless passive income. Some of it has the faintly pyramidal smell of an online gold rush, a few visible success stories and an awful lot of people selling shovels.

My guess is that a reckoning is on the way. AI slop is already irritating, and there’s only so much of it a platform can carry until it is rendered practically unusable, especially given their persistent refusal to let users filter AI slop. Fake people pretending to be real are an even more intimate version of the same problem. But beyond labels and enforcement of existing rules, platforms mostly seem content to see what happens. To platforms, engagement is still engagement, whether it comes from a fake creator or a real one. So long as synthetic creators keep posting and don’t stray outside of existing rules, there seems to be little incentive to crack down.

There’s also a question of how sustainable the whole idea of having AI avatars running around online is. If so many are built just to make money from human users, what happens when the pool of human users dries up? There’s only so many people who will be willing to buy courses and tools to build influencers of their own, for example. That’s presuming social media can survive the influx of AI influencers. By definition, it requires some critical mass of humanity to keep things social. If left unchecked, networks will collapse under the weight of these fake people, as human users are inevitably driven away.

That could change if public anger keeps building. Backlash over deepfakes, impersonation, and synthetic spam is already forcing lawmakers and regulators to pay closer attention, particularly after incidents involving nonconsensual sexual deepfakes generated with tools like Grok. Europe’s AI Act could be a driver, at least as its transparency obligations for AI-generated content come into force. The regulations will require deployers of generative AI systems to clearly disclose AI-generated or manipulated content, which could pressure companies to step up flagging AI content or face potentially hefty fines. But even then, the focus is still largely on content, not whether the account posting it represents an actual person.

As with so much on social media, the burden falls back on users. Many platforms have effectively delegated the task of moderating AI content to users, relying on them to spot and report suspicious profiles. But self-moderation is a poor and unsustainable answer to something designed to evade notice. There is already a growing appetite for AI-free spaces. If platforms refuse to draw boundaries between real and unreal themselves, I expect users will draw them instead.