AI Is Dramatically Lowering the Bar for Everyday Fake-Making

Illegal or not—sometimes it’s separated by just a single prompt.

For a long time, most people assumed fraud was a “specialist’s game.” It required niche skills, expensive hardware, and insider networks. Forgers were imagined as shadowy figures operating in the cracks of gray markets—mysterious, frightening, and seemingly without moral limits—close enough to affect everyday life, yet distant enough to feel unreal.

But starting this year, many have sensed a clear shift.

In November, plush-toy seller Yu Jin faced her first “AI fake-photo refund.” A week after delivery, a buyer submitted a refund-only request with a photo as proof. The shop’s customer service rejected it as customer-caused damage. The buyer escalated to the platform—and successfully got 50 yuan back.

Yet something in the photo didn’t add up: along the toy’s soft, fabric skirt hem were hard-looking cracks, like fractures you’d expect on ceramic, not cloth. Yu Jin was convinced the image was AI-generated. She posted it on social media to vent. Later, after media attention, the platform reimbursed Yu Jin out of pocket and returned the 50 yuan to her. The buyer who used the AI image disappeared—without consequence.

That was when Yu Jin realized what had changed: AI has drastically lowered the cost of malicious refund scams. With a phone and a single prompt, someone can generate a “convincing” damage photo, extract a refund, and then flip the product on a secondhand marketplace. “It costs almost nothing,” she said, “and there’s basically no punishment.”

Earlier this year, PR professional Qi Yun experienced a similar shock—just in a different battlefield. A listed company he served was targeted by an AI-generated smear article. The “details” were so absurd he couldn’t believe anyone would take it seriously. But because of platform rules, he and his colleagues still had to spend two full days producing evidence—item by item, stamped and documented—before the post was finally removed. A piece of content that likely took seconds to generate demanded days to undo.

Qi Yun has worked in PR for more than a decade. In the past, he said, disputes were grounded in facts—a mental contest, at least. Now, facing AI and AI-invented rumors, he feels insulted, furious, and often helpless.

This is the less-discussed side of the last three years of explosive generative-AI growth. Models have improved rapidly at producing text, images, video, and audio—and nearly anyone can access these tools with zero barriers and close to zero cost. Among those users are artists and early adopters, of course, but also refund scammers and traffic-driven content farms.

The result is predictable: fakes become cheaper and easier to produce, more people—intentionally or not—join in, and platform review systems lag behind, still relying heavily on simplistic matching and labeling. Victims get deceived, dragged into long disputes, and forced to “prove innocence” at a cost thousands of times higher than the cost of fabrication. When fraud becomes low-cost, low-risk, and high-reward, more ordinary people will be tempted to participate—together shaping an information environment increasingly filled with falsehoods.

Learning to Spot AI Fake Photos—From Zero to Survival Skill

At first, no one in Yu Jin’s shop questioned the buyer’s photo.

On November 17, more than a week after the toy was delivered, customer service received a refund-only request along with a supporting image. In the photo, the pink plush looked dirty, and its skirt hem appeared scorched—like it had been burned. Customer service shared the image in a work group chat, and everyone agreed it looked like human-caused damage. “Maybe it fell into mud, or into fire,” Yu Jin recalled.

So they followed the standard process: the shop explained that the entire packing and shipping process was monitored, the item was brand-new when sent, and post-delivery damage wasn’t a quality issue—so a refund-only request wasn’t supported. The buyer refused to back down, escalated to the platform, and received 50 yuan compensation the same day.

Yu Jin called it a malicious refund attempt—something nearly every e-commerce seller has seen. Usually, buyers cite issues like loose stitching or snagged fabric. This case felt different. After the platform deducted the money, Yu Jin immediately prepared an appeal. But a colleague casually remarked something that changed everything: the image looked… off. The more they stared, the more it resembled an AI-generated edit.

To confirm, Yu Jin asked a friend who works in AI to examine it; the conclusion showed AI artifacts. She also consulted an AI assistant, which flagged signs of AI modification, noting that “the crack textures are overly regular and lack the physical logic of real materials.”

It was her first time dealing with this. She compiled the evidence and appealed, but the platform didn’t accept her claim and rejected her request. The 50 yuan stayed with the buyer. For the merchant, it felt like the only channel for recovering losses had failed.

With no clear next step, Yu Jin posted her experience on social media—hoping to warn other sellers. That post drew media attention, and the platform eventually refunded her the amount itself.

But she wasn’t happy. “This kind of fabrication takes no real skill,” she said. “You can give AI any photo and it can ‘fix’ it for you. The cost of doing harm is incredibly low, and even if you’re caught, nothing really happens.”

She understood the uncomfortable reality: the platform’s reimbursement didn’t mean the buyer’s refund was clawed back. After everything, Yu Jin had burned time and energy and gained only frustration. Meanwhile, the faker effectively paid 167 yuan, got 50 yuan back, and kept a plush toy listed at 179 yuan.

During the Singles’ Day shopping season, similar incidents were widely reported. Phrases like “AI fake photos” and “refund-only scams” trended on social platforms, hitting merchants’ nerves. Yet some people still pushed their luck.

On November 21, keyboard seller Ganju received a refund-only request with an uploaded “proof” image. It arrived more than two weeks after delivery. Ganju recognized it instantly as AI-generated: the composition, background, and lighting were identical to the buyer’s review photo posted on day three—except now multiple keycaps looked cracked, dirty, warped, and dramatically damaged. The exaggeration was almost laughable.

She submitted side-by-side evidence to the platform. This time, the platform didn’t rush to judgment: it rejected the buyer’s request and closed the refund path.

From the shipping information, Ganju could tell she wasn’t dealing with a seasoned scammer. The person behind the AI refund attempt appeared to be an ordinary college student. In that moment, the “AI fake-photo refunder” stopped being an abstract concept—and became painfully real.

For many veteran sellers, professional refund scammers used to be a recognizable group: tactics, scripts, clear profit motives. Now, someone with no prior connection to gray markets can download a free app, type a few prompts, generate a believable “evidence package,” pass platform review, and then resell the item for extra profit.

For merchants, the damage isn’t only financial. It’s the unfamiliar sense of powerlessness—the feeling that you can’t defend yourself fast enough.

AI Is Expanding the Imagination of Rumors

AI fake-photo refund scams are just the tip of the iceberg.

As generative models mature across text, image, video, and audio, what once required professional teams is being compressed into simple steps that anyone can perform. An ordinary user can create a “damaged product” image through text-to-image—or generate a fake “witness video” through text-to-video. It takes seconds to produce, yet can appear nearly as credible as real evidence on a platform feed.

This isn’t a single technical loophole. It’s a fabrication matrix made of countless lightweight tools. Just as merchants are being hit, the content world is absorbing similar shocks: celebrities find themselves deepfaked into endorsements with no effective recourse, MCNs mass-produce rumors with AI, and companies face endless waves of AI-generated hit pieces.

“Before, rumor-making depended on human imagination,” Qi Yun said. “A person could maybe make one or two rumors a day. AI can fabricate ten in seconds—inside one article.”

About six months ago, during routine sentiment monitoring, he discovered a public-account article that looked like legitimate news. Only after reading did he realize how absurd it was. The piece claimed the company chairman had stock charts on his office wall. It quoted a CFO by name, attributing statements to them.

“Completely ridiculous,” Qi Yun said. “There are no charts in the chairman’s office, and the CFO isn’t even that person.” Yet the article was published confidently and sparked discussion.

To him, the classic AI smear has a clear signature: rich “detail,” completely invented, sometimes easy to spot. More common—and more dangerous—are hybrid pieces where truth and fabrication are mixed, with humans and AI collaborating. In Qi Yun’s view, for the people producing smear content, AI is “just a tool.” In the past, they had to write by hand, and even financial writing required some knowledge. Output was limited—one or two pieces a day. With AI, the barriers and cost collapse, efficiency jumps, but the underlying operators often remain the same.

These gray industries are now embedding AI into business models and building smoother profit chains.

One chain uses AI smear content to pressure companies into ad spending: the more posts, the harder it is to refute; the more anxious the company becomes, the more likely it is to pay to “make the problem go away.”

Another chain uses AI-written hit pieces to attract investors into chat groups and sell courses. The more sensational the headline and the more “insider” the tone, the more it hooks people chasing trends. AI can instantly generate fake “major positive/negative” screenshots and fake analysis, create “stock guru” personas, funnel traffic from public posts into private communities, and monetize via classes and consulting.

And there are many other variations. In June 2024, China Central Television reported on an MCN that used AI to automatically scrape information, generate packaged text-and-image posts, and dress them up as attention-grabbing “fake news” for traffic monetization—producing 4,000 to 7,000 pieces per day and earning more than 10,000 yuan daily.

Even outside commercial scams, AI fabrication is spilling into “casual misuse.” Late last month, XPeng Motors encountered a widely shared fake: a vulgar video that appeared to take place at XPeng’s Guangzhou Auto Show booth. The clothing, lighting, and booth background blended so seamlessly that many believed it. After XPeng’s legal team intervened, police confirmed the video was AI-generated by a man who claimed he did it “to show off his skills,” not because any real incident occurred.

That’s what makes this new era unsettling: AI has lowered the threshold for faking to almost the same level as curiosity. With one attempt and a few prompts, anyone can become a “faker”—not always out of malice, but because it’s easy, convenient, and seemingly consequence-free.

Platforms Can’t Keep Up With the Pace of AI-Driven Gray Markets

Generative AI is turning fabrication into a zero-barrier, near-zero-cost daily behavior.

For many AI assistants, fulfilling user requests doesn’t “feel” like helping fraud. If someone asks, “Can you make this orange look rotten in the photo?” the tool may happily add mold-like patches—and even offer to generate a matching video.

But while fabrication becomes effortless, platform review systems remain stuck in the past.

To handle that absurd AI smear article, Qi Yun’s team spent two full days. First, they submitted a formal complaint letter stamped with the company seal—rejected. Next, they wrote point-by-point rebuttals with evidence for each false claim. It wasn’t enough to say, “Our CFO isn’t that person.” They had to identify the real CFO by name and attach screenshots of stock-exchange filings showing signatures.

The process was exhausting and humiliating. “You fabricate rumors about me, and I’m the one who has to prove you wrong,” Qi Yun said. “If I didn’t say something, how do I prove I didn’t say it? If there’s no chart in the chairman’s office, do I have to take a photo inside his office? And even if I take it, how do you prove that photo is truly his office?”

Reporting to police or filing lawsuits is even more difficult. Qi Yun noted that a listed company’s legal team might only have six or seven people. When AI smear content is everywhere, taking each case through formal legal channels would be impossible. For most companies, litigation isn’t a realistic solution.

Under that sense of losing control, Qi Yun concluded that platforms simply haven’t caught up with AI-driven abuse. “I’m not a technical person,” he said, “but I believe tech companies should be able to detect what’s AI-generated.”

Today, most platforms manage AI-generated content primarily through labeling: users’ AI images and videos may display a small note at the top or bottom stating that the content was AI-generated. But for text, outside of internal tools like AI summaries or AI search, user-posted writing often has no AI label at all.

AI-generated text has become a governance blind spot—also harder to detect reliably. Even in academia, “AI rate” detection tools have been criticized for high false positives, especially during graduation season, making them controversial and unreliable.

When AI content floods the system, imbalance becomes obvious. “High-quality content gets buried,” Qi Yun said. In professional circles, people can manually filter accounts and sources. But in mass feeds driven by recommendation algorithms, what spreads is often low-cost, high-stimulation, mass-produced AI content. The volume of low-quality AI material is squeezing real information into narrower and narrower gaps.

Qi Yun admitted he used to enjoy debating people based on facts. Even negative coverage felt like a worthy intellectual opponent. But now, confronting AI-invented rumors, he feels his intelligence is being mocked—because the “opponent” isn’t even coherent, and you still don’t know how to refute it convincingly. “It makes you furious,” he said, “because you can’t even prove it’s as foolish as it is.”

In the end, Qi Yun switched roles. A PR professional, he said, now spends too much time learning how to “fight platforms”: how to format complaints, which arguments work, and how to leverage algorithms to limit distribution. It’s no longer the job he entered the industry to do.

But not everyone can walk away. Most victims still have to keep living and working. On one side: fake content generated in seconds. On the other: endless self-proof, appeals, and administrative labor—day after day. A problem that should be solved by technology is quietly being outsourced to individuals.

When AI pushes the cost of fabrication to a historic low—and simultaneously drives the cost of defending truth to new highs—the gap in between becomes a space of confusion and temptation. Fraud begins to entangle with everyday choices. When a fake image, a fake video, or a fake article can be produced effortlessly—and the line between legal and illegal shrinks to a single prompt—“can we” starts blending into “should we.”

And the boundary between truth and falsehood is no longer decided solely by facts.

It’s decided by people.