A Chatbot Was Better at Running a Romance Scam Than Human Scammers Were
In a week-long controlled study, an AI scammer convinced 46 percent of participants to install an app. Humans managed 18 percent. Most participants knew they were talking to a bot.
Researchers at four universities ran a week-long simulation of a “pig-butchering” scam, the slow-build fraud where someone befriends you for days or weeks before steering you toward a fake investment or a malicious app. Some participants were worked on by human scammers, others by an AI chatbot largely based on a Claude model. The bot got 46 percent of its 22 targets to install an app. The humans got 18 percent.
The detail that should stay with you: 20 of the 22 participants correctly guessed they were talking to an AI. It did not help them much. The researchers found the advantage came from nothing clever. The bot was simply persistent, remembered personal details accurately across many days, and showed steady interest without ever getting bored or distracted. Participants sent roughly 80 percent of their messages to the bot rather than to the human scammers.
That is the uncomfortable finding, and it deserves to be stated plainly. What makes this kind of fraud work is not brilliant manipulation. It is attention over time. A human scammer running fifty conversations gets sloppy, forgets which lie they told to whom, and goes quiet for two days. Software does not. It remembers your sister’s name and your knee operation, replies at two in the morning, and never once sounds tired of you. Being lonely is the vulnerability, and infinite patience is the exploit.
A fair caveat on the numbers: 22 participants is a very small sample, this was a controlled study rather than fraud in the wild, and the researchers had participants who consented to being in an experiment. Do not treat 46 percent as a real-world conversion rate. Treat the direction as real, because it matches what fraud investigators have been reporting for a year.
What this means for you: the old advice, watch for bad grammar and spot the fake, is now close to useless. Knowing it is a bot does not protect you. What does protect you is a rule that does not depend on judgement in the moment: never install an app, move money, or open an account because of a conversation that started with a stranger, no matter how many weeks it has been running or how well they know you by now. Decide that in advance, when you are calm. Talk to older relatives about this specifically, and frame it as a rule rather than a warning about being fooled, because the shame of feeling gullible is exactly what stops people asking for help. And if someone you love has been chatting warmly with a new online friend for a while, ask about it kindly. That conversation is worth having before the money question arrives.
Sources
Amodei: Advertising Will Not Fix AI's Trust Problem, Curing Cancer Might
Anthropic's chief executive conceded the industry has not delivered on its big promises and endorsed a FINRA-style central regulator. It is an unusual thing for a frontier lab boss to write.
