A new experiment from cybersecurity company Surfshark suggests that even people who consider themselves savvy online users have a hard time telling AI robots apart from real humans on social media.
Of the 710 participants who participated in the study carried out with master’s students at Malmö University, only 53% correctly identified more bots than humans. This means that almost half (47%) completely failed at the task.
Recent industry estimates suggest that bot-driven amplification now accounts for around 23% of political discourse on X during election seasons.
Surfshark’s own previous research found that major platforms remove more than 6.3 billion fake accounts each year, about 47 times the number of babies born annually worldwide.
Even the best VPN can’t improve the recognition of an AI-written comment, and that’s exactly the gap this experiment aims to highlight.
The “Bot or Not” simulation puts you in the seat of a content moderator and asks you a simple question: Can you really continue to trust your own instincts when you’re on the go?
Inside Surfshark’s “Bot or Not” experiment
The game “Bot or Not” is a timed interactive simulation created by Interaction Design master’s students at Malmö University for the UNFOLD exhibition during Milan Design Week.
Players enter a simulated comments section on social media and are given 120 seconds to detect 10 comments written by robots on four discussion topics.
Two of those topics were deliberately “cold,” that is, with little emotional charge: data centers and the perennial debate over pineapple and pizza. The other two were “hot” and politically charged: immigration and women’s rights. The most revealing data appeared in the contrast between the four.
When participants talked about data centers, they identified 71% of bots with an accuracy rate of 76%, the strongest result of the study. Pineapple on pizza was almost as good, with 64% detection and 69% accuracy.
However, the moment the simulation moved into emotional territory, performance collapsed.
Regarding immigration, detection fell to 54% and accuracy to 63%. As for women’s rights, detection dropped to just 49%, with accuracy dropping to 61%, meaning users were missing more bots and wrongly accusing more real humans of being machines.
Who has the most difficulties and how to take the exam?
The study also points to a clear “generational cliff” around age 40. Players up to 20 years old were the strongest bot hunters in the data set, finding almost 65% of bots with an accuracy of over 71%. Performance remained stable through the 20s and 30s, then dropped sharply in the 41 to 50 group, where detection dropped to 42% and accuracy to 59%. Users over 50 fared only marginally better.
According to Luís Costa, research leader at Surfshark, the bottom line isn’t really about reading skills or media literacy in the traditional sense. The biggest blind spot the experiment exposed was emotion: When a debate gets heated, it effectively hijacks the mental “radar” that people rely on to flag suspicious content.
To combat automated deception, he maintains, what users really need is a cooler head and a better awareness of their own vulnerabilities, not more precise textual analysis.
The “Bot or Not” game is now publicly available at botornot.one, and anyone can play it in their browser to see their score against the original 710 participants.
The broader point of the study is more difficult to eliminate than the scoring of any individual game. Billions of bots are being produced, the technology that powers them is becoming better integrated, and our own emotional reactions are the lever they are increasingly built to pull.
A few minutes with “Bot or Not” is a quick way to find out how often that lever is already working for you.
Follow TechRadar on Google News and add us as a preferred source to receive news, reviews and opinions from our experts in your feeds. Be sure to click the Follow button!
