- New Runway study asks volunteers to identify AI videos
- Clips were correctly identified only 57.1% of the time.
- The videos were generated with the new Runway Gen-4.5 model
It seems like we’re through the looking glass when it comes to the verisimilitude of AI-generated videos: In a new test by AI video company Runway involving more than 1,000 participants, volunteers were only able to correctly identify AI or real videos 57.1% of the time.
Considering that guessing at random would get you to around 50%, that’s a worryingly low figure. Even Runaway co-founder and CTO Anastasis Germanidis admitted to failing “quite a bit” at the task (via The Information).
If members of Runway’s own team are struggling, then the rest of us don’t have much of a chance. The fake and real clips were set to the same resolution and length, and volunteers had 10 seconds to decide which ones showed real people in the real world and which were created by AI (reflecting the fast pace of social media scrolling).
The test was commissioned to mark the broader launch of Runway’s new Gen-4.5 model, which the company promises offers “unprecedented visual fidelity and creative control,” as well as results that are “cinematic and highly realistic.”
‘Have a more critical mindset’
Look
Participants involved in the test performed better when human faces, hands, or actions were involved, with accuracy rates ranging between 58 and 65% for these clips. Germanidis says that could be because these videos are where the “strange” elements are most noticeable, even if the AI videos themselves are still the same quality as the real ones.
Germanidis encourages people to “have a more critical mindset” when evaluating anything they see on the Internet. “At this point we are crossing the threshold and it is difficult to distinguish the generated videos from the real ones,” he told The Information.
Runway is working on ways to reliably flag AI output, and its videos already contain metadata that identifies them as AI-generated by default. It’s something governments are keen to see implemented, understandably, as the line between real and computer-generated disappears entirely.
However, Runway Gen-4.5 is still not perfect. Runway says it may still have issues with disappearing objects, causal reasoning (so doors can open before the handle is pulled), and success bias (so misdirected kicks still hit the net in soccer scenes, for example).
Follow TechRadar on Google News and add us as a preferred source to receive news, reviews and opinions from our experts in your feeds. Be sure to click the Follow button!
And of course you can also follow TechRadar on TikTok for news, reviews, unboxings in video form and receive regular updates from us on WhatsApp also.
The best business laptops for every budget
