Scientists say AI is too easily fooled by 'alien hoaxes', and that's a problem for research

Researchers have detected problems with AI pattern matching in scientific data
It could mean false flags from life signatures on other planets.
AI can still be useful, but controls need to be built in

One of the ways AI can be most useful is by trawling through masses of scientific data that human researchers don’t have time to analyze, looking for patterns, but this use case is proving problematic when it comes to the search for life beyond our planet.

A new study by researchers at Michigan State University suggests that artificial intelligence systems can be too easily tricked into identifying signs of life in the universe where none exist. We need these flags to be accurate so we know where to point our telescopes next, so it’s important that the detection processes work.

The researchers created a digital simulation that included a key sign of life: the ability of molecules to replicate and mutate. Software was used to generate tens of thousands of digital organisms with and without this ability, which were then used to track a neural network to detect the difference with 99.7% accuracy.

Space and beyond

AI scans

A representation of the simulation in which the AI was tested (Image credit: University of Michigan)

It is worth keeping in mind the limitations of this research: these tests were carried out in an artificial digital simulation and were therefore not based on any real data. The researchers also deliberately looked for errors, rather than letting them occur by chance.

However, the study methods are robust enough to be concerning. The concern is that a Mars rover or deep-space telescope could identify a sign of life with a high degree of confidence, without necessarily having a human in the loop to verify it.

The researchers found that there were a large number of sequences that could also trip up the AI, meaning the risk of error is more likely. While the digital organisms incorrectly identified by the neural network were close to what it had been trained to detect, they were not complete matches, even though the AI thought they were.

These problems could also arise outside of space exploration. The same errors can appear when looking for patterns in medical scans, security camera footage, and anywhere else technology is used.

That said, the researchers want to emphasize that AI can still be useful in these scenarios; it just needs careful controls and supervision. “AI has an Achilles heel: it can see a pattern and completely misclassify it,” says Christoph Adami, one of the team members. “There needs to be a human being in the loop.”

Google logo on black background next to text that says