- Websites risk losing position and brand authority if they don’t leverage AI
- AI-based discovery may not even detect a user navigating to the actual website
- The solution is greater governance over outright bans
A new Hostinger study of more than 66 billion bot requests across five million websites has revealed how ingrained AI has become in our search habits and how important it is for businesses to incorporate optimizations beyond typical SEO.
The report found that companies that block AI robots to protect their intellectual property may be losing control over how AI assistants interpret their content, as well as weakening their brand position.
Hostinger explained the shift in Internet discovery, where AI assistants summarize and recommend products and services without users necessarily navigating to the website.
Is AI replacing traditional search?
“Across the 66.7 billion records, one message stands out: AI trackers are rapidly increasing their reach, even as AI training bots face growing resistance from content creators,” Hostinger wrote.
There is a clear reluctance to accept training robots, and coverage of the GPTBot website fell from 84% in August to 12% in November. Meta’s ExternalAgent also fell from 60% to 41%. On the other hand, OpenAI’s SearchBot and Applebot crawlers increased their coverage.
Despite the growing activity of AI crawlers, traditional search crawlers have remained fairly stable, leading Hostinger to assume that AI is “adding a new decision layer on top.” [traditional search]” rather than replacing it: good news for Google, whose discovery spans both traditional search and AI chatbots.
For publishers and content sites, Hostinger explained that website owners may want to increase the visibility of the AI assistant’s response by using tools like Web2Agent and llms.txt files.
“The real risk for businesses is not access to AI itself, but losing control over how pricing, positioning and value are presented when decisions are made,” added AI Director Tomas Rasymas.
Looking ahead, the report calls for better governance around outright bans, so that website owners ensure they are discovered on emerging AI channels without losing brand position.
Follow TechRadar on Google News and add us as a preferred source to receive news, reviews and opinions from our experts in your feeds. Be sure to click the Follow button!
And of course you can also follow TechRadar on TikTok for news, reviews, unboxings in video form and receive regular updates from us on WhatsApp also.
