- Most companies struggle to implement agent AI effectively, and legacy infrastructure is one of the key reasons, report says
- Google Surveyed IT Leaders and 83% Said Infrastructure Upgrades Are Needed
- IT leaders are also concerned about the hidden costs of agent AI, such as increased power consumption and operational complexity.
If there was a single message to take away from this article, it is that the infrastructure that all businesses depend on today was not built to handle agent AI.
Google surveyed more than 1,400 senior IT leaders about their AI ambitions and found that 83% of organizations say they need infrastructure upgrades to realize the full benefits of production-grade agent AI.
Additionally, many respondents also see unexpected costs arise when trying to run agent AI on legacy infrastructure. 62% said they had seen a significant inference tax driven by data egress fees, excess storage and idle specialized hardware, along with 82% who said scaling AI introduces hidden costs of operational complexity. 79% also cite security, governance, and MLOps as a key barrier to scaling agent AI.
Updates are needed to get the full benefit of Agent AI
To combat these limitations, Google has several recommendations for organizations crippled by legacy infrastructure.
Leveraging fluid computing “to dynamically match the right silicon to the right task while minimizing operational overhead” is Google’s first recommendation, providing computing power for agent AI tasks without reducing capacity for general workloads, avoiding the need for excessive memory usage to run agent workloads that use large context windows.
For those struggling with agent dispersion caused by a cascade of new tasks across platforms and teams, Google recommends making use of enterprise-grade governance tools, which are typically available through cloud partners that businesses are already using. Google offers its own platform, Agent Gateway, as an example of a solution that provides visibility and oversight into how agents communicate, the data they access, and their workloads.
Organizing data more effectively prevents AI agents from using more compute when running heavy queries in an attempt to access isolated data. Organizations looking to improve the efficiency of agent AI should work to use a unified data layer that automatically annotates unstructured data, allowing agents to understand where the data is without having to navigate channels. An additional benefit of using a unified data layer is that it helps avoid data duplication, saving additional costs of excessive storage in the long term.
Moving your AI to the edge (deploying agents directly to the site they use most) is an additional recommendation that organizations are actively pursuing. 90% of organizations surveyed by Google said this was a consideration in their AI initiatives. By deploying on-premise agents in manufacturing plants, retail stores, or hospitals, agents benefit from reduced latency, increased resiliency (in the event of a centralized cloud outage), and increased profitability by reducing costs per token with highly optimized on-premises models.
As with businesses of all sizes, energy costs are a key consideration. When selecting new hardware, 91% of leaders now consider power consumption as a factor, especially when navigating power availability in regions without expanding capacity, regulatory compliance, and reducing the cost of ownership of AI systems.
Follow TechRadar on Google News and add us as a preferred source to receive news, reviews and opinions from our experts in your feeds.
