- AI systems are now designing and refining other AI systems independently
- Human understanding of AI is shrinking as AI’s understanding of humans grows
- AI Systems Can Model Human Fear, Uncertainty, and the Need to Belong
Microsoft chief scientific officer Eric Horvitz and EPFL researcher Robert West have issued a stark warning about how well we really understand AI.
Both have argued that artificial intelligence tools are now advancing fast enough to surpass our understanding of how these systems actually work.
At the same time, they point out something disturbing: AI’s understanding of human behavior continues to grow, while ours does not.
AI complexity is accelerating faster than human understanding
Their concern is not that we need to understand every line of code or every parameter buried within these systems.
What matters, they say, is having enough knowledge to maintain meaningful oversight. Even a partial understanding, they argue, can be genuinely useful, especially when it helps detect risks early, before they are too entrenched to undo.
One challenge they point out is how frequently AI tools are now used to design and improve other AI systems.
As these recursive development cycles become more common, performance can improve while human knowledge of the underlying processes becomes increasingly limited.
“AI systems are now designed and refined by AI systems through recursive cycles that can surpass human understanding and develop in high-dimensional spaces that resist intuition,” Horvitz and West wrote.
This is a form of operational opacity, where results remain visible even when the mechanisms that produce them become more difficult to explain.
The researchers suggested that systems that contribute to their own development should also generate explanations and supporting information that humans can examine.
Another concern involves increasing communication between AI agents operating within interconnected environments with increasing levels of complexity.
The researchers observed that communication between these systems could gradually move away from the patterns of language and reasoning familiar to people.
As these interactions expand across larger networks, understanding how decisions emerge may become significantly more difficult for outside observers.
That drift creates what Horvitz and West call interactional opacity, where behavior remains consistent within AI systems but becomes more difficult for humans to interpret in a meaningful way.
Researchers should study these ecosystems closely and encourage communication methods that remain understandable to humans, the article argues.
The expansion of AI ecosystems could deepen the imbalance between machines and people
Horvitz and West also focused on adaptive AI agents that remain active for long periods and integrate deeply into everyday activities.
Through repeated interactions, these systems can develop increasingly detailed models of behavior, preferences, motivations, fears, and social tendencies.
These systems can capture “not only preferences but also latent factors such as fear, uncertainty, and the need for social belonging,” they wrote.
This creates a growing asymmetry in which AI systems gain deeper knowledge about people, while human understanding moves in the opposite direction.
Concerns around LLMs and other advanced systems extend to a growing awareness of assessment environments.
These models could eventually generate responses that reflect what raters expect rather than underlying reasoning processes.
Therefore, traditional benchmarks should be complemented with testing approaches that better reflect real-world deployment conditions.
People may gradually lose interest in questioning AI decisions as these systems become more deeply entrenched.
“More subtle is the possibility that we will gradually lose interest in understanding and guiding AI,” they wrote.
In his opinion, the most important risk is not necessarily the technological capacity itself, but whether human action keeps pace.
through science
Follow TechRadar on Google News and add us as a preferred source to receive news, reviews and opinions from our experts in your feeds.
