Microsoft warns that AI agents are being modified with 'AutoJack' to deliver RCE payloads browsing untrusted websites

Microsoft Defender Security Research Team Reveals “AutoJack,” a String Vulnerability in AutoGen Studio That Allows RCE Through Malicious Websites
The flaws included misuse of the localhost channel, skipped login checks, and arbitrary code execution, which allowed agents to execute programs provided by the attacker.
The issue existed only in early versions of GitHub, and was fixed before release; highlights the need for strict authentication and isolation of local control aircraft

Microsoft’s Defender security research team has revealed a vulnerability chain in AutoGen Studio that allows a single malicious website to perform remote code execution (RCE) on a device running an AI agent.

AutoGen Studio is a program created by Microsoft Research to develop AI agents. The vulnerability chain was named “AutoJack” and consists of three flaws that, analyzed separately, are not particularly concerning. However, chained together is a completely different story.

“The technique, which we call AutoJack, turns the agent into the attacker’s last-mile delivery vehicle by crossing the local host trust boundary that many development tools rely on,” Microsoft explained in its report.

Patching the bugs

First, AutoGen Studio had a local control channel that only accepted “localhost” connections, which is a good way to block external attackers.

However, an AI agent’s web browser also counts as “localhost”, meaning these connections will also be accepted. Then for this particular channel the login checks were skipped.

The app had several ways to request a username and password, but the part of the code that handles this specific local channel was left open.

Finally, the channel would execute almost anything it was told. Microsoft researchers managed to execute an arbitrary program, meaning threat actors could do the same, albeit with malicious code.

In theory, the attack would work like this: the victim would instruct its AI agent to summarize a specific website. Doing so would prompt the agent to download and execute malicious code that could be anything from backdoor malware to information stealers.

The good news is that Microsoft found this issue and reported it before it reached regular users. The official downloadable version of AutoGen Studio never had this problem, as it only existed in an initial version under development on GitHub. The AutoGen team has since managed to fix it.

“If an agent can browse untrusted pages and also talk to privileged local services, the loopback can become an attack surface and control planes must be authenticated, authorized, and isolated,” Microsoft concluded.