AI agents must be treated as untrusted systems: Researchers

Security for artificial intelligence-powered agents should be built into the entire system, not just around the model itself, to better prevent failures and attacks from bad actors, according to a new research paper.

The amended paper, released on May 20 by researchers from Google, Gray Swan AI, EmbraceTheRed, and several universities, argued that agent security must be approached as a systems problem and that AI agents should be treated as an untrusted component.

“Through this lens, efforts to increase model robustness, the dominant viewpoint in the community, are insufficient on their own. Instead, we must complement existing efforts with techniques from the systems security domain,” the researchers said.

“Towards this end, we propose viewing agent security as an instance of computer security. This domain has long dealt with powerful attackers and motivated decades of research on principles and techniques that deal with such adversaries.”

AI agents are becoming increasingly popular among crypto users. Some crypto executives have speculated that AI agents in the space could explode in the next few years. Circle CEO Jeremy Allaire predicted in January that billions of AI agents would be operating on users’ behalf within five years.

Core security protections could stop most attacks

The researchers said that after studying a range of attack case studies, “three mechanisms” could “eliminate a large fraction of attacks.”

They argue that AI agents should clearly distinguish between instructions and untrusted data to avoid attackers duping the agent by hiding malicious instructions within data. The AI agent should also only have the minimum permissions necessary to perform a task, rather than full access, according to the researchers.

The researchers said that standard security setups include trusted and untrusted systems, and that AI should be treated as an untrusted system. Source: Agent Security is a Systems Problem

At the same time, the wider system should control where sensitive information is allowed to go, not the agent, to ensure it can’t be manipulated into sending sensitive data to unsafe destinations.

In a recent case, the AI-powered crypto trading assistant Bankr said it disabled transactions on May 20 after identifying an attacker who had gained access to at least 14 wallets. Security experts speculated that the bot could have been exploited by a hacker.

AI agents are being used to build Web3 applications, launch tokens and interact with services and protocols autonomously, with some platforms exploring AI for trading.

Aaron Ratcliff, attributions lead at blockchain intelligence firm Merkle Science, told Cointelegraph last year that from a security standpoint, giving an AI agent access to a wallet adds a layer of trust to something designed to be trustless, and it can be safe if the system is built correctly.

Related: Exodus launches AI agent-focused stablecoin on Solana

“I’d want proof that the AI can catch front-running, apply slippage limits, spot scam tokens, and audit contracts in real time before it makes a trade. It should also sandbox prompts, prevent injection, and block man-in-the-middle access,” he said.

Meanwhile, Sean Ren, co-founder of the AI-native blockchain platform Sahara AI said model context protocols are the gold standard for safety when set up correctly, but users should still pay attention to every action performed by an AI agent.

“They essentially act as a gatekeeper between the AI model and your wallet. The agent can only perform specific, approved actions—such as checking balances or preparing a payment for you to confirm—rather than freely moving funds or changing wallet settings,” he said.

Magazine: Crypto scammers face death, Aussie CGT makes Asian hubs attractive