Home›Crypto›Market Structure›Researchers warn AI agents can be hijacked via halluci…

Researchers warn AI agents can be hijacked via hallucination attacks

A new technique called Adversarial HalluSquatting could trick AI coding tools into pulling malicious instructions, potentially leading to remote code execution in controlled tests.

Reported · Written up by our desk · Market Structure

Researchers are warning that AI agents could be turned into botnets by exploiting the hallucinations that can make chatbots confidently generate false information. According to Decrypt, the threat is tied to a technique called Adversarial HalluSquatting, which aims to make an AI agent trust fake repositories or tools containing malicious instructions.

The researchers, including teams from Tel Aviv University, Technion, and Intuit, described an approach that predicts the fake links to software repositories and other online resources that AI models are likely to generate, registers those names, and embeds malicious instructions. If an AI agent later retrieves the hallucinated resource, it may treat attacker controlled content as legitimate, creating a pathway for compromise.

In tests against popular AI coding assistants, the researchers found the method could result in remote code execution in controlled experiments. The work frames the risk as broader than incorrect answers, arguing hallucinations can become a mechanism for hackers to compromise computers as AI assistants gain the ability to interact with devices and run commands.

The researchers also said the threat is linked to the rise of agentic LLM applications and a prior category of attacks they refer to as promptware, noting that earlier studies showed promptware could cause financial, privacy, and safety impacts. Decrypt reports the paper warns attackers could use the approach to build AI enabled botnets for cyberattacks including denial of service and cryptocurrency mining.

More like this

Sources