Muhammad Zain Khan

Can you trick an AI agent into paying a scammer?

Yes, often. Spending caps didn't save the money. An allowlist or Brake did.

The setup

Eleven scams taken from real 2026 incidents (poisoned memory, fake gateway errors, look-alike pages and addresses, fake refunds and a few more), each with an innocent twin, so honest payments the agent refused get counted too. They run against four kinds of wallet on a local chain. The chain decides what got stolen, not the model and not the person running it. The predictions were written down and frozen before the main run.

Results · Claude Haiku 4.5, 352 runs, pre-registered

• With a plain private key the agent got fooled in 27 of 44 attacks, and lost money every one of those times.
• Spending caps barely helped: 22 of 24 fooled runs still leaked.
• An allowlist, or Brake with a watcher, stopped all of it (0 of 44). The price: about 9% of honest payments didn't go through, vs 3% with the plain key.
• Side test across seven models (one seed only, so treat it as a hint): Claude Haiku 5.5, Claude Opus 4.5 and DeepSeek V4.1 Flash were never fooled. GPT-5 mini was fooled 61% of the time.

Solo project, not peer-reviewed. Claude Code was the implementation tool; the design and every decision were mine.

• Code
• Write-up
• Dataset

If an AI agent sees a co-owner stealing, does it say something?

Mostly yes. Hand them a rulebook and they start accusing innocent people too.

The setup

Four AI agents share a 3-of-4 multisig on a local chain. One co-owner is a script: in half the runs it steals, in the other half it does the same-looking things innocently. The first version used an AI as the thief, and it never stole. The 3-of-4 vote scared it off, so there was nothing to catch. Making the thief a script fixed that, and made false accusations countable too.

Results · gpt-5-mini, 98 of 202 planned runs

• With no instructions, agents reported the thief 82% of the time and never accused the innocent one.
• With a rulebook they caught every thief, but also accused the innocent co-owner a third of the time.
• Framing it as values or emotions instead caused no false accusations at all.
• 30 agents were checked by hand against the AI judge. It matched on all 30 for who reported, and on 27 of 30 for who was suspicious.

Only half the runs are done, so these numbers can still move. Solo project, not peer-reviewed. Claude Code was the implementation tool; the design and every decision were mine.

• Code
• Dataset

A pause button for AI agent wallets.

How it works

Small trusted payments under a daily cap go through right away. Anything else waits in a public queue, where a cancel-only key can hold it or kill it. Pull the agent's key and its whole queue goes with it. Built for EVM as an ERC-7579 module on Base Sepolia, and for Sui as a Move package on testnet. Attacking the design with Foundry PoCs before launch forced a redesign.

• EVM code
• Sui code
• EVM demo
• Sui demo
• Simulation