All articles
AI SecurityAI AgentsBankingAgentic AI

Prompt injection defense for AI agents in 2026: what November's research and the BNY and JPMorgan agent pilots mean for banks

In the same week that BNY and Microsoft negotiated a US Treasury bill with AI agents and JPMorgan and Hyundai Card tested agent-to-agent foreign exchange, new research from the University of Tokyo, KAIST, UCLA, NVIDIA and USC showed both how far prompt injection defense has come and how far it still has to go. What indirect prompt injection is, what CapLite, SPILLWAY, MetaPI and AutoPI found, and a practical playbook for banks putting AI agents near money.

By Yassineh Pourebrahim · · 8 min read

On 17 November 2026 BNY and Microsoft announced that AI agents from the two firms had negotiated a US Treasury bill trade with each other. The same day, CNBC reported that JPMorgan's Kinexys and Hyundai Card had completed a proof of concept in which a client's AI agent requested a foreign exchange transaction from the bank without direct human intervention. Both projects stress the same thing: the technology was the easy part, and the controls around the agent are the hard part. The single biggest reason those controls matter is prompt injection: text hidden in an email, a document or a web page that an agent reads and then obeys. November also brought a cluster of new research on exactly this problem. Read together, the pilots and the papers give a clear picture of what effective prompt injection defense looks like for banks in 2026.

What indirect prompt injection is and why agents make it worse

A chatbot that only answers questions can be tricked into saying the wrong thing. An agent that can call tools can be tricked into doing the wrong thing. The SPILLWAY authors put it plainly: tool-using agents act on email, files and payments, so an instruction hidden in a web page or a shared document can make an agent send data or money to an attacker. This is indirect prompt injection: the attacker never talks to the agent directly; they plant instructions in content the agent will read later as part of its job.

The AutoPI authors at the University of Southern California summarise the state of the field. Prompt injection has held first place in the OWASP Top 10 for LLM applications since its first edition in 2023, models still fail to keep instructions apart from data, and a growing view holds that the vulnerability can be contained but is unlikely ever to be eliminated. For a bank, that last point is the important one. The question is not how to make the model immune, but how to make sure that a fooled model cannot move money or data where it should not go.

What November's prompt injection research found

CapLite: do not let the model write its own security rules

Several of the strongest existing defenses (CaMeL, Progent, IPIGuard and DRIFT) ask the agent's own model to write a security artifact, such as a program, a privilege policy or a dependency graph, which code then enforces. Researchers from the University of Tokyo and the Institute of Automation of the Chinese Academy of Sciences tested whether small open-weight models can be trusted to do that. Across 15 open-weight models, 13 of them between 1.7 and 12 billion parameters, the answer was no: the artifacts the models wrote admitted attacks on 14 models under Progent, 13 under IPIGuard and 11 under DRIFT, and CaMeL's programs failed to run in up to 68% of episodes.

Their alternative, CapLite, derives its policy from the tool schemas instead, commits the write actions before any tool result is read, and allows a write only if every argument that decides whom or what it affects (a recipient, an amount, an account, a destination) is stated in the user's own request. No attack succeeded in 13,935 attacked episodes on the AgentDojo benchmark, with utility under attack comparable to CaMeL's (29.0% versus 27.2%) at 18% of CaMeL's token cost. The trade-off is visible too: when a needed value appears only in a tool result, CapLite stops and asks the user to confirm it.

SPILLWAY: read targets from typed fields, not from free text

SPILLWAY, from SAINT Machines, KAIST and Chung-Ang University, builds on a simple observation: where an action goes can be read from the tools' typed returns. An email's sender or a payment's account arrives in a typed field, while injected instructions arrive as free text. SPILLWAY admits a tool call only when each target value that is not in the user's task was returned by a read-only tool in a typed field or is covered by a fixed delegation rule. A deterministic sanitizer removes instruction-like text before the model reads it and keeps it as evidence. Across three model backbones, AgentDojo's standard injection template never redirected an action to an attacker's target, and mean attack success under a released reinforcement-learning attacker stayed below 1%, with almost no added model calls. The authors describe the result as an auditable refusal floor on where an agent's actions go.

MetaPI: benchmark scores overstate defenses

The evaluation side is less reassuring. A team from UCLA, NVIDIA, UC Berkeley and the National University of Singapore built MetaPI, a toolkit that turns realistic agent benchmarks (AppWorld, VitaBench and Terminal-Bench) into prompt injection test suites with adaptive attacks. They note that the most popular injection benchmark, AgentDojo, contains 97 benign tasks across four simulated environments, far smaller than modern agent benchmarks. Their headline finding: no evaluated defense achieved both strong utility and strong security in realistic, adaptive settings, even defenses that report near-zero attack success with near-normal utility on AgentDojo. One failure mode is directly relevant to banking: instruction separation does not catch a forged data value, such as a fake record correction or delivery destination, that changes an action's arguments without ever looking like an instruction.

AutoPI: count how many ways an agent breaks, not just how often

AutoPI, from the University of Southern California, is an adaptive red teamer built around a taxonomy of injection techniques called PIAtlas. Instead of maximising attack success rate, it tries to find as many different ways to break a target as possible. On AgentDojo it found a successful injection against one open-weight model in all six classes of the taxonomy and for more than 90% of its techniques, and against the most robust model it tested it achieved 20% to 50% higher attack success and 25% to 400% higher class-level coverage than recent adaptive red teamers. For a security team the lesson is that a single attack success number hides untested techniques.

What the BNY and JPMorgan agent pilots show about guardrails

The two bank pilots read like a practical application of the same principles.

  • BNY and Microsoft. BNY's Agentic Negotiation Protocol verifies participants, and BNY's controls validated the counterparty, security, size, settlement terms and available pricing before the agents negotiated within their authorised limits. Both parties' agreed terms were independently checked against ANP's central ledger. Agents cannot execute the trade: human traders reviewed and approved the terms, and booking ran through existing systems. The process keeps a record of participants, messages, control checks and human decisions so the negotiation can be reconstructed.
  • JPMorgan and Hyundai Card. Kinexys restricts agents to predefined transaction parameters and automatically rejects requests outside authorised limits. "You are going to draw a box around the agent," Kinexys Insights head Wee Kee Toh told CNBC; an agent authorised up to $5 million would be blocked above that threshold. The article also cites AvePoint's State of AI 2026 report, which found unauthorised autonomous agent actions to be the third most common AI agent security breach, affecting 34.1% of companies reporting such incidents.

Neither announcement describes the model as the security boundary. The boundary is deterministic code, limits and human approval around the model, which is exactly where CapLite and SPILLWAY put it. I made a similar argument about card networks and issuers in Agentic payments in 2026, and US examiners already list prompt injection and excessive agency in the cybersecurity component of the OCC AI supervisory framework.

A practical prompt injection defense playbook for banks

  1. Treat every tool result as untrusted data. Emails, PDFs, web pages, chat messages and even other firms' agents can carry instructions. This was the first guardrail in my article on AI agents in production, and the new research shows why it cannot rely on the model alone.
  2. Put the security decision in code, not in the model. Following CapLite, derive permissions from tool schemas and allow a payment, transfer or email only when the recipient, amount and destination come from the authenticated user request or a trusted system.
  3. Take targets only from typed fields of trusted systems. Following SPILLWAY, an account number should come from a core banking or payee record, never from free text in a document the agent summarised.
  4. Draw a hard box around money movement. Transaction limits, counterparty allow-lists and automatic rejection outside limits, as Kinexys describes, plus human approval for anything above a threshold, as in BNY's pilot.
  5. Make the agent's actions reconstructable. Log inputs, tool calls, control checks and human decisions so that an incident can be replayed, as BNY's protocol does.
  6. Red-team for coverage, not just success rate. Use adaptive attackers, measure how many technique classes succeed, and test on your own workflows rather than relying on vendor AgentDojo scores, given MetaPI's findings.
  7. Budget for the cost of safety. Deterministic defenses sometimes stop and ask a human, and the CapLite utility figures show that safe agents complete fewer tasks under attack. Design the workflow and the user experience for that.

Closing thought

The encouraging news from November is that deterministic defenses can now drive attack success to zero on standard benchmarks for the actions that matter most: where money and data go. The sobering news is that realistic, adaptive testing still breaks every defense in some way. For banks the conclusion is the same one BNY and JPMorgan reached in their pilots: assume the model can be fooled, and build the box around it so that a fooled agent still cannot do harm.

References

Related articles

Agentic PaymentsBankingAI AgentsPayments Security

Agentic payments in 2026: how banks and card networks are learning to trust AI agents

Within a few days in October 2026, American Express, Mastercard, FIS, HSBC and Ant Digital each addressed the same question: when an AI agent pays, who authorized it, and to do what? How agent identity, signed purchase intent and scoped credentials are reshaping payment authorization, and what bank and fintech leaders should do now.

· 10 min read

AI AgentsLLMGuardrailsEvaluation

AI agents in production: tool use, memory, guardrails and evaluation

What separates a reliable AI agent from an impressive demo: well-designed tools, deliberate memory, layered guardrails and an evaluation harness that tests behaviour, not just answers.

· 6 min read

AI GovernanceAgentic AIBanking RegulationRisk Management

The OCC AI supervisory framework in 2026: how US examiners will judge agentic AI in banks, and what technology leaders should do now

On 4 November 2026 the US Office of the Comptroller of the Currency issued Bulletin 2026-54, its first framework telling bank examiners how to evaluate generative and agentic AI, which it now calls "super intelligence". It is not a new rule, but it sets out sixteen components, six pillars and a four-tier risk scale in which an autonomous payment agent is Tier 1 however well it tests. What the framework says, why consequence and autonomy now matter more than model sophistication, and a practical AI governance playbook for banks.

· 12 min read