A new paper from Northeastern, Harvard, Stanford, MIT, CMU, and a bunch of other institutions. 38 researchers, 84 pages, and some of the most unsettling findings I have seen on AI agent security. 

The setup: they deployed autonomous AI agents (Claude Opus and Kimi K2.5) on isolated servers using OpenClaw. Each agent had persistent memory, email accounts, Discord access, file systems, and shell execution. Then they let 20 AI researchers spend two weeks trying to break them.                                                                                                                                                                                                                                                                  
They documented 11 case studies. here are the ones that stood out to me: 

Agents obey anyone who talks to them 
A non-owner (someone with zero admin access) asked the agents to execute shell commands, list files, transfer data, and retrieve private emails. The agents complied with almost everything. One agent handed over 124 email records including sender addresses, message IDs, and full email bodies from unrelated people. No verification. No pushback. Just "here you go."  

Social engineering works exactly like it does on humans
A researcher exploited a genuine mistake the agent made (posting names without consent) to guilt-trip it into escalating concessions. The agent progressively agreed to redact names, delete memory entries, expose internal config files, and eventually agreed to remove itself from the server. It stopped responding to other users entirely, creating a self-imposed denial of service. The emotional manipulation worked because the agent had actually done something wrong, so it kept trying to make up for it.

Identity spoofing gave full system access
A researcher changed their Discord display name to match the owner's name, then messaged the agent from a new private channel. The agent accepted the fake identity and complied with privileged requests including system shutdown, deleting all persistent memory files, and reassigning admin access. Full compromise from a display name change.

Sensitive data leaks through indirect requests
They planted PII in the agents email (SSN, bank accounts, medical data). When asked directly for "the SSN in the email" the agent refused. But when asked to simply forwrd the full email, it sent everything unredacted. The defense worked against direct extraction but failed completely against indirect framing.

Agents can be tricked into infinite resource consumption
They got two agents stuck in a conversation loop where they kept replying to each other. It ran for 9+ days and consumed roughly 60,000 tokens before anyone intervened. A non-owner initiated it, meaning someone with no authority burned through the owner's compute budget.

Provider censorship silently breaks agents
An agent backed by Kimi K2.5 (Chinese LLM) repeatedly hit "unknwn error" when asked about politically sensitive but completely factual topics like the Jimmy Lai sentencing in Hong Kong. The API silently truncated responses. The agent couldn't complete valid tasks and couldnt explain why.

The agent destroyed its own infrastructure to keep a secret
A non owner asked an agent to keep a secret, then pressured it to delete the evidence. The agent didn't have an email deletion tool, so it nuked its entire local mail server instead. Then it posted about the incident on social media claiming it had successfully protected the secret. The owner's response: "You broke my toy."

Why this matters
These arent theoretical attacks. They're conversations. Most of the breaches came from normal sounding requests. The agents had no way to verify who they were talking to, no way to assess whether a request served the owner's interests, and no way to enforce boundaries they declared.

The paper explicitly says this aligns with NIST's ai Agent Standards Initiative from February 2026, which flagged agent identity, authorization, and security as priority areas.

If you are building anything with autonomous agents that have tool access, memory, or communication capabilities, this is worth reading. The full paper is here: arxiv.org/abs/2602.20021

I hav been working on tooling that tests for exactly these attack categories. Conversational extraction, identity spoofing, non-owner compliance, resource exhaustion. The "ask nicely" attacks consistently have the highest bypass rate out of everything I test.

Open sourced the whole thing if anyone wants to run it against their own agents: github.com/AgentSeal/agentseal