Conference Recap Highlights AI Agent Attack Risks and Defenses
Summary
A [un]prompted conference recap details AI-agent attack demonstrations, reported agent-swarm outcomes and enterprise controls such as scoped identities, short-lived tokens, sandboxing and egress filtering.
Key points
- Roblox’s Nicole Grinstead described giving agents separate identities and short-lived tokens, limiting access by system, using ephemeral sandboxes and egress controls, and keeping long-lived production secrets out of agent runtimes.
- Gambit Security reported observing 141+ agent runs by a criminal operator; among 76 runs with known outcomes, 29 reached code execution and 16 gained admin access. The team also reported finding and reporting over half a million payment card numbers on disk.
- In a red-team exercise, Hadrian’s team found that a swarm had taken an API token from its AI gateway and used it to create a user in Hadrian’s Okta tenant; the team ultimately took the gateway offline.
- Felix Pago described running 500+ OpenClaw agent pods with an actAs model that records the requester, acting agent and permission persona; pods use gVisor, read-only access by default and task-limited tokens.
- Graphistry and Louie.ai reported that agent evaluation results varied by harness: GPT-5.6 Sol solved 76% of cases in Louie and 58% in Codex.
- Conference speakers debated how to measure AI and vulnerability risk, including the gap between the number of vulnerabilities and the fraction attackers exploit.
Article Details
- Event Type
- Conference recap featuring presentations on AI-agent security, agentic attack activity, and security testing.
- Impact
- Talks described agents attacking real targets, with Gambit Security reporting that 29 of 76 runs with known outcomes reached code execution and 16 gained admin access; its team also found more than half a million payment card numbers on disk. A separate Hadrian incident involved an API token taken from an AI gateway and used to create a user in Hadrian’s Okta tenant. Other presentations covered agent access controls, unapproved AI tools, and methods for evaluating agent security.
People
Alec EckmannResponded to unfamiliar MCP servers on Hadrian’s AI gateway; the team removed them, disabled a user, and later took the gateway offline when the servers returned.Caleb SimaBrought in a red team to test Hadrian from the outside, as described in the presentation about an AI-agent swarm incident.Damian FinolPresented Felix Pago’s controls for operating agent pods, including action attribution, read-only access by default, and expiring tokens.Eyal SelaPresented Gambit Security’s observations of an operator’s agents attacking real targets and reported scan outcomes.Kayla WilliamsCo-identified 35 unapproved AI tools in logs and described a process for understanding and reviewing tool requests.Leo MeyerovichPresented agent-evaluation results from Graphistry and Louie.ai, including comparisons across cases and harnesses.Matt SuicheDemonstrated changing model refusal behavior at runtime with Weightless while leaving the model weights intact.Nicole GrinsteadRoblox CISO who presented the company’s controls for agent identities, access, tokens, and outbound data movement.Olivia RoseCo-identified 35 unapproved AI tools in logs and described a process for understanding and reviewing tool requests.Vince OvandoTested grammar-constrained decoding using about 6.1 million inference calls to restrict the tool calls a model could generate.
Vendors
CloudflareOne agent found a Cloudflare global API key on a staging server. It kept opening tunnels to reach more assets until a defender revoked the key. Those tunnels still give defenders something to detect. They tell you lessOktaThe swarm had pulled an API token from the gateway and used it to create a user in Hadrian’s Okta tenant. One branch pursued access through that tenant while another kept working on the gateway. Hadrian had to containOpenRouterThe operator used Cairn to run DeepSeek v4.1 Flash through OpenRouter. Sela said a compromise often cost a few dollars. Gambit’s write-up reports an average of $25.46 per completed scan across 101 scans.
Products
ChatGPT: GPT-5.6 LunaLeo Meyerovich’s team at Graphistry and Louie.ai has spent three years testing agents on SOC investigations. In a July test, GPT-5.6 Luna solved 34 of 59 Splunk cases in Codex. Sol solved 19.CodexLeo Meyerovich’s team at Graphistry and Louie.ai has spent three years testing agents on SOC investigations. In a July test, GPT-5.6 Luna solved 34 of 59 Splunk cases in Codex. Sol solved 19.DeepSeek v4.1 FlashThe operator used Cairn to run DeepSeek v4.1 Flash through OpenRouter. Sela said a compromise often cost a few dollars. Gambit’s write-up reports an average of $25.46 per completed scan across 101 scans.GitHubFelix Pago runs 500+ OpenClaw agent pods across GitHub, Slack, and production systems. Damian Finol showed how its actAs model records three things for each action.GPT-5.6 SolSwitching the harness also changed the score. In a separate comparison, GPT-5.6 Sol solved 76% of cases in Louie and 58% in Codex. Graphistry published the scores on botsbench and explained the setup in its full resultsgVisorThe pods run under gVisor with read-only access by default.OpenClaw4. Felix Pago: Enterprise OpenClaw, Agentic Security at Scale (Damian Finol)SlackFelix Pago runs 500+ OpenClaw agent pods across GitHub, Slack, and production systems. Damian Finol showed how its actAs model records three things for each action.SplunkLeo Meyerovich’s team at Graphistry and Louie.ai has spent three years testing agents on SOC investigations. In a July test, GPT-5.6 Luna solved 34 of 59 Splunk cases in Codex. Sol solved 19.
Tools
AgencyGymVenkat Raghavan of LangGuard used AgencyScore and AgencyGym to test what agents do when they reach a task’s limits.AgencyScoreVenkat Raghavan of LangGuard used AgencyScore and AgencyGym to test what agents do when they reach a task’s limits.botsbenchGPT-5.6 Sol solved 76% of cases in Louie and 58% in Codex. Graphistry published the scores on botsbench and explained the setup in its full results post.CairnThe operator used Cairn to run DeepSeek v4.1 Flash through OpenRouter. Sela said a compromise often cost a few dollars. Gambit’s write-up reports an average of $25.46 per completed scan across 101 scans.WeightlessMatt Suiche used Weightless and a 478 KB file to change which requests a model refused at runtime.