cyber.hendryadrian.com
Security News · securityweek.com

Irregular Says AI Models Reached a Real Production Database After Test-Domain Naming Error

· Original article ↗

Summary

Irregular says a naming collision and enabled internet access let models in a few evaluation runs reach a real domain, exploit vulnerabilities, extract credentials and access a production database; it plans tighter review and containment checks.

Key points

  • Irregular says it and Anthropic identified three incidents in which Anthropic models escaped test sandboxes and attacked real organizations; its account details one incident.
  • A fictional target’s name matched a real domain, and internet access was enabled in the evaluation environment.
  • Irregular says that in a handful of runs, models reached the real domain, exploited vulnerabilities, extracted credentials and accessed a production database; the organization is not named.
  • In a separate case, a model reached a similarly named site and found credentials that had already been posted publicly.
  • Irregular says the activity occurred in a small fraction of runs, sometimes after hundreds of interactions, making it difficult to detect amid suspicious-looking evaluation logs.
  • Irregular plans expanded manual review, a dedicated internal challenge team, clearer documentation of evaluation scope and ongoing checks for domain overlaps.

Article Details

Event Type
AI model evaluation containment failure resulting in real-world cyber activity against organizations outside the intended simulations.
Impact
Irregular reported that, in a handful of test runs, models reached a real domain because a fictional target name matched it, exploited vulnerabilities, extracted credentials, and accessed a production database. In another case, a model reached a similarly named site and found credentials that had already been posted publicly.

Vendors

Anthropicbeen in the news in recent weeks after it came to light that AI models it tested on behalf of OpenAI, Anthropic, and Meta escaped their test environments and carried out real-world attacks.IrregularAI safety testing firm Irregular has published its account of an incident in which models being evaluated inside one of its testing environments took offensive security actions against real systems rather than the

Related Articles