XBOW Finds Mythos Strong at Vulnerability Discovery, With Limits Elsewhere
Summary
XBOW’s tests found Mythos Preview particularly strong at source-code vulnerability discovery, native-code analysis and reverse engineering, but mixed in judgment and not best-in-class on cost-normalized benchmarks.
Key points
- XBOW described Mythos Preview as a significant step up in vulnerability discovery and highlighted its strength in source-code audits, native-code analysis and reverse engineering.
- XBOW found Mythos more effective when testing live software alongside source code than when examining source code alone.
- Its judgment was mixed: it rejected false positives better than predecessors but sometimes missed true positives when evidence did not meet its criteria.
- XBOW said Mythos could select appropriate browser actions in live websites, though it was not perfectly accurate with exact coordinates.
- Anthropic said Mythos would cost five times as much as an Opus model; XBOW’s cost-normalized benchmarks did not make it best-in-class.
- For web vulnerability discovery with a fixed token budget, XBOW found Mythos outperformed Opus 4.6 but was outperformed by GPT-5.5.
Article Details
- Event Type
- XBOW evaluation of Anthropic's Mythos Preview AI model
- Impact
- XBOW reported that Mythos Preview is particularly strong at finding candidate vulnerabilities in source code and at native-code discovery and reverse engineering. It performed best when testing live systems alongside source code, while its judgment was mixed and its exploit validation less powerful. XBOW also found that, at estimated running cost, Mythos Preview was not best-in-class on its benchmarks: it outperformed Opus 4.6 but was outperformed by GPT5.5 for web vulnerability discovery under a fixed token budget.
Vendors
AnthropicAnthropic’s Mythos AI model has been making waves since its announcement in early April, primarily because of its reputed ability to unearth considerably more vulnerabilities than any other AI model.XBOWXBOW, an autonomous offensive security firm, has aimed its own AI testing armory against Mythos Preview to check the validity of this and other Mythos capabilities.
Products
Claude Mythos PreviewXBOW, an autonomous offensive security firm, has aimed its own AI testing armory against Mythos Preview to check the validity of this and other Mythos capabilities.GPT5.5finding web vulnerabilities with a fixed token budget, Mythos outperforms Opus 4.6 but is outperformed by GPT5.5.Opus 4.6on our benchmarks either.” For finding web vulnerabilities with a fixed token budget, Mythos outperforms Opus 4.6 but is outperformed by GPT5.5.