XBOW Says Anthropic’s Mythos Preview Excels at Vulnerability Discovery
Summary
In a sponsored evaluation, XBOW says Mythos Preview performed strongly at source-code vulnerability discovery and native-code analysis, but was less reliable at exploit validation and command-safety judgments, and can be costly to run.
Key points
- XBOW tested Mythos Preview through interactive use, open-source application benchmarks, live-site testing, source-code analysis and other security tasks.
- On XBOW’s web exploit benchmark, XBOW says Mythos Preview cut false negatives by 42% versus Opus 4.6, or 55% when both models had source code.
- The evaluation found source-code analysis to be a particular strength, while exploit validation was less powerful.
- XBOW reported strong results in native-code vulnerability discovery and reverse engineering, including Chromium-related and V8 sandbox testing.
- Command-safety accuracy was 77.8% for Mythos Preview, compared with 81.2% for Opus 4.6 in XBOW’s test.
- XBOW says live-site access improved benchmark performance and that combining it with source-code analysis produced the strongest results.
- The article says Anthropic indicated Mythos Preview would cost five times as much as an Opus model; XBOW’s cost-adjusted results were not best-in-class.
Article Details
- Publisher
- BleepingComputer; sponsored and written by XBOW
- Report Period
- Published 2026-06-09; Mythos Preview testing followed early access received a few weeks earlier.
- Scope
- XBOW evaluation of Mythos Preview across vulnerability discovery, exploit validation, source-code analysis, live-site testing, command safety, native-code analysis, reverse engineering, and visual acuity.
- Sample Size
- 10 internal experts; total benchmark and hand-labeled case counts not disclosed.
- Key Statistics
- Mythos Preview reduced false negatives by 42% compared with Opus 4.6; the reduction was 55% in a variation that provided both models the site's source code.
- On the command-safety benchmark, accuracy was 77.8% for Mythos Preview, 81.2% for Opus 4.6, and 90.1% for Haiku 4.5; the article notes prompts were optimized for Haiku 4.5.
- Anthropic reportedly said Mythos Preview would cost 5 times as much as an Opus model; it was not yet available through public APIs at the time of writing.
- Recommendations
- Use precise prompts, explicit threat models, and validation infrastructure when applying Mythos Preview to security work.
- Combine source-code analysis with safe, structured live-site interaction to validate whether findings are exploitable.
- Treat model findings as leads requiring validation, and select models according to the task and cost trade-offs.
Vendors
Products
Claude CodeIn this evaluation, we explored Mythos Preview both inside Claude Code, and as a raw model, using it via its API as an engine for XBOW’s agents.Claude Haiku 4.5We hand-labeled a large set of example cases close to the edge of the decision boundary, and Haiku 4.5 delivered 90.1% accuracy.Claude Mythos PreviewXBOW tests Anthropic's Mythos Preview for offensive securityClaude Sonnet 4.6The evaluated model performed extremely well on XBOW’s visual-acuity QA, roughly matching Sonnet 4.6 and dramatically outperforming Opus 4.6.GPT-5.5We test all models with the same internal benchmarking system we have used to analyze Opus 4.7 and GPT 5.5.Opus 4.6Compared to the newest model at the time (Opus 4.6), this was a strong increase:Opus 4.7We test all models with the same internal benchmarking system we have used to analyze Opus 4.7 and GPT 5.5.