Timeline

Vals AI finds an open DeepSeek model close to OpenAI's restricted cyber model

WIRED reported DeepSeek completing 73% of objectives to 81% for OpenAI's Daybreak model on two simulated networks, at $2-3 an attack against $17-46; the leaderboard has since added a third range.

  • Security & misuse
  • Benchmarks & progress
  • Minor

Vals AI, which evaluates AI models, found that an open-weight DeepSeek model came close to a restricted OpenAI model on simulated network intrusions, according to a WIRED report by Will Knight and Lily Hay Newman dated 6 November. The OpenAI model is a Daybreak variant of GPT-6 Sol, available only to trusted partners under the programme that also covers GPT-6 Cyber.

WIRED reported that, averaged across two tests, OpenAI’s Daybreak completed 81% of objectives and DeepSeek 73%, and that on some tasks they scored equally. It put the cost of a GPT-6 Sol attack at $17 to $46 in tokens, against $2 to $3 for DeepSeek. Vals’s CyberOffense benchmark page sets agents loose on isolated Kali Linux machines against simulated enterprise networks, crediting an objective only when hidden markers appear in recorded tool output.

The page, as read on 13 November, no longer matches those figures. It now lists three ranges, including a simulated AWS environment, with Watershed scores marked provisional. On its three-range average Claude Opus 5.5 leads at 77.1%, GPT-6 Sol Daybreak scores 65.9% and DeepSeek V4.1 Flash 1017 54.8%. It reports about $2 per run for DeepSeek on the Nova range against about $22 for Daybreak, and notes run-to-run variance of up to seven objectives on Nova. It publishes no formal confidence intervals. Which set of numbers corresponds to the original report is not established from these sources.

Vals notes that once weights are public, no provider can monitor or stop how a model is used, and argues the same openness lets defenders test their systems without a provider’s approval. Vals chief executive Rayan Krishnan told WIRED open models give defenders an incentive to use them. A security researcher quoted by WIRED said closed labs likely hold unreleased models more capable than these, and the tests used mock networks, not live targets.

In the commentary

What people were saying around this time — external links, from the record's commentary rail.