UK AISI and US CAISI jointly assess Moonshot AI's Kimi K3 for cyber capability

The UK AI Security Institute and the US Center for AI Standards and Innovation published a joint preliminary assessment of the cyber capabilities of Kimi K3, the open-weight model Moonshot AI had released a week earlier. It was the second such joint evaluation of a Chinese open-weight model that month, following a CAISI assessment of Zhipu’s GLM.

The evaluation used two benchmarks: ExploitBench, a Carnegie Mellon set of 41 tasks requiring exploit development against post-2023 vulnerabilities in the V8 JavaScript engine, and “The Last Ones,” a 32-step simulated attack on a corporate network. On ExploitBench, Kimi K3 failed to develop a single working exploit that achieved arbitrary code execution, against an average of 20 of 41 for the frontier US closed models tested — which were run with their safety safeguards disabled to measure maximum underlying capability rather than what a deployed product would allow. On the network-attack benchmark, Kimi K3 averaged step 17 of 32, against 28.5 for the leading US models, though in one of ten attempts it completed the full scenario unaided.

Kimi K3 did outperform GLM-5.2, the other Chinese open-weight model in the comparison, on both benchmarks. The agencies caveated the results as preliminary: the simulated network scenario lacked active defenders or any penalty for triggering alerts, and Kimi K3 itself was tested on a narrower set of benchmarks than the closed US models because of constraints in how it could be hosted for evaluation.

The assessment fits a pattern of UK and US government bodies publishing dangerous-capability evaluations of Chinese open-weight releases within days of launch, treating offensive cyber capability as a standing question for each new frontier model regardless of where it was trained.