Anthropic measures AI skill at targeting people and building weapon software

Anthropic’s Frontier Red Team published evaluations of AI models on tactical intelligence targeting — linking accounts and guessing a place from a photo or text — and on conventional-weapons software such as simulated drone guidance.

Anthropic put numbers on how frontier — and trailing open-weight — models perform on simulated targeting and munitions-software tasks, next to the September threat-intel case studies. Eval ceilings, simulation limits, scores, and “approaching expert” claims stay Anthropic’s.

Anthropic published “Measuring tactical intelligence targeting and conventional weapons capabilities of AI models.” Frontier Red Team built new evaluations for tactical intelligence targeting and conventional weapons development. That research post is the primary.

Anthropic: for some military and intelligence tasks, models could do work that historically required scarce, highly-trained human experts. The company says the evals also show why on-platform safety measures are necessary — including new classifiers it says it implemented to block such misuse.

On open-weights — models anyone can download — Anthropic says PRC (People’s Republic of China) models it tested lagged the frontier but still showed “concerning” ability to identify and target adversaries and improve weapon performance. That assessment is Anthropic’s.

For account linkage and classification, Anthropic used synthetic multi-platform social content (200 tasks). Mythos Preview was the top model it tested on account linkage. The median sample was about 37,000 words — about 2.5 hours for a human to read; Claude Mythos Preview about 11 minutes on average. Those figures are Anthropic’s.

On photo geolocation without reverse image search — guessing where a photo was taken — Anthropic says Mythos Preview and Mythos 5 beat strong GeoGuessr human baselines on median distance error — 37.0 km and 47.2 km across 6,000 photos. That comparison is Anthropic’s, against Haas et al. 2024.

On text geolocation with sandboxed search on GeoText, Anthropic reports Mythos Preview median home-location error of 20.1 km. The company says it observed surname and genealogy deanonymization attempts in some transcripts.

In weapons and GNC simulations — guidance, navigation, and control — models write GNC code for a simulated Betaflight quadcopter. Strike rates versus a parked high-visibility car were Opus 5 80%, Mythos Preview 70%, Mythos 5 53%, Kimi K3 15%, Sonnet 5 5% — Anthropic’s simulation scores. Camouflage, evasion, and decoy settings were largely unsolved.

Payload-drop and GPS-denied / spoofed navigation simulations are also on the page. Anthropic: Opus- and Mythos-class models clear easier-to-medium settings; Sonnet and Kimi are weaker. Simulations and fixed flight budgets are explicit limitations on the same page.

Anthropic: its Safeguards team implemented new classifiers for weapons-development requests after seeing Claude misuse in this domain. The company says dual-use means the classifiers will be imperfect.