AI safety benchmarks stories

Jul 31, 2026

Medical AI Community Grapples with Benchmarking Standards as Clinical LLMs Proliferate

Multiple research publications released in late July 2026 highlight growing concerns about how to properly evaluate large language models for medical applications, as clinical chatbots gain traction despite questions about reliability and appropriate benchmarking methods.

Jul 31, 2026·5 sources

Apr 12, 2026

AI Models Prefer Guessing Over Asking for Help When Information Missing, Research Shows

ProactiveBench testing of 22 multimodal language models found that almost none ask users for help when visual information is missing, instead choosing to guess. Simple reinforcement learning can improve this behavior.

Apr 12, 2026·1 source

Study Finds AI Agent Skills Fail Under Realistic Conditions Despite Strong Benchmark Performance

Research testing 34,000 real-world AI agent skills found that modular instructions designed to give agents specialized knowledge fall apart under realistic conditions, despite performing well in benchmarks.

Apr 12, 2026·1 source

Apr 5, 2026

Study Finds AI Offensive Cyber Capabilities Doubling Every Six Months

New research shows AI models' ability to exploit security vulnerabilities has been doubling every 5.7 months since 2024, with Claude Opus 4.6 and GPT-4o demonstrating advanced offensive capabilities.

Apr 5, 2026·1 source

Top claims

▪For 114 validations where a manually curated hypertrophy network has a functional accuracy of 94.74%, large language model-generated networks achieve an accuracy of only 6.14% to 33.33%.
▪Large language model-generated models of fibroblast activation and mechanosignaling exhibit validation accuracies of 16.87% to 21.69% and 5.81% to 14.53%, respectively, against independent literature.
▪General-purpose large language models generate 24% to 65% of the reactions of literature-curated signaling networks for cardiomyocyte hypertrophy, myofibroblast activation, and mechanosignaling.