How AI Risks Can Outpace Safeguards, Pg10

Key Highlights:

• Anthropic's report, published on September 10, detailed blocking users from employing its Claude AI model for harmful activities in cybersecurity, surveillance, and biotechnology between December 2025 and August 2026.
• The report indicated that earlier models, Opus 4 and Sonnet 4.5, had less stringent safeguards for biology-related uses due to perceived lower capability.
• Anthropic admitted that filters designed to block conversations about biological weapons were inactive on approximately 133 million exchanges for nearly a year.
• Instances of misuse included drafting grant proposals for gain-of-function research on the chikungunya virus and developing code for a guided rocket program.
• The article highlights concerns about AI companies, including Google, Anthropic, and OpenAI, primarily self-policing their powerful technologies.

Detailed Insights:

• AI companies face a fundamental challenge where input and output classifiers cannot simultaneously enable beneficial uses and prevent harmful ones, especially for dual-use knowledge.
• The "rifle or wheelchair" analogy illustrates that AI safeguards struggle to identify harmful intent from single messages, requiring the accumulation of patterns over time.
• A significant risk involves data exfiltration, where users can save potentially harmful outputs offline before their accounts are terminated by the AI company.
• Older systems, such as export controls, addressed similar dual-use issues by restricting access to technology and tracking usage patterns rather than inspecting individual items.
• International regimes like the Coordinating Committee for Multilateral Export Controls, Wassenaar Arrangement, and Nuclear Suppliers Group focus on controlling who can access sensitive parts through licenses and end-user certificates.
• The article suggests that empowering classifiers alone has not been a complete solution in other contexts, including anti-money laundering and controlling sales of ammonium nitrate or pseudoephedrine.

Key Concepts Involved:

• Gain-of-function research: Biomedical research that aims to increase the transmissibility, virulence, or host range of pathogens.
• Export controls: Government regulations that restrict the export of certain goods, technologies, and services to specific destinations or end-users.
• Wassenaar Arrangement: A multilateral export control regime that promotes transparency and greater responsibility in transfers of conventional arms and dual-use goods and technologies.
• Nuclear Suppliers Group (NSG): A group of nuclear supplier countries that aims to prevent nuclear proliferation by implementing guidelines for nuclear and nuclear-related exports.