New CNAS Report Outlines a Path for Advancing DoD AI Assurance

Washington, March 13, 2025 — Today, the Center for a New American Security (CNAS) released a new report, Safe and Effective: Advancing Department of Defense Test and Evaluation for Artificial Intelligence and Autonomous Systems, by Josh Wallin. The report details how the Department of Defense (DoD) can ensure these systems are both reliable and aligned with U.S. military policy and operational needs.

With AI and autonomy set to shape modern conflicts, the United States must develop agile and iterative testing and evaluation (T&E) processes that match the pace of technological advancement. The report highlights the need for a dynamic approach to T&E that balances opportunity with responsible risk management as AI plays an expanding role in military operations.

The report offers a series of actionable recommendations for the Department of Defense, including:

• Developing a set of common characteristics to identify AI and autonomous systems that present novel assurance and sustainment concerns;

• Requiring greater information sharing about system design techniques and engineering practices through contract language;

• Embedding operators with developers early in the program lifecycle;

• Emphasizing the importance of explainability for operators from the outset;

• Developing rigorous, testable metrics for compliance with Responsible AI principles;

• Providing common, validated modeling and simulation tools to support development efforts;

• Establishing an early and ongoing dialogue between program offices, testers, and operators; and

• Broadening test safety policies to address AI-enabled and autonomous systems.

By adopting these recommendations, the DoD can build a foundation for AI and autonomous system development that drives technological innovation while maintaining America's edge in an increasingly competitive global landscape.

This report comes just days after the launch of the CNAS Defense Program’s new initiative on AI in future warfare. Subscribe to the Defense mailing list to keep up with the program.

For more information or to arrange an interview with the report’s author, please contact Alexa Whaley at [email protected].