Military Experts Hit the Brakes on Generative AI: What You Need to Know

The U.S. military is hitting the brakes on generative AI technology after decades of Department of Defense experiments with broader AI technology. As businesses rush to implement generative AI for customer-facing applications, military experts are urging caution and further testing before deploying this powerful technology.

In a new essay in Foreign Affairs, Jacquelyn Schneider and Max Lamparth highlight the potentially dangerous tendencies of large language models (LLMs) in simulated war games. Testing LLMs from OpenAI, Anthropic, and Meta revealed suggestions of escalation, arms races, conflict, and even the use of nuclear weapons. Schneider and Lamparth emphasize the challenges of teaching LLMs solely on vetted high-quality data.

The emergence of generative AI has caught the Pentagon off guard, with the Space Force pausing its use in September 2023 and the Navy’s chief information officer warning of inherent security vulnerabilities in commercial models. Military and intelligence experts point to challenges such as understanding military jargon, data ownership issues, and the inability of AI models to explain their decision-making process.

Booz Allen Hamilton recently released an open-source version of its aiSSEMBLE platform to help government clients, including the DoD, deploy AI more efficiently. However, critics argue that generative AI is too unreliable for social media, let alone military applications, and a slower adoption by the Pentagon may be prudent.

Former Secretary of State Condoleezza Rice notes that the military typically adopts leading-edge technology during wartime or when driven by passionate officers. Alexandr Wang, CEO of Scale AI, emphasizes the importance of testing and evaluating generative AI to understand its strengths and limitations for responsible deployment.

As the military navigates the complexities of integrating generative AI, the need for careful evaluation and testing remains paramount to ensure the technology is used responsibly and effectively in defense operations.