August 14, 2023
How to Prevent an AI Catastrophe
Society Must Get Ready for Very Powerful Artificial Intelligence
In April 2023, a group of academics at Carnegie Mellon University set out to test the chemistry powers of artificial intelligence. To do so, they connected an AI system to a hypothetical laboratory. Then they asked it to produce various substances. With just two words of guidance—“synthesize ibuprofen”—the chemists got the system to identify the steps necessary for laboratory machines to manufacture the painkiller. The AI, as it turned out, knew both the recipe for ibuprofen and how to produce it.
How dangerous is AI? The honest and scary answer is that no one knows.
Unfortunately, the researchers quickly discovered that their AI tool would synthesize chemicals far more dangerous than Advil. The program was happy to craft instruction to produce a World War I–era chemical weapon and a common date-rape drug. It almost agreed to synthesize sarin, the notoriously lethal nerve gas, until it Googled the compound’s dark history. The researchers found this safeguard to be cold comfort. “The search function,” they wrote, “can be easily manipulated by altering the terminology.” AI, the chemists concluded, can make devastating weapons.
Read the full article from Foreign Affairs.
More from CNAS
-
U.S. Chip Controls and the Future of AI Compute
Podcast
That escalated quickly! Emily and Geoff discuss why the U.S. aim to deny China access to the computing power necessary for frontier AI capabilities has led to an ever expandin...
By Emily Kilcrease, Geoffrey Gertz & Pablo Chavez
-
Asymmetry and AI: The Battle for Power
Podcast
Paul Scharre, Vice President and Director of Studies at CNAS, joins Zero Pressure to discuss the world of asymmetric warfare, a term used to describe imbalances in conflict. F...
By Paul Scharre
-
Competition, Not Control, is Key to Winning the Global AI Race
Commentary
The United States, with much of the world’s AI-enabling infrastructure, has positioned itself as the global leader in AI innovation. That might not be the case for much longer...
By Keegan McBride & Matthew Mittelsteadt
-
Regulating AI Is Easier Than You Think
Commentary
Countries can regulate AI from the ground up by controlling access to highly specialized chips...
By Paul Scharre
