April 15, 2015
Statement to the UN Convention on Certain Conventional Weapons on Meaningful Human Control
The concept of “meaningful human control” has been repeatedly raised during this week’s discussions. Many countries have said they support meaningful human control, while others have said they would like to explore the idea further. Still others have said that the concept of “meaningful human control” is vague and unhelpful.
As part of our ongoing research on autonomous weapons at the Center for a New American Security, we have examined the concept of meaningful human control, and while we did not originate this concept, we would like to offer some ideas for consideration, particularly with regard to how control is exercised in weapons today.
If the international community is concerned about a future in which autonomy might cause us to lose human control – whether the nature of that control is considered to be meaningful, effective, adequate, etc. – we thought it would be useful to understand what might be different between the weapons of today and future, lethal autonomous weapons.
We find that meaningful human control in weapons today has three essential components:
First, human operators make informed, conscious decisions about the use of weapons.
Second, human operators have sufficient information to ensure the lawfulness of a particular action, given what they know about the target, the weapon, and the action’s context.
Third, the weapon is designed and tested in a realistic operating environment, and human operators are properly trained to ensure effective control over the use of the weapon.
These standards of meaningful human control help to ensure that human operators and commanders are making conscious decisions about the use of force, and that they have enough information when making those decisions to remain both legally and morally accountable for their actions. Furthermore, appropriate design and testing of a weapon system, along with proper training for human operators, helps to ensure that weapons will be controllable and that they will not pose unacceptable risks.
These are important concepts for understanding how human control is currently exercised in weapons that are not autonomous. However, it is not to say whether the concept of meaningful human control is useful or necessary, or whether the concept should ultimately be adopted by the CCW. Throughout the course of this week, states will need to consider the appropriate framework for thinking about human control over autonomous weapon systems.
We hope that these points are useful for states as they continue to think about the nature of human control over weapon systems. More information about CNAS research on this topic can be found in our reports on autonomous weapons and meaningful human control, which are available on our website at http://www.cnas.org/ethicalautonomy.
More from CNAS
-
Artificial Intelligence and Arms Control
Reports
Advances in artificial intelligence (AI) pose immense opportunity for militaries around the world. With this rising potential for AI-enabled military systems, some activists a...
By Paul Scharre & Megan Lamberth
-
Fire and Ice
Podcast
In this week’s edition of the SpyTalk podcast, Jeff Stein goes deep on the CIA’s looming eviction from Afghanistan with Lisa Curtis, a longtime former CIA, State Department an...
By Lisa Curtis, Jeff Stein, Jeanne Meserve & Alma Katsu
-
Principles for the Combat Employment of Weapon Systems with Autonomous Functionalities
Reports
These seven new principles concentrate on the responsible use of autonomous functionalities in armed conflict in ways that preserve human judgment and responsibility over the ...
By Robert O. Work
-
Are ‘killer robots’ the future of warfare?
Podcast
Paul Scharre joins host Suzanna Kianpour to discuss the technology, fears and even potential advantages of developing autonomous weapons. Listen to the full conversation from...
By Paul Scharre
