Speaking of Responsibility: Autonomous Weapon Systems, State and Individual Responsibility

Liron Shilo
Georgetown Law, SJD Candidate

The debate around autonomous weapon systems (AWS) is intensifying as technology advances. This post grapples with the notion of state and criminal responsibility in the context of AWS.

Scholarly approaches to criminal liability in the context of AWS run the gamut. One extreme argues for the existence of complete responsibility for every action (legal or illegal) AWS will commit. It assumes AWS are deterministic since they were created by humans. They therefore cannot surpass the knowledge and intent of their creators (Michael Schmitt). The other extreme warns against complete impunity for any crimes committed by AWS, by virtue of their unpredictability and lack of control that stems from their autonomous character (Human Rights Watch). Other scholars recognize that applying criminal responsibility on AWS will prove to be difficult. Instead, and in light of the many prospective virtues of AWS, they put forward a utilitarian argument for collective (state) responsibility.

It is the last argument, endorsed by Matthew Waxman and Kenneth Anderson, that strikes me as the most dangerous. It endorses the repetition of past errors stemming from complete reliance on the regime of state responsibility in the context of war. And it constitutes a complete deviation from the trend towards individual responsibility initiated in the Nuremburg Trials after WWII, which still plays a significant part in recent criminal tribunals and courts such as the ICTY, ICTR and ICC.

Prior to expounding on this argument, a short description of AWS’ underlining technology and its interaction with International Criminal Law (ICL) is necessary. By and large, autonomous systems are systems capable of completing a certain task without human input. The machines are able to sense, decide and act upon certain circumstances without human intervention.

Scharre and Horowitz emphasize that autonomy means different things to each of the above actions. Pointing a spotlight on decision making and actions, they pose that the former describes the amount of control humans have over the decisions of the AWS i.e. whether humans actively partake in the decision-making process (man in the loop), supervise it (man on the loop) or are completely detached from it (man out of the loop). The latter describes the AWS’ ability to execute an action not directly predictable by its programmers, i.e. the foreseeability of AWS’ actions. At the high end of the technological spectrum, with more sophisticated systems, AI, and more specifically machine learning, negates both the foreseeability and human control over AWS’ actions. Although machine-learning produces better results than that of humans (e.g. IBM’s ‘Watson’ winning Jeopardy and diagnosing cancer), it does not enable humans to examine the causal logic applied in order to reach such results (AKA the black-box), as well as to foresee the nature of the outcome.

In the context of ICL, when applying individual criminal responsibility, the International Criminal Tribunal for the Former Yugoslavia (ICTY) in Prosecutor v. Furundzija required that the accused have a criminal intent (mens rea) in the form of “knowledge that [the accomplice’s] action will assist the perpetrator in the commission of the crime” (¶245). Assuming no malintent existed in its programming, it is impossible to prove the required intent when one cannot foresee the actions of AWS. In addition, when interpreting the notion of command responsibility, the ICTY in Prosecutor v. Delalic (Celebici) indicated that the required actus reus is effective control (“…material ability to prevent and punish the commission of these offences")(¶¶ 377-8). The Court articulated the required mens rea in Prosecutor v. Halilovic as knowledge or the assertion of having knowledge of the crimes committed (¶54). In light of the above, it is highly unlikely to hold a commander criminally liable for not ex ante preventing or ex post factum punishing AWS, where she does not have any material ability to do either.

Waxman and Anderson recognize the above issues with individual criminal responsibility in the context of AWS, although they minimize their importance. They argue that state responsibility, as a traditional tool for “effective adherence” to international law, is more suitable, and that society should not a priori forego any future gains stemming from the application of AWS merely due to lack of human accountability. Even though their argument sounds appealing, as most utilitarian arguments do, it is legally and politically impractical and it ignores post WWII trends towards applying individual criminal responsibility.

From a legal standpoint, states can only be held accountable for attributable illegal actions that were conducted by their armed forces i.e. human beings (Art. 43 of Additional Protocol I). Hence, any crime committed by AWS would not result in direct state responsibility. In addition, due to the commander’s inability to predict or fully control the actions of AWS in the field, states will also not be able to assume responsibility by proxy.

There is also a practical inability to hold states responsible for their actions in the international realm, especially the five countries who have the power of veto in the U.N. Security Council. Political consideration and lack of effective international institutions do and will negate any such possibility (e.g. see Russia’s annexation of Crimea). These five countries are also the most likely to obtain and use these technologies in the near future.

Moreover, a trend towards individual criminal responsibility has emerged in the late 1940’s, post WWII. It started with the creation of the Nuremberg and Tokyo Tribunals, trying and punishing Nazi and Japanese officials who committed war crimes during WWII. It continued with the creation of several other ad-hoc tribunals, such as the ICTY and ICTR trying individuals for war crimes, crimes against humanity and genocide committed during the 1990’s in the Balkans and 1994 in Rwanda.

Most recently, this trend culminated with the creation of the first permanent International Criminal Court, the ICC, aiming to “hold those responsible accountable for their crimes and to help prevent these crimes from happening again”. Admittedly, the ICC is heavily criticized in terms of its institutional powers, effectiveness and case selection. However, its emergence at the beginning of the 21st Century, with the initial endorsement of highly active and relevant state actors such as the U.S., Israel and the United Kingdom (some later retracted their signatures), indicates that the shift towards individual accountability has not yet come to a halt.

It will be impractical and counterproductive to rely solely on state responsibility as a guiding and mitigating tool for the expected deficiencies of AWS. Without prejudice to any specific solution vis-à-vis AWS, such line of argument will defy the objectives of international law and diverge of its current direction. As articulated by Robert Sparrow, “it is a fundamental condition of fighting a just war that someone may be held responsible for the deaths of enemies killed in the course of it.” Therefore, any future technological, policy or legal solution, should seek to facilitate the applicability of both state and individual responsibility for crimes committed by AWS.