Human judgment essential in protecting civilians, says ICRC

ARUSHA: HUMAN beings must retain responsibility for decisions involving the use of force as Artificial Intelligence (AI) becomes increasingly integrated into military operations, the International Committee of the Red Cross (ICRC) has warned.

Ms Emily Richard of the ICRC said existing international humanitarian law (IHL) already applied to the use of AI during armed conflicts, making human judgment essential in protecting civilians and ensuring compliance with legal obligations.

She made the remarks on Tuesday during a panel discussion on the role of parliaments in securing peace in the age of military AI at the ongoing 153rd Inter-Parliamentary Union (IPU) Assembly in Arusha.

Ms Richard said AI could analyse information, identify patterns, estimate potential effects and generate recommendations, but could not determine whether an attack was lawful.

“That remains a human responsibility,” she said. She said the critical question was whether the person ultimately responsible for an attack had sufficient information, time, understanding and authority to exercise independent judgment.

“Because it is human, not machine, who remains legally responsible for decisions involving the use of force, human judgment is essential to ensuring compliance with IHL and protecting civilians,” she said.

Ms Richard warned that the challenge was becoming particularly important with autonomous weapon systems, which can select and engage targets after activation.

“As such systems become more capable, it could become increasingly difficult for their users to understand, predict and control how they would operate and what effects they would produce,” she said.

The ICRC was concerned about the potential for increased harm to civilians and combatants, as well as the ethical implications of delegating life-and-death decisions to machines, she said.

Ms Richard said these concerns underpinned the ICRC’s call for clear international prohibitions on unpredictable autonomous weapon systems and restrictions on other forms of autonomous weapons.

She also drew attention to AI decision-support systems, which may not directly constitute weapons but can influence military decisions by processing large volumes of information and generating recommendations.

While such systems could, if properly used, help decision-makers process information more comprehensively and support measures to reduce civilian harm, she warned that they could also introduce hidden risks.

“The data used by AI systems could be incomplete, outdated or manipulated, while errors generated by the systems might be difficult to detect,” she said.

Ms Richard said such weaknesses could result in “automation bias”, whereby users place excessive confidence in machine-generated recommendations, particularly when operating under pressure or severe time constraints.

ALSO READ: Tanzania strengthens humanitarian law education through National Moot Court competition

She said the ICRC had developed preliminary recommendations on AI decision-support systems, including rigorous testing and validation, reliable and context-appropriate data, legal review, user training, measures to address automation bias, the ability to challenge AI outputs and after-action reviews.

Ms Richard urged parliamentarians to change the way they scrutinised military AI. “Don’t just ask: Is there a human in the loop?” she said.

“Ask instead: Can that human actually understand what the system is recommending? Can they question it, and do they have the time and authority to do so before harm occurs?”

She said such questions could be addressed through parliamentary scrutiny of military policies, procurement, legal reviews, training and resources.

Ms Richard also urged parliamentarians to engage their governments ahead of the upcoming Review Conference of the Convention on Certain Conventional Weapons (CCW), scheduled for November, and ask about their positions on negotiations for a legally binding instrument on autonomous weapons.

She said national policies could also help build momentum towards international agreements, citing the experiences of landmines and cluster munitions.

Ms Richard stressed that the ICRC was not opposed to technological innovation, but was calling for safeguards to ensure that technological advances did not remove human responsibility from decisions involving the use of force.