Russia has deployed autonomous drones on the Ukrainian battlefield that use artificial intelligence (AI) to choose targets on their own without human control. This is the first confirmation that civilians were killed by a Russian drone whose target was set by AI. As lethal weapons that bypass human judgment become a reality, concerns are growing over mistaken attacks, accountability, and violations of international law.
According to the New York Times (NYT) on the 24th local time, on the 6th of last month a small Russian drone crashed and exploded near a gas station in Zaporizhzhia in southeastern Ukraine. The attack killed three civilians who were studying accounting at Zaporizhzhia National University.
Drone experts, Ukrainian military commanders, and a forensics team that examined the wreckage concluded the drone was guided by an experimental AI system rather than a human pilot. A human operator entered a route to a specific gas station, but after approaching the site, the drone chose its target on its own, believed to be a propane tank.
An analysis of the drone wreckage by Ukrainian authorities found an Nvidia "Jetson Orin" minicomputer inside. The drone also had no antenna for external communications. Because the Nvidia chip was not encrypted, investigators were able to view the terrain images and program code loaded into the drone. The code contained target types that had been trained for the drone to recognize and attack, such as propane tanks.
Katerina Bondar, a senior fellow at the Wadhwani Center for AI under the Center for Strategic and International Studies (CSIS) in Washington, said this incident is the first documented case of civilian deaths caused by a Russian drone equipped with AI that sets its own targets. Bondar said the Nvidia module is "the most conclusive evidence" showing Russia was experimenting with autonomous AI.
Conventional AI drones used AI only in the final step just before an attack after a human operator selected the target. In contrast, the fully autonomous drone believed to have been used this time excluded humans even from the process of selecting the final target. After training on thousands of images, the AI analyzes objects captured by the drone's camera to decide whether to attack.
Vadym Kushnikov, a drone expert at the Kharkiv Aviation Institute in Ukraine, described the drone that killed three civilians as "a preprogrammed device trained in Virtual Reality (VR) to track specific objects."
Autonomous drones do not require radio-based remote control, so they are immune to jamming, and they are more accurate than weapons that strike preprogrammed coordinates. There is also no need to deploy a separate drone pilot. Some experts argue that because AI can assess the surrounding situation just before an attack, it could actually reduce civilian harm compared with unguided bombs or conventional shells.
However, experts note that if AI cannot properly distinguish civilians from combatants or civilian facilities from military facilities, it can lead to mistaken attacks. Because AI derives its own decision criteria by finding patterns in massive data, even developers find it difficult to fully predict the specific decision process. It is also unclear who should be held legally responsible when a mistaken attack occurs.
The International Committee of the Red Cross (ICRC) and human rights groups oppose the use of AI weapons that do not involve humans in the decision-making process. That is because it effectively leaves to machines the judgment of what constitutes a lawful military target under international law.
Col. Serhii Minaiev, commander of the Zaporizhzhia air defense unit, said, "This is dangerous for the world," adding, "In a few years we will be living in the movie 'Terminator.' Machines are deciding whether to attack."