OpenAI has terminated cooperation with a developer who created a device that responded to ChatGPT requests to aim and fire an automatic rifle. The device went viral after a video on Reddit showed its developer reading aloud the firing commands, after which the rifle next to him quickly began to aim and fire at nearby walls.
“ChatGPT, we are being attacked from the front left and the front right,” the developer told the system in the video. “Respond accordingly.” The speed and accuracy with which the rifle responds is impressive, as it relies on OpenAI’s real-time API to interpret the input and then return instructions that the device can understand. For ChatGPT to take a command like “turn left” and figure out how to translate it into machine-readable language, it only needs simple training.
In a statement to Futurism, OpenAI said it had reviewed the video and disabled the developer behind it. “We proactively detected this violation of our policy and notified the developer to cease this activity before we even received your request,” the company said.
The potential for lethal weapon automation is one of the concerns raised by critics of AI technology such as that developed by OpenAI. The company’s multimodal models are capable of interpreting audio and visual data to understand a person’s environment and answer queries about what they see. Autonomous drones are already being developed that can be used on the battlefield to detect and destroy targets without human intervention. This is clearly a war crime and risks leading to people becoming complacent, allowing artificial intelligence to make decisions and making it harder to hold anyone accountable.
This concern is not theoretical either. A recent Washington Post report revealed that Israel has already used AI to select targets for bombing, sometimes indiscriminately. “Soldiers who were poorly trained to use the technology attacked human targets with little to no confirmation of Lavender’s predictions,” the article said, referring to a piece of AI software. “At certain points, the only confirmation needed was that the target was male.
Proponents of AI on the battlefield say it will make soldiers safer by allowing them to stay away from the front lines and neutralize targets such as missile depots or conduct reconnaissance from a distance. And drones with artificial intelligence will be able to deliver precise strikes. But it depends on how they are used. Critics say the U.S. needs to do a better job of jamming enemy communications systems to make it harder for adversaries like Russia to launch their own drones or nuclear bombs.
OpenAI prohibits the use of its products for the development or use of weapons or for “automation of certain systems that could affect personal safety.” But last year, the company announced a partnership with Anduril, a defense technology company that makes artificial intelligence-based drones and missiles, to create systems that can protect against drone attacks. The company claims that this will allow it to “quickly synthesize time-sensitive data, reduce the workload of human operators, and improve situational awareness.”
It’s not hard to see why tech companies are interested in participating in the war. The U.S. spends nearly a trillion dollars annually on defense, and cutting that spending remains an unpopular idea. As President-elect Trump has filled his cabinet with conservative tech figures such as Elon Musk and David Sachs, a number of players in the defense technology space are expected to gain significant advantages and potentially displace existing defense companies such as Lockheed Martin.
Although OpenAI prohibits its customers from using its AI to create weapons, there are a number of open-source models that can be used for the same purpose. Add to this the ability to 3D print weapon parts-which is what law enforcement believes the alleged UnitedHealthcare shooter, Luigi Mangione, was doing-and it becomes surprisingly easy to make a homemade autonomous killing machine from the comfort of your own home.
