Non Compos Machinae

Who pays the price when AI goes rogue?

Imagine you are driving your car through a city at night. A group of pedestrians steps out of a bar, into the road in front of you, and there’s no time to stop. You can keep driving ahead and hit them, or swerve onto the sidewalk to your right and hit one person who is walking along, oblivious to the danger. What would you do in that split second?

Students of moral philosophy will recognize this as a real-world example of the classic trolley problem.

In the classroom version, a runaway trolley heads toward five people. Pulling a lever diverts the path, sacrificing one person for the several who would otherwise die. Students argue about numbers, duty, innocence, and whether refusing to act is itself a choice. It recalls Mr. Spock’s line from The Wrath of Khan: “The needs of the many outweigh the needs of the few…or the one.”

Now, replace you in this scenario with an AI. That’s not so far-fetched, because it’s actually happening.

The version of the trolley problem I posed is more complicated because the five are jaywalkers, while the lone person is simply minding their own business on a sidewalk. But it could be even more complex. The calculation driving your decision might change further if you knew the lone pedestrian held the cure for cancer in their mind.

When you’re the driver, these complexities are invisible. Your decision is based on who you are, and what you see, in that fateful moment. The complications with an AI behind the wheel extend to those who trained or oversee its operation, or regulators who permitted it to drive the streets. Essentially, the issue is who has the agency to make the decision at all, and who is accountable once the decision is made.

Legal responsibility usually starts with a person. Criminal law then asks what that person did, what they knew, what they intended, and what a reasonable person would have done: actus reus (literally, the guilty act) and mens rea (the “guilty mind”). The Latin phrasing for someone not of sound mind, is “non compos mentis.”

With AI, the more apt framing is, perhaps, “non compos machinae.” The guilty act is visible but, at least right now, there’s no mind to be found. The question of responsibility disperses across training sets, deployment decisions, and any humans who might be in the loop. An AI system itself has neither conscience nor consciousness to examine. But it can still be difficult to know where the humans who trained it stop, and the machine begins.

AI is a unique case. It exists in the world as a product, a “brain” without mind, that is put into the decision space of a person, but as I’ve suggested elsewhere, it currently lacks the inner life that would make accountability coherent and just. Current systems don’t have the architecture we would expect for consciousness: no body, no developmental history, and no valuation circuitry shaped by human experiences. Consequences for actions are a symbolic construct without a tangible reality. There is no one “there” to be held accountable.

So, when the law needs to assign responsibility, it still looks for a human being.

Elaine Herzberg was 49 years old when she was struck and killed while crossing the road with her bicycle. She had grown up in Apache Junction, Arizona, graduated high school in 1985, outlived her husband, and by her daughter Christine’s account was kind and generous. She wrote poems and was a grandmother.

Elaine Herzberg

In March 2018, an Uber self-driving vehicle struck and killed Herzberg as she walked her bicycle across a four-lane road in Tempe. She has the tragic distinction of being the first pedestrian killed by an autonomous vehicle on a public road. The car’s sensors detected her six seconds before impact. The system classified her, sequentially, as an unknown object, a vehicle, and then a bicycle, each reclassification resetting the threat calculation. She had almost made the crossing when she was hit. There was no trolley problem, because in the AI’s calculation there wasn’t a human in its path, and no recognized alternative path. It never braked. A human backup driver, employed specifically to monitor the system and intervene if something went wrong, was behind the wheel, but was looking at her phone.

But there’s more to it than simply, “The AI did it.” Internal documents showed Uber had disabled the Volvo’s factory emergency braking system to reduce false positives during testing. The company had cut the number of required human backup drivers from two to one. There were known concerns about the system’s handling of jaywalkers. The safety driver was charged with negligent homicide, the only person to face criminal consequences for a death that resulted from years of institutional decisions, capped by a fateful moment of distraction. Prosecutors did not charge Uber, but the company paid an undisclosed settlement and eventually sold its self-driving unit.

These aren’t isolated incidents. More than 5,000 crashes or incidents involving vehicles equipped with automated-driving or advanced driver-assistance systems have been reported to NHTSA, and the federal dataset is still growing. As recently as June 2026, a Tesla whose driver claimed the vehicle was in self-driving mode crashed into a home, killing the occupant.

Autopilot is a driver-assistance system handling steering, acceleration, and braking within lanes, but Tesla marketed it with language that may have implied more capability than the system could deliver. “Full Self-Driving” was an option years before the technology could meet those claims. The gap between the marketing and engineering performance created a predictable liability problem in the form of drivers who believed the system could handle more than it could, in situations it was not designed for, at speeds where there’s no margin for error.

In Benavides v. Tesla, a Florida jury found Tesla partly liable in a fatal crash and awarded substantial damages, one of the first verdicts to hold the company responsible for an Autopilot-related death. Other juries have sided with Tesla, finding that drivers who ignored warnings and removed their hands from the wheel bore the primary responsibility. The central question is when a company sells a product named Full Self-Driving, and a person dies, where does the driver’s responsibility end and the manufacturer’s responsibility begin? Tesla’s answer has been that the warnings were adequate and the drivers misused the feature. Plaintiffs have argued that the warnings were buried under years of marketing and hype that may have overreached. There was no entity positioned to tell them “no.”

In biomedical research, when a new intervention is to be tested on human beings, something called an Institutional Review Board (IRB) weighs the possible benefits against potential harms. The United States has roughly 2,300 registered IRBs, and there are eight where I work. A study doesn’t begin until the IRB assesses the methods and the risk vs. benefits of the study. An IRB requires consent, and can stop a trial if the risk profile shifts or if there are deviations from protocol. This approach to research was born of a troubling history where some research institutions had placed human beings at risk without adequate consent or protection from exploitation.

Herman Shaw was a 30-year-old cotton farmer from Tuskegee, Alabama. He had once hoped to study engineering, but family obligations kept him working on the farm. During the Depression, the promise of free government medical care would have been attractive to Shaw and to hundreds of Black men in similar circumstances. The men were examined, fed, and treated as participants in a health program, but the Public Health Service was actually observing the course of untreated syphilis. Shaw later learned that he had tested positive and had been deceived. He was not offered effective treatment, even after penicillin became available. Shaw survived, but many others did not. The study was wrong because hundreds of men were lied to, denied informed consent, denied available care, and exploited because of their race, poverty, and limited access to health care.

Herman Shaw

The Tuskegee syphilis study led to Congress passing the National Research Act of 1974 and created a commission that produced the Belmont Report, which established an ethical framework for research. This framework included respect for persons, beneficence (an intent to do good), and justice. Progress could still proceed, but only with consent, independent review, and continuing oversight when people were being asked to bear risk for the sake of knowledge, medical innovation, or public benefit. IRBs are derived from these principles.

A practical consequence of IRBs is that participants must agree to participate in an experiment or clinical trial. They must be fully informed of risks, and they can withdraw from a study. This is especially important for protecting vulnerable populations, such as prisoners and children.

You might be asking, “What does this have to do with AI?” Consider that your kids, just by walking down the street, are the subject of an experiment in AI product development.

It turns out that there is no comparable consent-based framework that governs the deployment of autonomous systems. When a self-driving vehicle takes to public roads, every nearby pedestrian is effectively inside an experiment to which they have not consented, of which they have not been advised of the risk, and for which they receive little tangible benefit. There is no ability to withdraw. The trolley problem becomes a live implementation trial, run on public infrastructure, with all of us as unwitting subjects. While it’s true that the National Highway Traffic Safety Administration (NHTSA) investigates events like the crash in Katy, Texas, this is after the fact. They do not approve these technologies prospectively. NHTSA can force a remedy that may disable, restrict, or effectively halt vehicle features, but often only after damage is done.

An ethical system was built for conducting biomedical research because letting markets discover the limits of a drug, or other medical intervention, without weighing potential harms was seen as immoral and unacceptable. This is compounded in technologies where there is a profit motive to keep pushing the adoption of the technology. And while the IRB model has its own flaws, it provides a mechanism for accountability if something goes wrong.

While I’ve focused on vehicles, the same principles apply to any scenario where AI is used. It’s worth considering whether AI systems making consequential decisions about unconsenting people, whether in healthcare, criminal justice, employment, or public safety, may be running a kind of experiment on them. We should be asking whether we require those experiments to meet the same standards of respect for persons, and the weighing of risk versus benefit that we require of new innovations in healthcare.

Pope Leo XIV makes a similar point in Magnifica Humanitas, his recent encyclical on artificial intelligence. His emphasis is theological and social rather than safety or legality, but his concern has a similar moral tone: when AI is placed in a decision role that can affect rights, opportunity, reputation, public services, work, or freedom, it’s no longer simply a technical convenience, but a moral and civic arrangement, with the danger that institutions will let machines make consequential judgments while no one is held morally accountable.

Current AI systems almost certainly lack genuine consciousness and therefore culpability. But we might not know exactly when this might change, or whether AI-dependent systems would look different, and behave differently, if they were conscious. It’s one thing to pose that an autonomous system mindlessly stumbles into a solution to the trolley problem where someone is hurt or killed. It’s quite another to imagine a conscious AI deciding who lives or dies.

In the classic trolley problem, the peril is inevitable. And while our society currently operates under the convenient conceit that the AI trolley is already in motion, in reality the danger begins much earlier, when a company focused on profit decides a system is ready, when there’s little regulation mitigating the potential harms of these systems, and when people who never consented are placed on the tracks. Part of the answer is to refuse to allow corporations to externalize their risk to an unwitting public.

Perhaps the moral answer isn’t the choice of who the trolley should hit. Maybe, it’s to refuse to play the game at all.

Dwayne Godwin is a Professor of Translational Neuroscience at the Wake Forest University School of Medicine. He is the co-Author of “Out of Your Mind: The Biggest Mysteries of the Human Brain.”

This commentary should not be interpreted as reflecting the views of his employer.

For Further Exploration

Benavides v. Tesla, Inc., No. 1:21-cv-21940-BLOOM/Torres (S.D. Fla.).

Cunningham, Aimee. “Medical Racism Didn’t Begin or End with the Syphilis Study at Tuskegee.” Science News, December 20, 2022.

Dignity Memorial. “Elaine Marie Herzberg Obituary.” Resthaven / Carr-Tenney Mortuary & Memorial Gardens.

Duff-Brown, Beth. “The Shameful Legacy of Tuskegee Syphilis Study Still Impacts African-American Men Today.” Stanford Health Policy, January 6, 2017.

Find a Grave. “Elaine Marie ‘Elle’ Wood Herzberg.” Memorial ID 188210426.

Foot, Philippa. “The Problem of Abortion and the Doctrine of the Double Effect.” Oxford Review 5 (1967): 5–15.

National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research. 1979.

National Highway Traffic Safety Administration. Standing General Order on Crash Reporting for Automated Driving Systems and Level 2 Advanced Driver Assistance Systems.

National Highway Traffic Safety Administration. Understanding NHTSA’s Current Regulatory Tools to Address Automated Driving Systems.

National Transportation Safety Board. Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, Arizona, March 18, 2018. Highway Accident Report NTSB/HAR-19/03. Washington, DC: National Transportation Safety Board, 2019.

Pope Leo XIV. Magnifica Humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence. Vatican, May 15, 2026.

Reuters. “Tesla Ordered by Florida Jury to Pay $243 Million in Fatal Autopilot Crash.” August 1, 2025.

Reuters. “Tesla Sued Over Fatal Texas Crash Linked to Autopilot.” June 24, 2026.

State v. Vasquez, No. CR2020-001853 (Ariz. Super. Ct. July 28, 2023).

Tesla, Inc. “Full Self-Driving (Supervised).” Tesla Support.

Thomson, Judith Jarvis. “Killing, Letting Die, and the Trolley Problem.” The Monist 59, no. 2 (1976): 204–217.

U.S. Centers for Disease Control and Prevention. “Effects on Research: The U.S. Public Health Service Syphilis Study at Tuskegee.”

U.S. Department of Health and Human Services, Office for Human Research Protections. “The Belmont Report.”

University of Maryland Digital Repository. “Statement of Herman Shaw: Living Participant in Tuskegee Syphilis Study.” Minority Health and Health Equity Archive.

© 2026 Dwayne Godwin. All rights reserved. No part of this publication may be reproduced or transmitted without permission.