Solving the Trolley Problem Won’t Make AI Ethical

Here’s a thought experiment you’ve heard before: You’re on a trolley hurtling uncontrollably down the track towards five people. Luckily, if you pull a lever, you’ll switch to another track. But there’s a complication: there’s someone on that track, too.

Do you pull the lever? Sacrifice the one for the many? Does pulling the lever somehow make you more culpable for the death of that single person?

Here start countless dreary debates among 18-year-olds in philosophy 101 classes. But now, students sparknoting Thus Spoke Zarathustra aren’t the only ones trying to solve the problem. AI engineers are also attempting to find the optimal answer for their artificial decision-making systems.

In both cases, they’re missing the point. The true, original import of the trolley problem has nothing to do with deciding the fates of doomed hypothetical people. Instead, it asks a much more fundamental question. And that deeper question is what we actually need to tackle as we try to build ethical AI.

Machine minds (and machine hearts)

When it comes to AI and machine learning, there’s a lot to talk about. But we’re going to focus on AI in terms of machine learning algorithms that make the choices humans used to oversee. Think automatic resume readers, not Dall-E or ChatGPT. 

In some form, we’ve already had these tools for a while. And we haven’t found them noteworthy. Just think about the stoplight cameras that automatically take pictures of the license plates of cars running through red lights. They take input, apply a set of rules, and produce a tangible result: a fine.

Only now, instead of humans writing the code, systems are figuring out how to achieve their goals themselves. To be fair, this is a significant advancement, and the complexity of the two decision-making algorithms is separated by magnitudes.

And yet. 

Even if the system is finding its path to its goals without overwhelming human intervention, the outcome is identical. Algorithms end up applying strict, preordained rules to whatever input they get, and they produce an outcome. They make decisions in the same manner as the stoplight system does; because they are both limited machines, they confine the range of available decisions to baked-in, predefined possibilities.

In the stoplight setup, the programming says that, in all situations, if someone runs a red light, they deserve a fine. Unlike a human who has a malleable, context-dependent sense of justice, it can’t see that, for example, the driver is a mother rushing her fatally ill child to the hospital. It won’t take into account that she can’t afford the fine just like she couldn’t afford to call an ambulance. It won’t care that no other cars were approaching the intersection. It has one operant strand of logic: if the red light is run, the offender should be punished. 

The machine, in issuing a fine, has made an ethical decision. But it has no inherent sense of ethics. It just has instructions.

Not beyond ethics, but a reflection of them

A programmer wrote the code in the previous example. That person’s understanding of ethics informed how the machine operated. Likewise, AI systems arise within environments informed and shaped by ethics. But for AI, models build themselves. So instead of a person’s explicit ethical values getting hard-coded in, the model integrates the ethics that are implicit in the training data.

Let’s take a concrete example. AI systems are no longer just recommending songs. Now, they are also underwriting mortgages. They intake tons of data to decide who is worthy of credit and how much. In theory, that should be fine. But what if that AI takes location into account? What if it looks into lending practices over time starting with, say, 1934? What if it makes decisions with the assumption that it should follow similar trends?

You can likely see where we’re going with this. In this hypothetical, the system would intake data that people in certain neighborhoods have been historically excluded from mortgages. This is because redlining skewed the data the AI trained on. Redlining was the practice of refusing credit to people who lived in minority neighborhoods. Hot take: that’s pretty unethical. 

But the mortgage-worthiness system would have no way of knowing about this. It would just see that certain people got loans less often, take that as the norm, and perpetuate the discrimination as it followed that unquestioned trend. 

Bias would not be a bug in an AI system. It would be a natural outcome of the social principles latent in the training data.

If we allow the system to train on data that treats unethical practices as normal, we can’t expect that system to make ethical decisions.

The actual trolley problem

And here’s where we return to the question in the intro. The trolley problem has nothing to do with deciding the fates of hypothetical people. When Philippa Foot came up with the thought experiment in 1967, it was to show that we can’t know the exact outcomes of the ethical decisions we make. In addition, it underlined that we have to keep in mind that real-life decisions are always constrained by the context in which they’re made.

She intended the problem to demonstrate that thought experiments like it are, at their core, divorced from reality. After all, it’s not impossible that if you switched the trolley to the track where the one man was waiting, he couldn’t jump away. And that’s just one real-world consideration absent from the thought experiment.

Put yourself in the trolley once again. Really, try to imagine it. Why aren’t there emergency brakes? Why are these people on the tracks in the first place? Why is it suddenly the responsibility of an untrained civilian to redirect a trolley? What series of terrible, macabre, and casually cruel decisions led here? 

If this happened in real life, whether or not you pulled the lever wouldn’t prove that you were ethical or unethical. In all likelihood, you’d panic and make a snap decision. And no amount of debating consequentialist vs. deontological ethics would save you from needing years of therapy after having been made the unwitting instrument of negligent homicide.

Similarly, trying to figure out which decision an AI should make in a similar situation won’t make it ethical.

Put a machine at the lever or put a person at the lever — in either case, it’ll be nothing but a tragedy that might have been prevented.

What we can do about it

So, for AI, we answer the problem before the trolley leaves the station. We answer ethical questions before we train algorithms: Is it right to always fine people for running red lights? Is it okay to perpetuate housing discrimination? Then, after the AI model establishes its own decision-making logic, we examine the results and see if they match our intentions. 

We do not assume that these machines are beyond ethics. We assume that they are deeply, inextricably infused with ethics, and we interrogate these latent values over and over and over. 

We do that better by having more people adding their unique subjectivity to these discussions. People who think differently need to come to a consensus on how these machines should operate. 

And we’re talking about radically diverse coalitions — disabled people, undocumented people, LGBTQ+ people, women, men, white, brown, black, indigenous. Think of a person — we need their voice.

It’s utopian. It’s unrealistic. We know. But it’s one way we can attempt to make sure that, as we build AI systems that increasingly make decisions for us, they do so in a way that we consider ethical.

How we do that

Many articles talking about tech and practical ethics tend to end with a shrug, saying, “More inquiry is needed” or, “We must continue to examine this as it develops.”

We’re not going to do that. 

We need more diverse voices in tech. So, we have bootcamp scholarships for underrepresented groups. It’s not the only solution. It’s just a start. 

If your voice isn’t present in the rooms where AI is being discussed and built, we encourage you to check out our Data Science bootcamp to start to change that. 

The people behind these AI systems will be the unwitting ethical arbiters of the future. We have to get that right, so the tech industry needs people like you.