Why People Distrust AI - Algorithm Aversion

Salespeople barely glance at the created forecasts and dashboard. Marketers shrug off all the recommendations. Doctors hastily push away the AI-diagnosis. But why?

May 03, 2023

It’s algorithm aversion.

People are reluctant to use advice or recommendations generated by algorithms, even if it’s better than humans. And despite the hours of work put into it.

Cassandra was the daughter of King Priam and Queen Hecuba of Troy. She was a beautiful and intelligent young woman. Cassandra was also gifted with the ability to see the future, but there was a twist. Her gift came with a curse: nobody would ever believe her prophecies.

Despite this, Cassandra continued to try and warn her people of the dangers that lay ahead. She predicted the fall of Troy and the Trojan War, but nobody listened. She warned her family and her fellow citizens about the Trojan Horse, but they dismissed her as a madwoman.

She could see the future with startling clarity, but no one would listen to her warnings. She was doomed to watch as the city of Troy fell, as her family and friends were slaughtered, and as the Greeks sacked and burned the city to the ground.

When the Greeks finally conquered Troy, Cassandra was captured by Agamemnon, the leader of the Greek forces. She foresaw her own death and that of her captor, but again, nobody believed her. She was taken as a slave to the city of Mycenae, where she was eventually killed by Clytemnestra, Agamemnon's wife.

The power of knowledge is meaningless if it isn't believed. The most sophisticated algorithms and tools are pointless if they aren't adopted.

Why aren’t people using the algorithms?

Imagine you are presented with two options: a recommendation from a human expert or a recommendation from an algorithm.

The human recommendation might come with biases or errors, but it feels familiar and trustworthy because it comes from someone you can see and relate to.

The algorithm recommendation might be more accurate and unbiased, but it feels foreign and unrelatable because it comes from a machine.

We all have a natural inclination toward human decision-making. We feel more comfortable with the idea of a person making a recommendation because we can relate to their thought process and understand their motivations.

Algorithms operate on a level that is beyond our comprehension. We can't peek inside an algorithm and see how it works, so it feels like a black box to us. Even if an algorithm is highly effective and efficient, it may fail if the end-users are uncomfortable or familiar with its usage.

Even worse. The consequences of a machine's mistake are unforgivable. If a user entrusts an AI and it fails, the damage done to their trust is often irreparable. The aversion deepens. They immediately default back to their old way of doing things. Where we relate to the fallibility of humans and empathize with their intentions we spot a flaw of the design or programming in the AI.

People tend to distrust algorithms because they can't comprehend their inner workings, feel like they're losing control over their actions and decisions, and long for the pleasure of talking to another person instead of a machine.

Case-Study: AI in Healthcare

Healthcare providers have been quick to investigate Artificial Intelligence (AI) as a potential tool to improve patient care and optimize treatment outcomes.

AI has been applied in various areas such as medical image analysis, drug discovery, and clinical decision support systems. However, despite its potential benefits, adoption only creeps forward.

Patients look skeptically at the treatment recommendations and try to get a “second opinion”, doctors are wary of AI diagnosis and rather rely on their own gut and intuition, and healthcare organizations stockpile customer data instead of using it.

People instinctively dismiss the advice of computers when it comes to making decisions related to their health, assuming that nothing can replace the expertise of a human doctor.

Even when shown evidence of a computer's superior performance in comparison to a real physician, patients would still rather have a human provide them with care.

But what is the cause for this algorithm aversion?

Patients believe AI cannot meet their individual needs. If they have to choose for themselves, a friend, or another person perceived as 'unique', they choose a doctor. But they will side with a machine if it's for an “average person”.

Make AI approachable

As developers, designers, and product managers, it's our job to not only create effective solutions but also make them approachable, easy to use, and understand.

We want our users to break their natural inclinations not to use the AI and default to their old behaviors.

Let’s see how the biggest AI tools have gotten their users to use them.

Today’s most successful companies are all powered by AI. But they don’t highlight it. They use AI to enhance and improve their offerings in a way that is seamless and intuitive for the user.

  1. Amazon: The e-commerce giant uses AI to power its product recommendations, search results, and voice assistant Alexa. However, these features are a part of the experience and not the core of it. They are so seamlessly integrated into the user experience that most customers don't even realize they are powered by AI.

  2. Netflix: The streaming service uses AI to personalize content recommendations for each user based on their viewing history, and preferences. Most users simply accept them as part of the platform. AI powers their core product and solves the customer’s problem but behind the scenes.

  3. Google: The search engine giant uses AI to power its search algorithms, as well as other services like Google Maps, Google Photos, and Google Assistant. However, these features are so integrated that most people don't even think about the fact that they are powered by AI.

This means, focusing on the problem you are solving, and if AI can help with that, add it to your solution.

If this isn’t an option, demystify the algorithm. Make the decision-making process transparent and explain how the algorithm arrived at its recommendation. Help people feel more comfortable with the technology.

In the end, you will never come around to building trust through repeated successful interactions. If the algorithm consistently delivers, people will begin to trust it over time.

Interventions

  1. Focus on the problem you are solving. Then think about how to solve it. That’s the first point where you should even consider AI.

  2. Think about how you can integrate AI seamlessly into your products. Let the customer build trust with the tool without even realizing he is using AI.

  3. If this isn’t possible. Demistify the algorithm. Give explanations. Give examples.

Sources

Reply

or to participate.