Dancing with the Algorithm: Striking the Balance between Trust and Control in AI Adoption

Imagine a computer that can polish your essay, giving your sentences a new-found sense of meaning - not to mention a re-discovered proper use of grammar - or fix the bug that has been hunting you for days in thousand lines of code. AI promises a revolution for each industry not just in knowledge work. But it has made that promise for decades. Why does AI keep fumbling the ball?

The allure of AI's capabilities often overshadows a critical concern: the lack of control that users face when adopting these intelligent systems. Just as the hype surrounding AI's impact on jobs has sparked debates, another contentious issue emerges from within the AI community itself: the need to build systems that earn trust. Adoption stands as the pivotal factor that either catapults AI to soaring success or leaves it with a broken neck.

If you use LLMs like GPT-3 or ChatGPT, you would receive swift responses, seemingly confident, but not always accurate. Responses are often riddled with uncertainty. At the moment, companies deploy LLMs for the low-hanging fruit: chatbots for internal documentation, writing a company newsletter, and maybe using it to write a whitepaper. If it faces the customer, there remains a human in the loop that controls the output. They avoid deploying it in their customer-facing applications. It’s still too risky, too uncertain. You cannot guarantee that the same user query gets the same response; or a good response at all. If we cannot meet what the user expects, we erode trust.

Uncertainty leads to a feeling of Lack of Control

In the realm of AI, a user's ability to act upon an algorithm's judgment hinges on a fundamental condition: the user must maintain a sense of control and confidence to place their trust in the technology. However, the opposite rings true when the user finds themselves in the dark.

Users won’t give up their control if they face uncertainty from:

• Lack of understanding: users yearn for a glimpse into the inner workings of the technology they rely on. However, we often introduce AI without offering any intuition about its intricate mechanisms. We leave users in the dark, grappling with uncertainty as they try to fathom how the algorithm arrives at its decisions. It's no wonder they become hesitant to place their trust in a system they can't truly comprehend.

• Lack of interaction: It's like being invited to a fancy dinner but finding out you can only observe and not partake in the feast. When users are asked to surrender their control to AI systems without being allowed to offer input, it's as if their agency is locked away in a fortress, guarded by robotic gatekeepers. The absence of an interactive and iterative relationship between the user and the AI system takes away their ability to influence the technology, leaving them feeling like passive spectators in their own digital journey. The absence of an interactive and iterative relationship between the user and the AI system erodes the user's agency, amplifying their reluctance to surrender control.

• Missed expectations: This is the biggest fiasco. Users expect consistency. It's like ordering your favorite pizza, but instead of getting a delicious slice of cheesy goodness, you receive a plate of spaghetti with a side of pickles. When users expect consistency from AI systems but end up with outcomes that diverge from their expectations or, worse yet, encounter system failures, it's as if the trust they placed in the technology gets squashed like a bug under a heavy boot.

When users grapple with the idea of relinquishing control to AI, the fear of uncertainty takes center stage, casting a shadow over their willingness to embrace the technology.

The Path Forward

The path forward requires a careful examination of the relationship between humans and AI. To foster adoption and engender trust, AI systems must not only demonstrate their competence but also afford users a tangible sense of control. It is only when human decision-makers feel empowered, confident, and in command that they can fully place their trust in the judgments of algorithms.

Sources

1. S. Bikhchandani - Learning from the Behavior of Others- Conformity, Fads, and Informational Cascades

2. A. McAfee - Turning Technology Into Business Transformation

3. People + AI Guidebook - User Needs + Defining Success

4. D. Kahnemann - Anomalies The Endowment Effect, Loss Aversion, and Status Quo Bias

5. D. Norman - Design of Everyday things

6. F. Chimero - The Web’s Grain

7. J. Weisz - Toward General Design Principles for Generative AI Applications