Easy to Trust Isn't the Same as Trustworthy

As AI becomes part of everyday software, designing experiences that feel intuitive is only part of our responsibility. We also need to help people recognize when to question the system.

I've been thinking about the relationship between good design and trust.

For most of my career, I've worked to make complex software easier for people to understand and use. We research what people need, simplify complicated workflows, make information clearer, and help people feel confident in the decisions they're making.

That's a big part of what good UX should do.

But AI is introducing an interesting tension.

What happens when we create an experience that's so clear, polished, and easy to use that people trust it more than they should?

We're increasingly designing products that don't simply follow predefined rules. They interpret information, make recommendations, and sometimes take action on someone's behalf.

And while those capabilities can create tremendous value, they also introduce uncertainty that isn't always obvious to the person using the product.

We can make something incredibly easy to trust without necessarily making it trustworthy.

I think that's a distinction worth spending more time on.

When Good Design Creates Confidence

One of the things we work hard to do as designers is remove uncertainty.

We use clear language, thoughtful visual hierarchy, consistent interactions, and predictable behaviors to help people understand what's happening.

When an experience feels polished and intuitive, people naturally become more comfortable using it.

That's generally a good thing.

But imagine an AI assistant recommending which maintenance activities a facilities manager should prioritize.

The recommendation is clearly presented. The reasoning sounds sensible. There's a prominent button to accept the suggested plan.

Everything about the experience feels thoughtful.

What the facilities manager may not realize is that the recommendation was based on incomplete information.

Perhaps the system didn't recognize a regulatory requirement. Maybe a technician knows something about the equipment that isn't captured in its maintenance history.

The recommendation might look perfectly reasonable and still be wrong.

And the quality of the interface could make that mistake harder to recognize.

That's the part I find interesting.

We've spent years learning how to help people feel confident using software.

Now we also need to think about when that confidence is justified.

Not Every Decision Deserves the Same Level of Trust

I don't think the answer is to make AI experiences feel less reliable or surround every recommendation with warnings.

That would quickly become frustrating.

Instead, we need to understand the context of the decision.

If an AI assistant recommends reorganizing my calendar, the consequences of a bad suggestion are relatively small.

If it recommends postponing maintenance on critical equipment, that's a very different situation.

The information someone needs, the level of explanation that's appropriate, and the opportunity to intervene should reflect what's at stake.

This is where I think human-centered design becomes particularly important.

We need to understand the person's responsibilities, expertise, environment, and the decisions they're accountable for.

What do they need to know?

What would make them question a recommendation?

What information might they have that the system doesn't?

And what happens if they follow the recommendation and it turns out to be wrong?

Those questions go beyond making the interaction easy.

They're about helping someone make a good decision.

Human Oversight Needs to Be Meaningful

We hear a lot about keeping humans in the loop with AI.

I agree with the principle, but I think it's worth asking what that really means.

If someone is expected to approve an AI recommendation, do they have enough information to evaluate it?

Can they understand what the system considered?

Can they recognize when information might be missing?

Can they disagree, correct the system, or choose a different course of action?

An approval button doesn't necessarily create meaningful oversight.

At the same time, requiring people to approve every small decision can create unnecessary work and undermine the value of automation.

So we need to be thoughtful about where human judgment matters most.

That requires product, design, research, engineering, and domain experts to work together.

We need to understand which decisions can be safely automated, which benefit from human involvement, and which require stronger safeguards.

The goal isn't to keep people involved in every interaction.

It's to make sure they can exercise judgment when it matters.

What Happens When the System Gets It Wrong?

Another area I think deserves more attention is recovery.

We've always designed error states and considered exceptions. That's not new.

But intelligent systems can introduce different kinds of problems.

A recommendation might be plausible but incorrect. A system might act on incomplete information. An automated decision might create consequences elsewhere in a workflow.

In complex enterprise software, those consequences can extend beyond the person who initiated the action.

Imagine the facilities manager accepts a recommendation to postpone maintenance.

Later, someone discovers that the work shouldn't have been delayed.

Can they understand what happened?

Can they see why the system made that recommendation?

Can they correct the decision?

And can they identify anything else that might have been affected?

Sometimes the most important part of an experience happens after something goes wrong.

We spend a lot of time trying to make interactions seamless. But I think we also need to make sure people can recognize problems, understand their options, and recover without creating more problems along the way.

A trustworthy experience isn't one that never makes mistakes. It's one that helps people understand and manage them.

Designing for What Could Go Wrong

I don't want this conversation to become an argument for slowing innovation.

I'm excited about what AI can make possible, particularly in enterprise software where people manage complicated processes, large amounts of information, and decisions with real operational consequences.

There are tremendous opportunities to reduce repetitive work and help people make better decisions.

But I think we need to expand how we evaluate the experiences we're creating.

We should absolutely test whether someone can accomplish their task.

We should also explore what happens when the system makes a recommendation based on incomplete information.

We should measure whether an AI capability saves someone time.

We might also need to understand whether they can recognize when a recommendation deserves closer attention.

And when we're validating an experience, perhaps we should deliberately introduce conflicting information, unexpected outcomes, or situations where the AI gets something wrong.

Not because we're expecting failure.

Because understanding how people respond to those situations is part of understanding whether the experience actually works.

Trust Is Something We Have to Earn

I've always believed good product design starts with understanding people.

Their goals, responsibilities, environments, and the challenges they face.

That hasn't changed.

What is changing is the nature of the systems we're asking people to use.

As software becomes more intelligent, we're asking people to rely on recommendations and decisions they may not fully understand or be able to predict.

That creates an important opportunity for UX.

We can help make those systems understandable. We can design experiences that respect human expertise, communicate uncertainty appropriately, and make it possible to question or correct the system.

We can help teams think beyond whether a recommendation is useful to whether someone can use it responsibly.

And we can help ensure that the confidence created by a good experience is supported by the behavior of the product itself.

Because ultimately, our job isn't simply to make technology feel trustworthy. It's to help create experiences that deserve people's trust.