Trust Calibration, Not Trust Maximization

Almost every brief I've read about an AI feature contains some version of building user trust. It's always framed as a quantity, and the direction is always up.

I think that framing is responsible for a lot of bad interfaces, and once you see why, it's difficult to unsee.

There are two ways this goes wrong and only one of them gets attention.

Under-trust is the one everybody worries about. Somebody doesn't believe the system, checks everything by hand, and eventually stops opening it. The feature is dead and at least you can see the body.

Over-trust is the one that costs money. Somebody believes an output they should have questioned, acts on it, and nothing tells them for weeks. There's no drop in engagement to catch it. Every metric looks fine right up until somebody has to explain what happened.

Why hedging usually makes it worse

The standard response to this is to soften the language. Add a may or a might. Put a note at the bottom saying the system can make mistakes.

It doesn't work, and I'd go further and say it does harm.

The first problem is that it's uniform. The same hedge appears on the answer that's solid and the answer that's guesswork. A signal that never varies carries no information, so people learn to skip it, and they learn to skip it quickly.

The second problem is that a hedge that's always present eventually functions as a disclaimer rather than a warning. We know what happens to disclaimers. Nobody has read a cookie banner since 2019.

A signal that fires on everything is decoration for the legal department.

And there's a third problem that's more subtle. Hedging language applied evenly makes the whole system feel slightly unreliable rather than making specific claims feel uncertain. That doesn't calibrate anything. It lowers the floor under every claim on the page, the true ones included.

What moves calibration

The things that work all have one property in common. They're specific, and they vary.

Show the source, next to the claim

Put it next to the sentence it supports, close enough that checking it costs almost nothing. A panel or a link labeled learn more is too far away to do the job.

Mostly people don't read the sources. It works anyway, because the presence of a reachable source changes the character of the claim from an assertion into something that has been asked to account for itself. And on the occasion somebody does check, the answer is right there, which is when the calibration happens.

Let confidence vary visibly

If a system is more sure of one thing than another, the interface has to show that difference. On the readiness work I did at Oracle we labeled anything the system had inferred rather than computed, and inferred values couldn't become a plan until somebody validated them. It's a small mechanic. It's also the difference between a number and a number you can defend in a review.

Say what it didn't look at

This one is underused and it's my favorite. Stating the boundary of the evidence (the sources checked, the period covered, what was out of scope) does more for calibration than any confidence percentage I've seen. It tells somebody where their own judgement still has to do the work.

Let it refuse

A system that has visibly declined to answer something has told you more about its reliability than a hundred correct answers. The first time a person watches it stop, the character of everything else it says changes.

Designing trust downward

This is the part that gets me odd looks in reviews, so I'll state it plainly. Sometimes the correct design move is to make somebody trust the system less than they currently do.

If a feature is right most of the time, people will generalise to always, because that's what people do. Once they're there, the failures land in the worst possible way: unexamined, and downstream of a decision that has already been made.

So you deliberately introduce friction at the point where being wrong is expensive. A confirmation, an evidence view, a second pair of eyes on anything irreversible. Not everywhere, which would be exhausting and would train people to click through. Only where the cost of a wrong answer justifies the cost of the pause.

Yes, it will make your adoption numbers look worse. It will also mean that when somebody acts on an output, they meant to.

How to know whether you have it

Stop measuring how much people trust the thing. Measure whether their trust tracks its reliability.

In testing, that means putting a wrong answer in front of somebody and watching what happens. Do they catch it? What did they check first? How long did it take? A participant who accepts a bad answer in four seconds has told you something no satisfaction score ever will.

Which is more work than a survey, and it's the only version of this that means anything.