Is 95% right good enough?
An AI that's right ninety-five percent of the time sounds great. But one time in twenty, it's wrong. Product makers have to plan ahead: where does that mistake land?
A four-step ladder
Picture a ladder with four steps, from AI only suggesting to AI doing everything itself.
STEP 1
Suggest
Step one: suggest. Gray text appears as you type. Tap to accept, ignore it if you don't want it. If AI is wrong, no harm done.
STEP 2
Draft
Step two: draft. AI writes the whole letter, and you read it, edit it, then send.
STEP 3
Ask to confirm
Step three: AI gets everything ready, but before paying or booking, it asks: "Do you confirm?".
STEP 4
Do it alone
Step four: AI just does it, like a spam filter. This only fits when mistakes are easy to undo: an email filtered by mistake can still be recovered.
THE RULE
Hard to undo, stay low
The rule is: the harder a mistake is to undo, the lower the step should be. Plus three good habits: show sources, let users correct things, and be willing to say "I'm not sure".
PART 4
Measuring an AI product
It's designed and launched. But how do you know the product is working well? See you in part 4.
This article is based on the video Designing for a machine that's sometimes wrong from the Mark học AI channel. Watch the video (in Vietnamese) to see the animations.