A complaint
A customer complains: the chatbot said transfers were free, but I was still charged. How does the tech team trace what happened?
LOGGING
One line per question
With logs, a written record. Every question leaves a line in the log: the time, the question with private details hidden, the answer, how long it took and how much it cost.
TRACING
One thread through every step
But one question goes through many steps: the gate in, finding documents, calling the model, the gate out. A trace ties those steps into one thread, so you can see what each step received and returned.
FOUND IT
Where the mistake was
Following the trace, they found the problem: the document search step pulled last year's old fee table. AI didn't make it up, it read the wrong document. They fix that exact spot, then add this question to the eval set from part five.
OBSERVABILITY
Seeing clearly inside
Being able to see clearly inside a system like this is what engineers call observability.
PRIVACY
Logs must be kept private too
But logs also contain what customers said. So private data has to be hidden, only some people can view them, and they're deleted after a while.
NEXT
Knowing before things go wrong
Tracing problems after the fact is good. Knowing before they happen is even better. The last part: monitoring.
This article is based on the video Logging and tracing: AI's black box from the Mark học AI channel. Watch the video (in Vietnamese) to see the animations.