Ten at night
Ten at night, the on-call engineer's phone buzzes. An alert from the monitoring system.
DASHBOARD
The tracking board
The dashboard shows numbers called metrics: response speed, cost per hour, refusal rate, and how often customers click "not satisfied".
ALERT
Cross the line, the alarm rings
Each metric has a red line. When the needle crosses it, an alert rings, and the person on call takes a look right away.
TONIGHT
What happened?
Tonight, the "not satisfied" rate tripled. The engineer opens the logs and follows the trace, as in part six. It turns out the bank just launched a new card, customers are asking about it, and the documents don't cover it yet.
DRIFT
The world changes, the system stands still
That's drift: the world changes, users ask new kinds of questions, but the system stays the same.
FEEDBACK LOOP
Incidents become test questions
After the fix, they add questions about the new card to the eval set. The loop closes, called a feedback loop: incidents become lessons, and lessons become test questions.
RECAP
The whole series in one picture
The whole series in one picture: cloud, input guardrails, output guardrails, evaluation, logging, monitoring. Every two-second answer has a whole assembly line behind it.
This article is based on the video Monitoring: AI's control room from the Mark học AI channel. Watch the video (in Vietnamese) to see the animations.