How good is AI?
AI has finished learning. But how do we know how good it is? Every ad brags: our new AI scored 90%. Where does that number come from?
BENCHMARKS
AI takes tests too
AI takes tests too, like students taking final exams. AI's tests are called benchmarks: thousands of questions with answers ready, and every AI gets the same test.
SCORING
The score is the percent correct
Grading is simple: compare AI's answers with the answer key. Get 900 out of 1,000 right, and the score is 90%.
MANY SUBJECTS
A test for each subject
Some tests cover knowledge in many subjects, from history and medicine to law. Some are math tests, solved step by step. Some are coding tests: AI writes code, and a computer runs it to see if it works.
NOTE
One test, one skill
But each test measures only one skill. Getting an A in math doesn't mean you write well. An AI that's great at coding tests may not write good emails.
READING THE NUMBER
90% on which test?
So 90% on one test can't be compared with 90% on another. The number only means something when you know what the test is.
PART 2
What if AI saw the test first?
But what if AI saw the test before taking it? Can you still trust a high score? See you in part 2.
This article is based on the video AI's exams from the Mark học AI channel. Watch the video (in Vietnamese) to see the animations.