How to find which AI fits your work: write your own test in 4 steps

Write your own test for AI

Last time, we watched AIs in blind matches in the arena. But a general leaderboard doesn't grade your work. Today, you write the test.

EXAMPLE

Lan's online shop

Lan sells traditional dresses online. Every day she asks AI to write product descriptions and reply to customers. The AI at the top of the leaderboard may not be the one that best fits her shop's voice.

STEP 1

Take questions from real work

Step one: pick about 10 questions from the work you do every week. Not trick puzzles from the internet, but real customer messages, real spreadsheets, real reports.

STEP 2

Write down what good looks like

Step two: write down what good looks like before you read any answers. For example: correct price, friendly tone, under 100 words. Score each point from 1 to 5.

STEP 3

Same test, same measuring stick

Step three: give the same test to each AI and grade with the same points. Hide the names if you can, so you're not biased toward the one you like.

STEP 4

Keep the test, grade again

Step four: save the test. Whenever a new AI comes out, run it again and compare scores. Your ten questions say more about your work than any number in an ad.

PART 5

Grading what's hard to grade

But some things are harder to grade than right or wrong: is AI honest, is it safe? See you in part 5.