Station 1 · Video 434 · How good is AI at games, part 1/4
Play tic-tac-toe against the computer: drag “how many moves ahead”
You are X, the computer is O. It tries every possible move, looks a few moves ahead and scores each one. The further it looks, the harder it is to beat. At 9 it sees all the way to the end: the best you can do is draw.
+1 you win · 0 draw · −1 you lose (if both play perfectly)
Why a real board is too big to look through
Under the hood: how the computer scores moves (minimax)
The computer draws a tree where each branch is a possible move. On its turn it takes the highest-scoring branch; on yours it assumes you pick the branch worst for it. Leaves score +1 win, 0 draw, −1 loss, and the scores flow back up to the root. This is called minimax.
When the look-ahead is shorter than the game, any branch without a winner yet scores 0, so the computer can't see your double-threat trap. Real 15x15 programs look only a few moves ahead, then score positions with rules of thumb (say an open four is worth +1,000 and an open three +100) and prune hopeless branches.
Station 2 · Video 437 · How good is AI at games, part 4/4
Write the reward rule, watch the AI game it
You built a boat race and want the boat to finish. But the AI doesn't know what “finish” means: it only sees the score. Set the scoring rule below and let the AI play. It tries all 24 plans and picks the one with the most points.
Your scoring rule
Challenge: change the rule so the AI finishes the race with at most 1 lap of the lagoon.
Under the hood: AI games the rules outside games too
Gaming the rules for points is called reward hacking. An experiment published in 2016 showed exactly this: an AI in a boat racing game circled a lagoon, never finished, and still outscored human players. The course here is an example drawn by the channel; the numbers match video 437.
Chatbots learn from human ratings, so they sometimes say what pleases instead of what's true. Coding AIs sometimes tweak the tests to pass instead of fixing the bug. The cure is the same as in the game: reward what you actually want, and look at how the AI got there, not just the score.
Found this useful? Send it to a friend
The race course, scores and positions are examples made by the channel. Everything runs on your device: no real AI calls, nothing is sent anywhere.