How AI reads red lights and road signs with a camera

Who can see color?

Radar knows something is ahead and LiDAR knows its shape, but neither knows whether the light is red or green. Only a camera can see color and text.

STEP 1

An image is just numbers

A camera also works in three steps. Step one: an 8-megapixel image is 24 million numbers, three per pixel for red, green and blue, 30 frames every second.

STEP 2

AI learns to recognize things

Step two: computer vision AI learns from millions of images where people have already marked the lights, signs and pedestrians. Once trained, it draws a box around each object with a confidence score.

STEP 3

Two eyes judge distance

Step three is judging distance. One camera can't do it, so the car uses two cameras set apart, like a pair of human eyes. The closer an object is, the more its position differs between the two images, and measuring that difference gives the distance.

LIMITS

The farther, the less accurate

The catch is that distant objects shift by only a few pixels. At 15 meters, one pixel of error is about 1 meter off, but at 60 meters it's already 15 meters off.

COMMON MISTAKES

Easy to trick

Cameras also struggle in the dark and against the sun. In 2020, researchers stuck a strip of tape on a 35 mph sign, and the camera in an older car model read it as 85.