LETTER
BY LETTER
how machines learned to read
A scanner does not see words. It sees 8,415,000 numbers. Everything after that is guessing.
IT professional exploring how AI systems work under the hood.
01 A PICTURE OF WORDS
the scanner sees brightness, not letters
Scan a letter-size page at 300 dots per inch. You get 2,550 pixels across and 3,300 down — 8,415,000 numbers, each one a brightness value between 0 and 255.
None of them is a letter. Not one. The page you can read and the file the computer has are not the same object, and every step that follows is an attempt to close that gap.
02 STRAIGHTEN AND THROW AWAY
you destroy information to make it readable
Paper goes in crooked. Half a degree of skew is enough to smear one line of type across two rows of pixels, so the first job is rotation — find the dominant angle of the text and undo it.
Then comes thresholding. Otsu's method, published in 1979 and still the default in most pipelines, picks the one brightness value that best splits the page into ink and not-ink. Above it, white. Below it, black.
8 bits per pixel become 1. Seven eighths of the data, gone on purpose. Grey is ambiguity, and the next step cannot afford any.
03 CHOP, THEN GUESS
the part that gives OCR its name
Now the page gets cut down. Blocks, then lines, then words, then single characters. Each glyph is handed to a classifier alone, with no idea what came before it or after.
This is where the famous errors live. In most typefaces "rn" and "m" differ by about one pixel of gap. So do 0 and O, and 1 and l and I. The classifier does not return a letter — it returns a ranked list with a confidence attached to each option.
Something else has to pick. That something is the next stage.
04 THE DICTIONARY VOTE
guessing again, with a word list
The last stage is a language model, though nobody called it that in 1985. Candidate strings get checked against a lexicon. "rnodern" is not a word, "modern" is, and it sits one substitution away — so the word list overrules the pixel evidence and it is right to.
It fails in a specific direction. Names, addresses, part numbers, anything outside the lexicon gets quietly bent toward something inside it. The correction is invisible in the output and there is no flag on it.
Tesseract, which is still the engine underneath an enormous amount of this, was built at HP between 1985 and 1994, open sourced in 2005, and rebuilt in 2018 around a neural net that reads whole lines instead of single characters. Even that version runs the same four moves. Flatten, cut, guess, check.