NO LETTERS
REQUIRED
what replaced the pipeline
A vision model has no character step. It never finds a letter, never cuts a word, never checks a dictionary. It reads the way it writes. By guessing what comes next.
IT professional exploring how AI systems work under the hood.
01 WHERE THE PIPELINE BREAKS
step three assumed a clean line of type
Every move in the classic pipeline rests on one assumption: that the page holds straight rows of separated characters. Most real documents do not.
A handwritten note has no character boundaries. A two-column page has no single reading order. A table's meaning lives in position, not sequence. A stamp across a signature line puts two marks in the same pixels.
None of that is an edge case. That is most of the paper in the world.
02 PATCHES, NOT CHARACTERS
an image is worth 16×16 words
The 2020 paper that started this is called "An Image is Worth 16×16 Words." The title is the method. Take a 224 by 224 image, cut it into a 14 by 14 grid of 16-pixel squares, and you have 196 patches. Flatten each into a vector, hand the sequence to a transformer.
Nothing in that describes a letter. There is no segmentation, no classifier, no word list. The grid is indifferent to what is drawn on it — a patch may hold half an "m", part of a coffee ring, or nothing at all.
That indifference is the whole trick. A model that never looks for character boundaries cannot be broken by their absence.
03 READING IS PREDICTION
it does not recognise, it continues
What comes out is generated, not extracted. The model emits one token at a time, and each token is conditioned on two things: the patches, and everything it has already written.
That is the same machinery behind any chatbot reply. Same tokens, same sampling, same next-token objective. The image is context, not a source of truth.
So the output is a plausible continuation of the image. Usually that is the same thing as a transcription. Usually.
04 THE NEW FAILURE
plausible is not the same as present
Here is the trade, stated plainly. Give both systems a receipt with a smudged total.
Classic OCR returns 1,2█8.OO. Visibly broken. You know not to trust it, because it looks like what it is.
A vision model returns 1,258.00, in clean digits, with no hedge. It filled the gap with the most likely number, which is exactly what it was built to do, and nothing in the output records that a gap was there.