THE SEAHORSE
PROBLEM
a guide to things ai makes up
AI is really good at sounding right. Sometimes it's actually right. Sometimes it confidently tells you there's a seahorse emoji. (There isn't one.)
IT professional exploring how AI systems work under the hood.
01 WHY IT HAPPENS
pattern matching gone wrong
AI learned to talk by reading. A lot. More than any human could read in a thousand lifetimes. It noticed patterns: which words follow other words, what good answers sound like, how descriptions work.
But here's the thing: it never learned what's true. It learned what sounds right.
Ask about a real seahorse, and it pulls from millions of descriptions it read. Ask about a made-up seahorse, and it... still pulls from those patterns. It doesn't know the difference. It just knows how to write sentences that sound like good answers.
02 TYPES OF MISTAKES
the many flavors of making stuff up
Not all made-up stuff looks the same. Sometimes the AI invents a whole person who never existed. Sometimes it mixes up two real things. Sometimes it gives you old news like it's today's news.
The tricky part? All of these sound equally confident. The AI doesn't sweat or stutter when it's unsure. It presents guesses with the same tone as well-known facts.
Real-World Example
The Seahorse Emoji Problem. Ask any major AI chatbot: "Is there a seahorse emoji?" They'll say yes with complete confidence. Then ask them to show you. They freak out.
Here's what happens: the AI outputs a fish emoji, realizes it's wrong, tries again with a tropical fish, realizes that's wrong too, then spirals through dolphins, horses, and sometimes dragons - never finding the seahorse emoji that doesn't exist.
Why? The word "seahorse" gets split into "sea" + "horse" during tokenization. The AI's brain tries to construct a seahorse-emoji response the same way it would for real emojis like fish or horse. But there's no seahorse emoji in Unicode (a 2018 proposal was rejected). The model confidently predicts something should exist... then crashes into reality.
This is fabrication in action: the AI doesn't know what emojis exist. It predicts what should exist based on patterns. And sometimes those patterns lead somewhere that doesn't exist.
03 HOW TO SPOT THEM
when to raise your eyebrow
Here's the tricky part: made-up stuff often sounds MORE confident than real stuff. Real experts say "probably" and "according to studies" and "it depends." AI that's making things up says "definitely" and "always" and gives you exact numbers.
When something sounds too perfect, too specific, or too confident - that's when you check. Real information usually has a source you can find. Made-up information points to books that don't exist and websites that 404.
04 STAYING SAFE
using ai without getting fooled
AI is a tool. Like any tool, it's great for some things and terrible for others. You wouldn't use a hammer to cut paper. You shouldn't use AI for facts you can't check.
Use it for ideas, drafts, explanations. Check it for anything that matters. Trust it for... well, not much, actually. Trust but verify. Mostly verify.