Procrustes turned nobody away. By morning, every guest fit the bed
On the road to Athens there was an inn, and an innkeeper with one iron bed and the friendliest sign in Greece: everyone fits. He kept the promise, every night. Guests who were too short for the bed he stretched. Guests who were too tall he trimmed. Nobody was ever refused, and nobody ever left the shape they arrived.
A machine that learned from examples makes the innkeeper's exact offer. Whoever you are, whatever you hand it, an answer comes back — sized, always, to what it has seen before. It never once says I have not met anyone like you. It has a bed.
Written for ages 12+, and for any adult who has been told the algorithm is biased a hundred times without once being shown the mechanism. Nothing here assumes you know anything. The real engineering words are all at the bottom.
the claim
The machine is not wicked. It is a summary.
When a machine gets somebody's name wrong, or somebody's face wrong, or somebody's voice wrong, over and over, the natural thing to say is that it has something against them. It does not. There is nothing in there to have anything with. No opinions, no grudges, nowhere to keep either.
What is in there is a pile of examples, boiled down. Faces it was shown. Voices it was given. Sentences somebody collected. Every answer it will ever produce is assembled out of that pile, the way every fitting at the inn was done against that one bed.
So this piece is about the least glamorous thing in machine learning, which is also the thing that decides who these systems work for: the pile. Who is in it, who is not, and what happens — silently, confidently, every single time — to the people who are not.
step one
Where the bed's length comes from
Nobody designed the bed. That is the first thing to get right. Procrustes did not sit down and choose a length out of malice; the bed in this story gets built automatically, out of measurements.
Training, underneath all its mathematics, is measuring your guests. Every example that passes through the machine leaves a small dent in it — the same nudging of dials you have met in every piece of this series, if you have read them. Add up a few million dents and the machine now holds a shape: its idea of what a face is like, what a voice is like, what a sentence is like. That shape is the bed.
Which means the bed has no opinions either. It is a diary. It records, faithfully and without comment, who happened to come down the road.
measure travellers and watch the bed decide its own length
Notice what settling means. Early on, one unusual guest could drag the bed noticeably. After a thousand guests, an unusual guest changes nothing — the bed has become very sure of itself. Hold onto that: sureness comes from repetition, not from being right about the next person through the door.
step two
Who walks past the inn
So the pile decides everything. Then everything turns on one question that sounds like accounting and is actually the whole subject: who got into the pile?
Not everyone, and not a fair sample of everyone. A pile is collected, and collecting is work, so piles fill up with whoever is cheap to collect. Text that was already on the internet. Speech from people who own good microphones and speak the languages that dominate it. Faces that photograph well under the lighting photographers already use. The inn does not survey Greece. It measures whoever happens to walk down its one road, because they are the ones who came to the door.
The treacherous part is that the pile can be enormous and still be the wrong shape. A million guests from one road tell you a great deal about that road. About the country, they can be almost perfectly silent — and the machine cannot tell the difference, because from the inside, a million guests feels like everyone.
collect from the road, then collect properly, and compare the beds
The bars never lie about what was measured. That is what makes this hard to catch: every number in the pile is genuine. The dishonesty is not in the data — it is in the road, and the road does not appear in the data.
step three
The fitting
Now the part of the story everyone remembers. A traveller arrives who is nothing like the bed. What does the inn do?
It does not turn them away. This is the detail to burn in: refusal is not on the menu. A machine asked a question produces an answer, always, by construction — the way you met in the earlier pieces, where even a hopeless block of static gets carved into something. Show a speech machine a voice unlike any voice in its pile and it will not say unknown. It will hand back the nearest thing the pile contains, said with its usual composure. Stretched, or trimmed, to fit.
In the older tellings, Procrustes kept two beds — a long one and a short one — and after looking you over he would offer you the one you did not fit. The fit was never the point. The fitting was the point. The machine's version is gentler but has the same shape: the answer you receive was never going to be you. It was always going to be the bed, with your details pressed into it.
And here is the quiet cruelty of the arrangement: for guests near the bed's length, the fitting barely touches them. The system genuinely works, beautifully, for the people the pile knows. Both things are true at once — which is exactly why the people it works for so rarely believe the people it does not.
set your height, then take the bed. try someone the road never carried
Try 170. Nothing worth mentioning happens, and if that were you, you would leave a good review. Now try 128 — a child — and watch what a system built on one road does to a child, briskly, politely, and without the smallest flicker of doubt. The error is not the scandal. The composure is.
step four
The innkeeper's ledger
Procrustes kept excellent books. Guests served: thousands. Fitted successfully: all of them. And the books are accurate, which is the problem worth an entire section, because this is the exact trick by which real systems pass inspection.
When somebody says a machine is 95% accurate, ask the question this whole piece has been building to: 95% of whom? An overall score is an average across everyone tested, so the people the pile knows well — who are most of the test, because the test set usually comes off the same road as the pile — can bury the people it fails. Ninety-nine for the many and sixty for the few averages out to a number everyone feels fine about.
The mistakes are the giveaway. An honest machine's errors fall like rain, a little on everyone. A bed's errors do not. They land on the same people, every time, in the same direction — and the moment you split the ledger by group, that pattern is impossible to miss. Which is precisely why the ledger so rarely gets split.
read the one big number, then read it by group. same machine, same day
This is why the honest habit in this field is so unglamorous: publish the table, not the average. When we test our own speech systems we score every language separately, every time, because an average over languages is the innkeeper's ledger with better formatting.
step five
More guests will not fix the bed
The reflex, once the problem is visible, is the modern reflex for everything: get more data. And it fails in an instructive way, because more arrives down the same road as before. Double a lopsided pile and you have a bigger lopsided pile. The bed does not move. The only thing that grows is the inn's confidence in it.
That is the sinister version of a fact you met at Instrument 01: repetition buys sureness, and only coverage buys correctness. It is also why we argue that “low-resource” is not a property of a language — no language is short of speakers or sentences; it is short of people who went and collected them. A machine fed twice as much of the same road becomes harder to argue with far faster than it becomes fairer — the confidence races ahead while the far villages crawl.
The real fix is exactly as boring as you fear. Somebody has to leave the inn. Walk to the villages the road misses, with a measuring stick and, usually, a budget — because the people a pile is missing are missing for reasons, and the reasons are rarely accidents. Collect there, deliberately, until the pile contains the country. It is slow, it is unfashionable, and it is most of the actual work.
This is not hypothetical for us. Phonetico exists because the world's speech machines were built on roads that do not pass through Addis Ababa — Amharic, Tigrinya and Afaan Oromo are spoken by well over a hundred million people and landed in the world's piles as rounding error. The first job was never clever mathematics. The first job was the measuring stick.
try the popular fix first. then try the one that works
There are cleverer patches — you can count the rare guests extra so each one dents the machine harder, and it helps, the way rationing helps a famine. No patch manufactures information the pile does not contain. Somewhere underneath every real fix, somebody walked the other road.
the trouble
The bed builds tomorrow's road
One more turn of the screw, and it is the one that keeps engineers up at night.
Suppose the machine hears your voice badly — you, personally, because the pile never met anyone who speaks like you. You try it, it garbles you, you stop using it. Perfectly sensible. But the pile of the future is collected from the people who use the machine today. Your leaving made tomorrow's pile even more like the bed and even less like you. The inn is no longer just measuring the road. The inn is bending it.
Once you see this loop you find it everywhere — which shops get recommended, whose faces unlock phones, whose questions the assistant understands on the first try. None of it requires anyone to intend anything. A summary, deployed, becomes an instruction to the world to go on resembling its pile.
And remember the diary from Instrument 01: the pile records yesterday, at best. Ask a machine to picture a doctor and it hands you the doctors of its pile's era, not yours. A bed is always a little antique. Deployed at scale, it is antiquity with enforcement.
the last thing
Theseus checked the bed
The story has an ending, and it is the right one. Theseus — the same Theseus who walked the Labyrinth back in piece one — came down the road, took the innkeeper's measure, and fitted Procrustes to his own bed. The bed was not argued with. It was measured.
That is the whole modern discipline, in one gesture. You do not fix these systems by asking them how they feel; they will report, in perfect good faith, whatever their pile taught them to report. You lay the system on its own claims, group by group: score it on the people it was never shown, publish the table, and make the table — not the average — the thing it has to survive. The field's word for this is auditing, and it is Theseus's move performed with spreadsheets.
And the hopeful part, which is real: unlike most curses in Greek mythology, this one is arithmetic. A bed built out of measurements can be rebuilt out of better ones. It happens slowly and on purpose and usually because somebody insisted — but it happens; we watch it happen, language by language, in our own numbers. So the next time an adult tells you the algorithm is biased, you can ask the two questions that actually locate the problem. Who was in the pile? And can I see the ledger by group? Everything in this piece is behind those two questions, and most people who use the word never ask them.
the glossary
The words the engineers use
| the pile of measured guests | the training data |
| the bed | the learned model — the training distribution, summarized |
| measuring a guest, one dent at a time | gradient descent |
| the road past the inn | convenience sampling, collection bias |
| people the road never carried | underrepresented groups |
| a guest unlike the pile | out-of-distribution input |
| the stretching and the trimming | confident misprediction — pull toward the training mean |
| refusal not being on the menu | models answer everything; abstention must be built deliberately |
| how sure the inn feels | model confidence — learned from the same pile, so miscalibrated off it |
| the innkeeper's ledger | aggregate accuracy |
| grading himself on his own guests | test sets drawn from the training road |
| reading the ledger by group | disaggregated evaluation |
| errors that land on the same people every time | bias, as opposed to noise |
| doubling the pile from the same road | scaling data without changing coverage |
| counting rare guests extra | reweighting, oversampling |
| walking the other roads with a measuring stick | targeted data collection |
| the bed bending tomorrow's road | feedback loops, distribution shift you caused |
| the bed being a little antique | the pile records the past |
| Theseus fitting the innkeeper to the bed | auditing — evaluating a system on its own claims, by group |
One sentence to take away
A machine is a summary of whoever was easy to measure, it never refuses to answer, and its mistakes do not fall like rain — they land on the people its pile forgot, and only a ledger read group by group will show you where.
Every argument about fairness in this field is, underneath, an argument about that sentence. Who was in the pile. Who reads the ledger. You now know to ask both.
the sources
Where these ideas come from
The bed is myth; the ledger is not. These are the studies that measured it. Each note says what the study actually found.
- J. Buolamwini & T. Gebru, “Gender Shades”, FAT* (2018). The study that read the ledger by group for commercial face analysis: error rates near zero for light-skinned men and up to ~35% for dark-skinned women — the same product, working and failing at once, depending on who stood in front of it.
- A. Koenecke et al., “Racial disparities in automated speech recognition”, PNAS (2020). Five commercial recognisers, matched interviews: word error rates nearly twice as high for Black speakers. This is our own field's ledger, and the finding this series' speech pieces are built to answer.
- T. Gebru et al., “Datasheets for Datasets”, CACM (2021). The measuring stick, made practical: every pile should ship with a datasheet saying who is in it, how it was gathered and where it fails — because you cannot fix a bed you have not measured.
- E. Bender, T. Gebru, A. McMillan-Major & S. Shmitchell, “On the Dangers of Stochastic Parrots”, FAccT (2021). The argument at web scale: an ever-bigger pile scraped from the internet does not average its unfairness away — it hardens it, fluently. The most-cited critical paper about size as a substitute for coverage.