Four Parables, One Lesson: The Broken Chain Problem

The Emperor’s Nose (Richard Feynman)

A village in a remote corner of an empire decided to erect a statue of the emperor. The village sculptor was hired, but the sculptor had no idea how large to make the emperor’s nose.

The elders conferred. Nobody in the village had ever seen the emperor, nor heard anything about his nose, so they all had wildly different estimates of the size of the emperor’s nose. The elders argued about the right size for hours, until one particularly wise elder stood to address the room.

“Our estimates are too noisy,” the wise elder declared, “In order to improve them, we should employ the wisdom of the crowd. We will ask each person in the village to estimate the length of the emperor’s nose. Then, we can average together the results to obtain an estimate of high precision.”

This proposition was put to a vote, and the elders quickly agreed, eager to end the hours of argument before they missed the early-bird special at the village buffet. The next day, each villager was asked to estimate the length of the emperor’s nose, in millimeters. The elders averaged together all the responses, and estimated the emperor’s nose was 15.49 mm long.

The Cargo Cult (Feynman again)

During World War II, many remote pacific islands became military bases for the American navy. On one such island, the native inhabitants worked with the sailors in exchange for food, clothing and other supplies. All these supplies were flown in by plane, and landed on an airstrip.

After the war, the Americans left the empty airstrip behind. The planes stopped delivering supplies.

The natives wanted to get more supplies. So, they tried to make the planes land. 

They went out to the airstrip and did everything the sailors had done. They lit small, regularly spaced fires along the sides of the airstrip. They had someone sit at the side of the strip talking into a wooden box while wearing pieces of wood shaped to look like headphones. They had others stand on the airstrip and gesture with sticks. In short, they did everything they had seen the sailors do.

But the planes just didn’t land.

The Missing Quarter (Boy Scout tradition)

A boy was pacing back and forth at night under a streetlamp, apparently searching the ground, when another boy passed by.

“What are you looking for?” asked the newcomer.

“My quarter. I dropped it,” replied the searcher.

“Oh. I’ll help you look,” offered the newcomer.

The two continued the search for a minute or so before a third boy came along.

“What are you two looking for?” asked the third.

“A quarter he dropped,” replied the second, indicating the original boy.

“Let me help,” said the third, and set to searching.

This continued for some time, and the crowd grew steadily. Finally, a girl showed up.

“What are you all looking for?” she asked.

“My quarter. I dropped it,” replied the original boy.

“Well where did you drop it?” asked the girl.

“Over there,” said the boy, indicating an area off to the side.

“So why is everybody searching over here?” asked the girl.

“Because there’s more light here,” replied the boy.

The Soviet Nail Factories (Historical/Folklore)

The soviets' central economic planners regularly set targets for each factory under their control. Factories which exceeded their targets were rewarded in various ways. Factories which fell short of their targets… well, it’s the soviets, you can figure it out.

Early on, nail factories each had to produce some number of nails to meet their target. For a while, things went well. Factories produced nails. But there was always an element of competition - the best-performing factories received rewards and the worst-performing were punished, so occasionally people would cut edges in order to get ahead.

In particular, the nail factories found that they could gain an advantage by producing slightly smaller nails than the competition. By producing smaller nails, they could produce a larger number with the same resources. But over time, all the nail factories figured this out, and they had to cheat a little more to gain an edge - the nails became even smaller.

This arms race continued until each factory was producing large numbers of tiny, useless “nails”, better suited to pinboards than to construction.

The central planners heard reports of the tiny nails. They decided to update their targets - henceforth, nail production would be measured by weight, rather than number of nails.

A few years later, all the nail factories were producing just a few giant, useless “nails”, better suited to ballast than to construction.

The Lesson: Don’t Pull a Broken Chain

In everyday life, things are connected by chains of cause and effect. 

Suppose I’m driving along at night when a deer wanders into the road ahead. Light from my headlights reflects off the deer’s hide, into my eyes. The light is absorbed by photoreceptors, which trigger a cascade of electrical signals in my brain. My brain pattern-matches what it sees, and concludes that there’s a deer ahead and hitting it would be bad. The chain of cause and effect links the deer in the road, to me realizing there’s a deer in the road.

Once I realize there’s a deer in the road, electrical signals propagate down my spine to neurons in my leg and foot. Those neurons activate muscles, lifting the foot from gas to brake pedal and then pushing. That force depresses the brake pedal, which applies pressure in a hydraulic system, multiplying the force and eventually squeezing disks connected to the wheels. The increased force on the disks increases friction, slowing the wheels, which in turn slows the car. The chain of cause and effect links my decision to brake, to the car slowing down.

In everyday life, we pull on chains of cause and effect, either to gain information or to influence the world around us. But in each of the four parables above, the chain is broken.

In the story of the emperor’s nose, the elders try to estimate the nose length using statistical techniques… but none of the townspeople know anything at all about the emperor’s nose, so the causal chain from the actual emperor’s nose to the elders’ estimate is broken.

In the story of the cargo cult, the locals mimic the surface actions of sailors at an airstrip, but they don’t understand the underlying chain of cause and effect which led planes to land. Absent that underlying chain, the planes don’t land.

In the story of the missing quarter, the first boy searching under the light causes the second boy to search under the light, and the third, and so on. But the first boy is searching in the wrong place - the chain is broken at the very beginning, even before the story starts. In fact, the first boy himself is pulling on a broken chain: light is helpful for searching, because the light might bounce off the quarter and into the boy’s eye etc. But if the light will never bounce off the quarter - because the quarter isn’t under the light - then that chain is broken.

The Soviet nail factory is the most complicated story. In a normal economy, a nail factory produces an economically valuable nail. That nail is sold, and the nail will only be bought if it’s valuable to the buyer (and the more valuable it is to the buyer, the more the buyer is willing to pay for it). The money from the buyer goes back to the nail maker, and serves as incentive. This chain of cause and effect runs from the nail makers producing an economically valuable nail, to the nail makers being rewarded for whatever value the nail provided for the end user.

But once the central planners step in and set targets in terms of number or weight of nails produced, the chain is broken: the nail makers are no longer rewarded based on the economic value of the nail to its end user. So naturally, the nail makers deprioritize economic value in favor of number or weight of nails. (This is a standard example of Goodhart’s Law: when a measure becomes a target, it ceases to be a good measure. Goodhart’s Law itself is is a special case of the broken chain problem.)

To summarize the lesson: don’t pull a broken chain. When you want to gather information, make sure that the thing you want to know about is causally connected to the thing you’re looking at directly. When you want to influence the world around you, make sure that your action is causally connected to whatever you want to influence. If the causal chain is broken, don’t pull it.