The Breakthrough How-To, Part 3

What sort of environment lends itself to breakthroughs? This is a pretty broad question. Environment covers a lot of variables, including many which affect cognition in general. I'll focus mainly on the social environment of breakthrough.

In the previous post, we compared designing an oil rig (a hard but non-breakthrough type of problem) to P-NP (a very hard breakthrough type of problem). Designing an oil rig is hard because it has lots of pieces, but each piece is straightforward. To design an oil rig, we need a large team of engineers. Each engineer needs to work on some straightforward part of the problem. The engineers will need to work together to make sure that their parts are all compatible, e.g. the structure is strong enough to support the pipe and drill motor. Management will be needed to make sure each part gets done and everyone works together smoothly.

Now imagine a similar team working on P-NP. We gather a large team of computer scientists, and management tells them to... um... do something. Yeah, go solve that problem! And the computer scientists sit there staring at each other.

There's an old parable about the wisdom of crowds. According to the parable, a certain emperor was never seen in public. A student wanted to find the size of the emperor's nose, but since the emperor was never seen in public, the student could not measure it directly. So, the student resorted to the wisdom of crowds: the student ran a huge survey of a major city, asked every resident to estimate the size of the emperor's nose, and averaged all the estimates. Of course, none of the people questioned had ever seen the emperor's nose either. Low and behold, the student wound up with a number which had nothing whatsoever to do with the actual size of the emperor's nose.

If nobody's ever seen the emperor's nose, then no matter how many people you survey, you won't get any closer to an accurate estimate its size.

There are a number of morals to that story, but the moral for us is that putting together a lot of people with no information does not create information. If no one has any idea at all how to approach P-NP, then putting a thousand such people in a room will not get you any closer to solving P-NP (no matter how much experience management has).

That does not mean that teamwork is useless for breakthroughs. History does show an abundance of insight by individuals (Isaac Newton formulated most of his ideas during a one-year stint in the countryside). But history also shows that certain kinds of groups make breakthroughs as well. Breakthrough just doesn't come from the kind of teams that produce oil rigs.

Let's consider a simplified, abstract model of a breakthrough. Let's say that our breakthrough involves getting from point A to point D. Anyone with a working knowledge of algebraic topology can get from A to B, a smart geneticist can get from B to C, and some basic but non-obvious high school algebra can get from C to D. One way to make this breakthrough is for a single generalist with a knowledge of both algebraic topology and genetics to sit down and play with the problem for a while. But how might a team make the jump?

Let's say our team is a mathematician and a biologist, each with the requisite skills for the problem. The main difficulty for the team is that the intermediate points B and C don't seem useful by themselves. The mathematician can see the connection A -> B, but B doesn't seem useful to the mathematician. B does seem useful to the biologist, because the biologist can see the B -> C connection easily. But C doesn't seem useful to either of them, at least until they play around with it a bit and realize that it's equivalent to D. In order to make the jumps, the mathematician has to show the A -> B connection even though it seems silly, and the biologist has to show the B -> C connection even though it seems useless, and they both have to play around a bit with C even though it seems irrelevant. After all, if the intermediate steps were obviously useful, someone would have made the connections immediately and the problem would not require any breakthrough at all.

This whole process is socially awkward. We're socially trained not to present ideas that seem useless, and in group discussions such ideas tend to get shot down. This problem was a major focus of Isaac Asimov's 1959 essay on how people get new ideas. Asimov concluded that in order for a group to make breakthroughs, their meetings had to be somewhat silly. By the very nature of breakthrough-type problems, people need to be willing to throw out ideas which may or may not be relevant, which seem silly or unrelated. People need to be willing to sound foolish, and anyone who is "unsympathetic" to the foolishness will quickly kill the mood and destroy any hope of making the requisite connections.

Asimov recommended small groups, no more than five, so that people would not feel the pressure of waiting to speak. He also pointed to a relaxed, pressure-free atmosphere as a key component. He pointed out that being paid for the meeting generally created more pressure, and might be undesirable. Similarly, if one person had a much higher reputation, it could chill discussion. Finally, Asimov believed that such sessions needed to alternate with people going off to think on the problem alone and process whatever came up in discussions. As he put it: "The creative person is, in any case, continually working at it. His mind is shuffling his information at all times, even when he is not conscious of it."

Asimov's ideas about the stifling effects of pressure on group behavior bear a remarkable resemblance to psychological research on the candle problem. The candle problem looks simple: put a few people in a room with a box of matches, some thumbtacks, and a candle. Their challenge is to mount the candle on the wall and light it. Of course, it's not as easy as it seems. Simply sticking the candle to the wall with the thumbtacks invariably fails. Solving the problem requires a tiny breakthrough, a novel use of the materials.

Setup for the candle problem.

Just as Asimov said, adding pressure to the group makes the candle problem harder. A group under time pressure with monetary rewards is less likely to find the solution at all, and takes longer to find it when they do. Similarly, larger rewards result in slower progress, and make people more likely to debate bad solutions rather than find the good solution.

However, there is a way to reliably improve performance in the candle problem. If participants are instructed to first discuss the problem as much as possible WITHOUT actually solving it, then they are far more likely to find the solution. In other words, to encourage breakthroughs, explicitly tell people that they should just try to explore the problem rather than solve it. Then people are much more willing to suggest ideas which don't seem immediately useful. After all, that's what the instructions say to do.