Ever tried. Ever failed. No matter. Try again. Fail again. Fail better.
– Samuel Beckett
I like Tim Harford.
This lecture was an overview of the concepts in his book Adapt. He uses “biology, statistical physics, psychology and of course economics to explore how complex problems are solved, and the crucial role of learning from our apparently endless ability to screw up“.
(You can see him giving the same lecture here.)
Failure Is Productive If You Learn From It
He started by describing the brilliant Toaster Project. A chap called Thomas Thwaites decided to build a toaster – the cheapest £3.99 Argos toaster – from first principles. And I mean first principles. He tried to smelt iron ore, make the plastics, and so on. He found it was really very difficult, but after trying hard, and failing lots, making use of short cuts, and help from other people, he built something that approximated a toaster. It kind of warmed the bread a bit, and then caught fire.
The point Harford makes from this story is that by admirable trial and error Thwaites got to a solution. He then used it to point out how useful markets are. A producer does not need to think about how its product be used; the miner doesn’t know whether his iron ore will go on to be used for a toaster or for a fence.
Complex systems like this have evolved over time with trial and error and often nobody has a full view of how they work. As such they’re not going to be perfect. He draws an analogy with evolution.
Harford described another example. Unilever created a detergent that is sold in small capsules. To produce these capsules the raw material is squirted through a special nozzle. To design the nozzle they didn’t get experts in fluid dynamics or other disciplines, rather they prototyped different shapes, recording the outcomes, then refining the shape based on their observations until they had a nozzle that works. They don’t know how it works, but they have proved that it does.
This example draws out the trial and error aspect of his approach to complex problems. Here the Unilever boffins experimented. We tend to over value expertise, and while we shouldn’t ignore that, trial and error is more germane for difficult problems or complex systems.
A key to ensuring trial and error works is the feedback loop: the information that is learnt from failure must be used to refine the next iteration. If this informational loop is hindered then failure looses its power as a learning process.
He discussed the issues that people have adopting this approach. The ability to fail is not always seen as a virtue. Alas this attitude means that innovation can be stunted.
He then spent a lot of time talking about our tendencies to conform to the group we’re in. He described some experiments such as this brilliant video of how people can be made to conform in a lift. He then focusses on the well-known conformity experiments of Solomon Asch.
Asch “asked groups of students to participate in a “vision test”. In reality, all but one of the participants were confederates of the experimenter, and the study was really about how the remaining student would react to the confederates’ behaviour.
Each participant was put into a group with 5 to 7 “confederates” (people who knew the true aims of the experiment, but were introduced as participants to the naive “real” participant). The participants were shown a card with a line on it, followed by another card with 3 lines on it labelled A, B, and C. The participants were then asked to say which line matched the line on the first card in length. Each line question was called a “trial”. The “real” participant answered last or next to last.
Asch hypothesized that the majority of participants would not conform to something obviously wrong; however, when surrounded by individuals all voicing an incorrect answer, participants provided incorrect responses on a high proportion of the questions (32%). Seventy-five percent of the participants gave an incorrect answer to at least one question.
The unanimity of the confederates has also been varied. When the confederates are not unanimous in their judgement, even if only one confederate voices a different opinion, participants are much more likely to resist the urge to conform (only 5–10% conform) than when the confederates all agree. This finding illuminates the power that even a small dissenting minority can have. Interestingly, this finding holds whether or not the dissenting confederate gives the correct answer. As long as the dissenting confederate gives an answer that is different from the majority, participants are more likely to give the correct answer.”
So if you’re in a group, rather than do the usual conforming to keep the peace and fit in, it’s worth speaking up to give an alternative opinion. Even if your view is rubbish you’ll at least be emboldening another member to put forward their idea, and the group is therefore more likely to come to a more reasoned conclusion, as the prevailing view will be challenged. All ideas should be tested. If they fail they are not up to the job, if they pass muster they are proved to be worthy.
At the end of the lecture it was asked whether it is worth using this approach given the cost of failure? How much should we experiment?
It depends on the upside and downside. Someone like Google should experiment lots, the more experimentation the better, whereas a car manufacturer should experiment less as we need cars that don’t kill people! There is a cost-benefit analysis required here: what is the gain given the failure rate?
Given mistakes are inevitable it’s best to think about the best way to make those mistakes. For example we can make them sooner in the process using trials and pilots. It’s important to manage the risk that comes from failure.
We should have the cojones to ask for honest feedback about ourselves so we can improve. It’s important to face down the ego and figure out why failure happens.
Mistakes are inevitable, in fact desirable. We need to work towards early discovery of failure, get better at managing risk and ensure good feedback loops. We’ll never get a perfect system but we can continue to refine the complex systems we have.