Mitch Wheat - Treat the cause, not the symptoms!

I’ve recently been reading “The Wisdom of Crowds” by James Surowiecki. It’s a well written, easy to read, intriguing and thought provoking book. It doesn’t contain any heavy mathematics; in fact it contains no mathematics at all, so should be accessible to the widest audience. I would definitely recommended reading this book.

The ‘wisdom of crowds’ refers to collective intelligence, namely how it is possible for a crowd to pick a solution which is better than a corresponding one made by an expert: “when our imperfect judgments are aggregated in the right way, our collective intelligence is often excellent”.

The wide variety of complex problems upon which collective intelligence can be brought to bear, is broken into 3 problem groups: cognition, coordination and cooperation. Cognition problems are defined as those that have or will have a definite solution, such as “Where would be the best place to build this new road?”. Coordination problems require members of a group to determine how to coordinate their behavior with each other, such as buyers and sellers trading at a fair price. Cooperation problems involve getting self-interested, possibly distrustful people to work together, even when self-interest would seem to imply no benefit from taking part, such as paying taxes.

One of the key concepts discussed is the ‘Information Cascade’, a situation where people make a decision based more on the decisions being made around them rather on their own private information, the outcome of which is not always beneficial. I loved the story of one such example, plank-road fever (pg 51)!

[The human brain can be viewed as a crowd; perhaps that’s why when you become expert in a particular field, you can become better still by learning about other fields. Diverse perspectives are more likely to come up with something new, but I digress…]

I came across an example of an information cascade that centers on the humble screw in 1860s America, which was vaguely reminiscent of the ongoing changes we have seen in software development over the last 30 - 40 years. Back in the 1860s, when the machine-tool industry was a rough analogy to the technology industry in the 1990s, screws were individually hand-made by machinists. In the absence of national or industry standards, when a nut or bolt was damaged or lost, either a new one would have to be hand crafted or a replacement sent for from the original builder. This obviously severely limited the possibility of mass-production, and as James notes, it also enabled individual machinists to protect their way of life. As a customer, if you have a reliance on something that is custom made, then you are locked in to that supplier. Here’s the line that resonated with me in terms of the software development industry: “But if the screws became interchangeable, customers would need the craftsman less and would worry about the price more”.

William Sellers, a respected machinist of his era, believed that mass production was inevitable. Over a period of approximately 6 years he designed and promoted a standardized screw that was easier, faster and cheaper to produce than any other. Each new customer he secured meant it more likely that others would follow suite, based on the experience of the predecessors.

James sums up an information cascade as follows: “In a cascade, people’s decisions are not made independently, but are profoundly influenced – in some cases, even determined – by those around them.”