Is there Wisdom in Crowds?

by Dave Chase, Altus Alliance

“The Wisdom of Crowds” by James Surowiecki is reviewed by Altus Alliance’s Dave Chase who shares what marketers and technology companies might learn from Surowiecki’s insights.

Surowiecki aims to be the next New Yorker contributor to have a mass appeal book, ala Malcolm Gladwell. He clearly wants to position this book as the next “The Tipping Point” — combining cognitive science and other disciplines into a book addressing business, politics, society and economies. The book’s relevance to technology companies may not be as obvious as The Tipping Point, although there are examples from our industry. As Surowiecki states, “Google is built on the wisdom of crowds. The core of the system is the PageRank algorithm — a calculating method — that attempts to let all the Web pages on the Internet decide which pages are most relevant to a particular search.” He goes on to say “With most things, the average is mediocrity. With decision making, it’s often excellence. The idea of the wisdom of crowds isn’t that a group will always give you the right answer but that on average it will consistently come up with a better answer than any individual could provide.”

While the stakes in technology companies may not be as high as space shuttle missions or stock markets (two of Surowiecki’s other examples), there are lessons in this book that apply in a variety of ways. Whether you’re making business strategy decisions or developing a product feature decision, many of us have seen the ill effects of “groupthink” versus bringing together diverse groups within an effective framework. Fortunately, the Internet can enable the elements of a “wise crowd.”

This book made me think about how collective wisdom could affect decisions such as product development decisions, ad campaign development (creative decisions, media buying, etc.), industry standards (surely there’s a way to move things forward more rapidly) and many other decisions made within your company or across teams. From my own experience working with technology companies, I can see many ways to apply principles in this book to product development, sales and marketing.

Corporations and industry bodies have generally been unwilling to improve their decision making by tapping the collective wisdom of their employees and members. Those who can harness the potential for wisdom that exists within crowds of people will have the world as their oyster.

Perhaps the most significant point for Seattle24x7 readers isn’t the perspective Surowiecki provides to the inward-facing, organizational structure/behavior perspective, but rather the outward-facing: how technology companies can better understand their customers — how they think, why they think that way, and how their ability to communicate with each other (rather than just with customer service and technical support) raises their collective IQ.

It could be said that the wisest crowd out there is the billions-big horde of Internet users, who consistently use the Web in smart ways that neither technologists nor marketers would have dreamt up. The companies and marketers who grasp the implications of this will in turn develop products and services more in tune with their customers’ needs, as well as have the accompanying Web sites and marketing campaigns to harness these insights.

While our society often trusts experts and distrusts the wisdom of the masses, Surowiecki argues that “under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them.” He uses a variety of examples ranging from simple challenges such as a crowd guessing the weight of an ox to incredibly complex: another crowd located a lost submarine where the best approximation was 20 miles wide and thousands of feet deep. It was eventually found 200 yards from where the group estimated it would be. This despite the fact that no one knew why the sub sank, no one had any idea how fast it was traveling or how steeply it fell to the ocean floor. Other compelling examples include how SARS was solved, and a method for predicting election outcomes with great accuracy.

The author outlines four elements required to have a wise crowd:

1. Diversity of opinion: Each person should have private information even if it’s just an eccentric interpretation of the known facts.
2. Independence: People’s opinions aren’t determined by the opinions of those around them.
3. Decentralization: People are able to specialize and draw on local knowledge.
4. Aggregation: Some mechanism exists for turning private judgments into a collective decision.

He also cites examples of groups where these elements are missing with sometimes disastrous consequences. Small groups can make very bad decisions because influence is more direct and immediate and small-group judgments tend to be more volatile and extreme. Large groups missing the four elements can also have disastrous results.

One significant example that Surowiecki describes concerns the Space Shuttle Columbia’s Mission Management Team (MMT). The team violated nearly every rule of good group decision making. As Surowiecki outlined, “the team’s discussions were simultaneously too structured and not structured enough. They were too structured because most of the discussions — not just about the debris strike, but about everything — consisted of the MMT leader asking a question and someone else answering it. They were not structured enough because no effort was made to ask other team members to comment on particular questions. This is almost always a mistake, because it means that decisions are made based on a very limited supply of analysis and information.”

One of the consistent findings from decades of small group research is that group deliberations are more successful when they have a clear agenda and when leaders take an active role in making sure everyone gets a chance to speak. In small groups, diversity of opinion is the single best guarantee that the group will reap benefits from face-to-face discussion.

Conversely, in a stock market bubble all the conditions that make groups intelligent — independence, diversity, private judgment — disappear. Whether it was the dot-com bust or the run-up of bowling stocks 40 years ago, stock markets have the potential to lose key elements that make them generally effective. As we are on the 75th anniversary of the 1929 stock crash, we need to remain conscious of the limits of the risks when all the elements of a wise crowd are absent whether it’s in politics, business or society.

The following are additional quotes and ideas from this useful book that are salient to the four elements of “wise crowds” — Diversity, Independence, Decentralization and Aggregation:

Diversity of opinion and background

o When there is a lot of uncertainty, such as in the early days of an industry where the winners and losers haven’t been sorted out, it’s key to have a system that encourages, and funds, speculative ideas, even though they may have only slim possibilities of success. Even more important is diversity — not in the sociological sense, but rather in a conceptual and cognitive sense. What makes a system successful is its ability to generate lots of losers, recognize them as such and then kill them off. Sometimes the messiest approach is the wisest.

o Diversity helps because it adds perspectives that would otherwise be absent and because it takes away, or at least weakens, some of the destructive characteristics of group decision making.

o We know that the crowds that make the best collective judgments are crowds where there’s a wide range of opinions and diverse sources of information, where people’s biases can cancel themselves out, rather than reinforcing each other. Individual irrationality can add up to collective rationality.

o Decision markets are well suited to companies because they circumvent the problems that obstruct the flow of information at too many firms: political infighting, sycophancy, and a confusion of status with knowledge. The anonymity of the markets and the fact that they yield a relatively clear solution, while giving individuals an unmistakable incentive to uncover and act on good information, means their potential value is genuinely hard to overestimate.

o Studies have found that groups of smart and not-so-smart people almost always do better in decision making than a group just of smart people. The development of knowledge may depend on maintaining an influx of the naïve and ignorant, because competitive victory does not reliably go to the properly educated. My take-away: Teams I’ve worked on always benefit from the fresh perspective of a newcomer.

o Homogenous groups are great at doing what they do well, but they become progressively less able to investigate alternatives. It also fosters the palpable pressures toward conformity that groups often bring to bear on their members.

o Diversity contributes not just by adding different perspectives to the group but also by making it easier for individuals to say what they really think.
Independence

o Paradoxically, the best way for a group to be smart is for each person in it to think and act as independently as possible.

o Independence doesn’t mean isolation but it does mean relative freedom from the influence of others.

o Independence is critical for two reasons 1) it keeps mistakes that people make from becoming correlated; 2) independent individuals are more likely to have new information rather than the same old data everyone is familiar with.

o If you want to improve an organization’s or economy’s decision making, one of the best things you can do is make sure, as much as possible, that decisions are made simultaneously rather than one after another.

Decentralization

o What do we mean by “decentralization?” Power does not reside in one central location, and many of the important decisions are made by individuals based on their own local and specific knowledge, rather than by an omniscient or farseeing planner.

o Decentralization’s greatest strength is that it encourages independence and specialization on the one hand, while still allowing people to coordinate their activities and solve difficult problems on the other.

o A decentralized system can only produce genuinely intelligent results if there’s a means of aggregating the information of everyone in the system.

Aggregation

o Groups generally need rules to maintain order and coherence, and when those elements are missing or malfunctioning the result is trouble. Groups benefit from talking to and listening to each other, but, paradoxically, too much communication can make the group as a whole less intelligent. [24×7]

“The Wisdom of Crowds: Why the Many Are Smarter than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations,” by James Surowiecki

Dave Chase is a partner with Altus Alliance, which specializes in driving revenue traction for emerging businesses. Before joining Altus Alliance, Chase spent nearly 20 years in the industry with the last twelve years at Microsoft in various senior marketing and general management roles, including his role as MSN’s managing director for industry marketing and relations. In that capacity, he was responsible for MSN taking a leadership role within the Interactive Marketing industry to grow Online’s share of the overall ad market in concert with AOL, CNET, Yahoo!, Google and other market leaders.

o Expect more from your email marketing
o There’s Gold in Them Thar CRM Hills
o Is Gmail good or bad for marketers?
o Online Advertising Year in Review 2003
o Why Online? Because Interactivity Provides Powerful Brand Boost