Saturday, June 16, 2007

Wisdom of Crowds

I don't remember how I came across Wisdom of Crowds by James Surowiecki but the moment I read the introduction I knew I had to have the book.

The book starts with the story of a famous scientist Francis Galton stumbling upon a weight-judging competition at a Stock Fair. An ox was put on display and passer-bys were asked to look at the ox and give their best estimates of its weight when "slaughtered and dressed". The result was astonishing - while individual estimates varied a lot, the average of the 787 estimates came to be only 1 pound off the real weight. In statistical terms that means that human ox-weight-estimation ability is unbiased - it has no systematic error.

So what is so exciting about research on human estimates and their properties? They may shed some light on some important things - most notably the institute of democracy. In principle, democracy has any chance of functioning only if people can give correct estimates in areas that they are not experts in.

So it would seem to me that the book on wisdom of crowds should investigate the question of systematic errors in human judgement to its fullest. To my utmost frustration, it did not. The term "systematic error" is not mentioned even once in the book. To be fair, the book talks about other conditions conducive to good group estimates and decisions - diversity and independence but the very basic issue of systematic errors is completely skipped over.

Galton's experiment notwithstanding, people do make systematic errors, in particular, when voting on economic issues - see new book by Bryan Caplan The Myth of the Rational Voter: Why Democracies Choose Bad Policies. Some of his papers are also available online.

Welcome!

This blog is nothing more than a hi-tech replacement of the e-mails that I usually write to my friends when something noteworthy comes into my view. As an added benefit all the links will be kept at one place.