The Black Swan: The Impact of the Highly Improbable

Four hundred years ago, Francis Bacon warned that our minds are wired to deceive us. "Beware the fallacies into which undisciplined thinkers most easily fall--they are the real distorting prisms of human nature." Chief among them: "Assuming more order than exists in chaotic nature." Now consider the typical stock market report: "Today investors bid shares down out of concern over Iranian oil production." Sigh. We're still doing it.

Our brains are wired for narrative, not statistical uncertainty. And so we tell ourselves simple stories to explain complex thing we don't--and, most importantly, can't--know. The truth is that we have no idea why stock markets go up or down on any given day, and whatever reason we give is sure to be grossly simplified, if not flat out wrong.
Nassim Nicholas Taleb first made this argument in Fooled by Randomness, an engaging look at the history and reasons for our predilection for self-deception when it comes to statistics. Now, in The Black Swan: the Impact of the Highly Improbable, he focuses on that most dismal of sciences, predicting the future. Forecasting is not just at the heart of Wall Street, but it’s something each of us does every time we make an insurance payment or strap on a seat belt.
The problem, Nassim explains, is that we place too much weight on the odds that past events will repeat (diligently trying to follow the path of the "millionaire next door," when unrepeatable chance is a better explanation). Instead, the really important events are rare and unpredictable. He calls them Black Swans, which is a reference to a 17th century philosophical thought experiment. In Europe all anyone had ever seen were white swans; indeed, "all swans are white" had long been used as the standard example of a scientific truth. So what was the chance of seeing a black one? Impossible to calculate, or at least they were until 1697, when explorers found Cygnus atratus in Australia.
Nassim argues that most of the really big events in our world are rare and unpredictable, and thus trying to extract generalizable stories to explain them may be emotionally satisfying, but it's practically useless. September 11th is one such example, and stock market crashes are another. Or, as he puts it, "History does not crawl, it jumps." Our assumptions grow out of the bell-curve predictability of what he calls "Mediocristan," while our world is really shaped by the wild powerlaw swings of "Extremistan."
In full disclosure, I'm a long admirer of Taleb's work and a few of my comments on drafts found their way into the book. I, too, look at the world through the powerlaw lens, and I too find that it reveals how many of our assumptions are wrong. But Taleb takes this to a new level with a delightful romp through history, economics, and the frailties of human nature. --Chris Anderson editor-in-chief of Wired magazine

Logical Fallacies and the Art of Debate

Torture Documents Released Under Freedom of Information Act

Senate Floor Statement on the Report of the Inquiry into the Treatment of Detainees in U.S. Custody

U.S. Senate Armed Services Committee Inquiry Into the Treatment Of Detainees In U.S. Custody, November 20, 2008 (Released April, 22 2009)

Other documents released under FOIA

Department of Defense Intelligence Science Board Study on Educing Information: This compilation of 10 articles on interrogation methods and their efficacy comprises the first phase of a larger project sponsored by the Intelligence Science Board, which was chartered in 2002 to advise senior intelligence officials on scientific and technical issues of importance to the Intelligence Community.

Communist Attempts to Elicit False Confessions from Air Force Prisoners of War by Albert D. Biderman. Presented at a combined meeting of the Section on Neurology and Psychiatry with the New York Neurological Society at The New York Academy of Medicine, November 13, 1956, as part of a Panel Discussion on Communist Methods of Interrogation and Indoctrination. This report is based on work done under ARDC Project No. 7733, Task 77314, in support of the research and development program of the Air Force Personnel and Training Research Center. Lackland Air Force Base, Texas.