The Improbability Principle: Why Coincidences, Miracles, and Rare Events Happen Every Day, by David J. Hand

• Category: Non-fiction
• Rating: 3 out of 5
• Tags: Probability, Statistics
• How I learned about it: Mentioned in a Long Read article in The Guardian

Probability theory is hard.  The subtleties and counter-intuitive approaches required to get the calculations right make it a tricky subject.  So a book like this for a popular audience is a welcome chance to understand some aspects of probability and statistics without having to tear out too much hair.

In some ways, the ideas presented in the book are very simple.  Even something with a very low chance of happening can and will happen if there are enough opportunities for it to occur.  Roll a fair six-sided die enough times and you will get surprising runs of the same face coming up in a row.  The chance of it happening may be incredibly tiny, but that just means you must roll the die that many more times.

Similarly, coincidences occur all the time, just because so many events happen.  Counting events as a match is prone to hindsight bias and fuzzy definitions (“The Law of Close Enough”), and since something is always happening, amazing coincidences shouldn’t be that much of a surprise.

Going deeper, depending on context and viewpoint, there are different kinds of probability.  In the frequentist version, it’s the statistical likelihood that something will happen.  Subjective probability is the confidence one has that an event will occur.  And there are more, including the classical, logical and propensity interpretations.

It’s fun to read the tales of strange coincidences, which the author also collects on his website, and helpful to know when something is so unlikely that we can say (almost!) that it won’t occur, depending on scale (one in a million on the human scale, 1 in 10⁵⁰ on a cosmic scale).  At these chances of occurrence, we can behave as if it won’t happen.  So I won’t be buying a lottery ticket any time soon, even knowing I can’t win if I don’t play.
Visit my blog at Books I’ve Read, where you can search and filter by category, rating, tag and date.

The Norm Chronicles: Stories and Numbers about Danger and Death, by Michael Blastland and David Spiegelhalter

• Category: Non-fiction
• Rating: 3 out of 5
• Tags: Danger, Risk Assessment, Statistics
• How I learned about it: Referenced in Doing Good Better by Will MacAskill

This is a fun book about coming to grips with statistical risk, mostly the risk of a sudden, violent or accidental death, but also longer-term risks to health that shorten life expectancy.  The dilemma lies in trying to assess one’s personal risk based on statistics – if there is a one in four hundred chance of dying from something, are you going to be the one or one of the other 399?

To help with this dilemma, the authors tell stories about risk involving fictional characters.  Norm, as his name implies, is exceptionally normal.  His height, his weight, everything about him, is freakishly average.  Norm does his best to make rational judgements about risk.  Prudence, on the other hand, worries constantly, taking every precaution, while the Kevlin brothers gleefully take risks and (mostly) get away with it.

Comparing the risks of different things can be challenging.  Is it riskier to go skiing or scuba diving?  To have that extra slice of bacon, or a second drink?  Fortunately, there are standard measures to calibrate these risks.  For the risk of sudden death, we have the MicroMort, defined as a one in a million chance of death.  For the average person in the US or UK, doing average, daily things, the daily risk of sudden death is around 1 MicroMort a day.  For longer-term risks, there is the MicroLife, equal to about 30 minutes, because the average person has around one million half-hours of adult life to live.  We expend our MicroLives naturally every day, but can diminish our stock more quickly through unhealthy habits.

Now we can compare risks, with some surprising results.  Scuba diving exposes you to a risk of around five MicroMorts, about the same as a general anesthetic.  Running a marathon is as risky as skydiving (seven MicroMorts).  Taking Ecstasy has about the same risk as riding horses (a potentially addictive activity called “equasy” by one writer), showing that decisions about risk-taking aren’t just about the numbers.  Is an outdoor activity that requires skill and fitness really as harmful as recreational drug use?

Comparing longer-term risks, one drink of alcohol adds a MicroLife, but two takes away that and more.  Twenty minutes of exercise adds two MicroLives, but after twenty minutes, you only add about the same amount of time as spent exercising.  “It’s as if time stops for you when you’re exercising.  And on a treadmill it can certainly feel that way.”  Two hours of TV takes away those two MicroLives you gained exercising.  People tend to discount risks that will only affect them in the future; the chance of harm seems far-off and less important.  But a MicroLife tells you how it affects you today – good habits like exercise and eating fruits and vegetables make your body age more slowly; it’s like your body is only 20 hours older each day, instead of 24.

For radiation risks, recognizing that bananas contain radioactive potassium leads to the BED, or Banana Equivalent Dose.  More fun than figuring out roentgens and Sieverts, you can see that ten minutes at Chernobyl is like eating 500 million bananas, while a transatlantic flight has 700 BED’s, shortening your life expectancy by 37 minutes.  People must be mildly radioactive, too, because sleeping with someone gives you half a BED, shortening your life-expectancy by 1 second.

 

Visit my blog at Books I’ve Read, where you can search and filter by category, rating, tag and date.

Factfulness: Ten Reasons We’re Wrong About the World – and Why Things are Better Than You Think, by Hans Rosling, with Ola Rosling and Anna Rosling Ronnlund

Category: Non-Fiction;  Rating: 4 out of 5;  Tags: History, Human Progress, Statistics, Bias

Factfulness is similar to the book Progress, by Johan Norberg, in that both books use data to show that significant improvements in human living conditions have been made in the last few decades.  Health, education, wealth and life expectancy have all improved.

Rosling starts with a little quiz to illustrate how our ideas of the world are out of date.  Most people badly fail, giving answers that show a view of the world that is far worse than reality; a view that reflects global living conditions of the 1960’s or earlier, not the present.  Global life expectancy is 70 years; 80 percent of one-year-old children have had a vaccination.  60 percent of girls in low-income countries finish primary school; by the age of 30, on average, men have attended ten years of school, and girls, nine.  Most of the world lives in middle-income countries; in the past 20 years, the proportion of the world living in extreme poverty has been cut almost in half.  But most people show a clear bias, systematically choosing the worst option on the multiple-choice quiz.

Instead of dividing the world into two classes, developed and developing, or rich and poor, Rosling suggests it is more accurate and more useful to envision four income classes.  Those in Level 1 exist on less than $2 a day; on Level 2, they earn up to $8; then up to $32 on Level 3; and more than that on Level 4.  Of the world’s seven billion peoples, there are roughly a billion on each of levels 1 and 4, with the rest in the middle.  Now Rosling introduces a helpful innovation: photographs to show what life is like on each of the four levels, in four categories: water, transport, cooking, and food.  A picture of dusty, bare feet vs a bicycle for transport nicely illustrates the differences in life on Levels 1 and 2.

In Progress, Norberg looked at ten areas in which human life has improved.  In contrast, Rosling looks at ten sources of bias preventing us from knowing the truth.  The first is the Gap Instinct.  If we divide the world into only two income classes, rich and poor, we neglect the majority of people who are in the middle, in the gap between the richest and poorest.  We also tend to imagine the worst, to extrapolate trends in straight lines, to be biased by fear, and to be swayed by large-sounding numbers.  We generalize and over-simplify.  Rosling provides tools to help overcome these and other sources of bias, insisting that our conclusions be based on data.

Factfulness is the antidote to bias and ignorance.

 

Progress: Ten Reasons to Look Forward to the Future, by Johan Norberg

According to what you hear on the news, you might think that violence, disease, famine and war are all getting worse.  In fact, you only hear about these things because they make good stories.  The truth is that all these things, and several other indicators of human well-being, are getting dramatically better.

This book could have been a tedious recounting of dry statistics; instead it provides a healthy dose of optimism to counteract the complaints and fear-mongering we hear from politicians and the media.  The media loves to report on crises and conflicts, and politicians might want to make us afraid so they can promise to keep us safe.  We hear about accidents, natural disasters, and war from around the world every day.  Some of us romanticize the better, simpler life our ancestor’s had.  But all of that creates a false impression.

Norberg takes us through it point by point.  The food supply is better, with global undernourishment down by a factor of five since the end of the Second World War.  Famine, which used to be a regular occurrence, is now rare.  Access to improved water (i.e. water protected from outside contamination) has gone from 50% of the world population to 90% since 1980.  Vaccines and antibiotics prevent diseases that used to be common.  Life expectancy at birth is up from 30 in the late 1800’s to 70 today.  Poverty and violent crime are way down.  If you’re worried about terrorism, consider the fact that you are more likely to drown in your bathtub or die falling down the stairs than you are to die from terrorism.  With greater wealth, countries can afford to focus on environmental protection.  Artificial fertilizer, while it pollutes water bodies, allows much more food to be produced per acre, saving an estimated three billion hectares from being converted to farmland.  Measures of freedom, literacy, and equality are all much better than they were one hundred to two hundred years ago.  And this happened while world population grew from 2.5 billion in 1950 to over seven billion now.

Some of these improvements illustrate the benefits of globalization.  It’s more damaging to go to war with a major trading partner (“As the Austrian economist Ludwig von Mises put it, if the tailor goes to war against the baker, he must henceforth bake his own bread”), and it’s easy to spread good ideas like better farming practices in a free and open environment.

Many things really are better than they used to be.

A Field Guide to Lies: Critical Thinking in the Information Age, by Daniel J. Levitin

Newspapers bemoan the post-truth political age, in which politicians lie freely, making up whatever claim they think will win them votes.  With easy access to vast amounts of information via the internet and social media, how can we discern the truth from the lies?  Daniel Levitin provides some tips to boost your critical thinking skills.

Start with statistics and graphs.  You need to think about how the numbers were collected. Was the sample representative?  Were the questions biased, either deliberately or inadvertently?  With graphs, you need to watch for tricks, and check the axes carefully for cheats like changes in the tick mark intervals.  Claims can be checked for plausibility by doing a few simple calculations, using reasonable estimates for the input parameters or by doing a bit of research.

Here’s one I’ve always found confusing: the reporting of changes in things expressed in percentages.  Say inflation goes up from 2% to 3%.  Is that an increase of 1% or 50%?  Levitin suggests saying “1 percentage point” for the former, and 50% for the latter.  Note that it’s not symmetrical:  if rates drop from 3% to 2%, that’s not a drop of 50%, but 33%.

Take care with terminology.  When someone says average, they usually mean the mean, but it could also mean the median (the point at which there are as many values higher as there are lower) or the mode (the value that occurs most often in the set).

Watch out for spurious correlations, as correlation does not prove causation.  I like the example correlating swimming pool drownings with the release of films in which Nicolas Cage appeared.  The correlation is pretty good, but that doesn’t prove that drownings provoke the release of new films, or that watching Nicolas Cage in a movie causes people to drown in their swimming pools.

Language is important.  Significant to a statistician does not mean noteworthy, just that a relationship is probably not random.  Definitions, categories, and sub-groups can be selected to tell a biased story, and most people won’t know about the careful selection that went on.

One of the sections I liked the best is on Bayesian conditional probability analysis.  Levitin provides a simple fourfold table containing all the possibilities, and a way to deduce the probabilities for each one given some basic information.  Then you can do a critical evaluation of things like breast cancer screening.  Say the probability of having breast cancer is 0.8%, and mammograms detect 90% of breast cancers.  But mammograms also have a false positive rate of 7%.  Now what’s the probability that you really have breast cancer given a positive mammogram result?  It’s 9.4%; still a worry if you’re the one facing that result, but a lot less than the 90% implied by the test’s accuracy rate.

David and Goliath: Underdogs, Misfits, and the Art of Battling Giants, by Malcolm Gladwell

Although I enjoyed Malcolm Gladwell’s latest book, I have to say it doesn’t seem to have the immediate impact of some of his previous ones, like “The Tipping Point”, “Blink”, or “Outliers”.  When I read the earlier books, I remember thinking right away that here was something novel, with new insights and interesting stories.  In “David and Goliath”, those reactions aren’t as strong.

Gladwell’s thesis is that greatness can arise from adversity.  He talks about the Theory of Desirable Difficulty, which proposes that a certain level of hardship stimulates in some people a response that makes them better, stronger, or more skilled in a way that contributes to later success.  This may seem a commonplace view, but Gladwell backs it up with specific stories.

There may not be a “Gee, wow” or a big “Ah-ha” moment with this book, but you will get some interesting ideas supported by data, and some good stories, well told, that you probably haven’t heard before.