« Why Has Violent Crime Dropped More in the US than Elsewhere? | Main | BOLD ALLIGATOR???? »

October 29, 2014

Confirmation Bias in Action

On the Epidemics in History post, Tom points out that what I wrote here was completely wrong:

The other interesting insight was that in a world where global population has exploded, borders are porous, and people have the ability and means to travel almost anywhere on earth in just a few hours, every successive epidemic kills fewer and fewer people.

As he points out in the comments, the dots aren't in chronological order. In fact, they're ranked by the number of deaths. I've been trying to figure out how I got the idea that the death toll from epidemics was shrinking over time. The only thing I can come up with is some combination of looking only at the part of the graph I posted (the entire thing was too long to fit) and my simply seeing what my mind preferred to see.

I tried sorting the epidemics chronologically and running several curve fits through the data points to see if there was any discernable downward trend in deaths over time. Frankly, despite having screwed things up the first time, I still expected to see what I wanted to see.

Wrong again:

curve fit1.png

The line does go down, but the first few data points are so extreme that they're driving everything else. I suppose one could say that we haven't had an epidemic/pandemic on the order of the Plague of Justinian (541) or the Black plague (1346) in modern history, so in that narrow sense the death toll is decreasing over time but even if you leave out those two data points, you don't get an unambiguous downward trend.

Here's a color coded (green = plague, red = cholera, blue = flu) and chronologically sorted list:

chrono.png

It's fascinating to me how our minds want to see clear patterns in data, sometimes when there isn't much of a pattern (or even any pattern at all). As a killer of human beings over time, plague seems to have given way to cholera and then to flu. Since 1960, about half of the epi/pandemics seem to be "one offs".

Anyway, thanks to Tom for spotting my mistake and giving me the change to take another look at the data (this time more carefully, though unfortunately I'm still a bit rushed). Please let me know if I've missed or misstated anything.

Posted by Cassandra at October 29, 2014 08:29 AM

Trackback Pings

TrackBack URL for this entry:
http://www.villainouscompany.com/mt/mt-tb.cgi/5364

Comments

100 million seems like a very high estimate for the Plague of Justinian. Wiki gives 25 million, which itself is astonishingly high for such an early date.

Posted by: Texan99 at October 29, 2014 09:47 AM

I agree - when I first looked at this, I remember thinking "How in the heck did they get those early numbers? And how reliable are they?"

Even the later numbers are probably not terribly accurate.

I deal with measurement and measurement error a lot in my work. I'm convinced that most measurement isn't nearly as accurate as people think it is, but also that we're better off measuring (even imprecisely) than not measuring because most people do a really lousy job of estimating. Even imprecise measurements provide some feedback about whether we're even in the right ballpark.

Posted by: Cass at October 29, 2014 09:55 AM

Population growth influences the data, too. When considering some pandemic, I'm more interested in the probability of a person dying than in the total number of deaths.

When I take a casual look at the "Epidemic Deaths over Time" graph, I easily concur with Cassandra's observation that the first points skew the projecting of a curve. And I totally get her face palm comment about getting bit by having wanted to see a pattern. Dry lab results don't happen merely in an education environment. (Sigh. Try discussing Anthrogenic Global Warming with somebody who not only has no clue, but does not have any experience with interpreting data to discern patterns.)

So, with those caveats, and without doing more than looking at the graph thus confessing using no statistical analysis, no curve fit tools, I'll make a SWAG. Discount the first two points. Total deaths increase from 1500 to present. But, given population growth, the probability of death declines significantly.

Parallel to the results of epidemics, ponder the question of deaths from war. I bet it would show growth over time, while simultaneously the probability of a given soldier dying decreases and also the probability of a given non-combatant.

Posted by: Roy at October 29, 2014 09:59 AM

Population growth influences the data, too. When considering some pandemic, I'm more interested in the probability of a person dying than in the total number of deaths.

Absolutely - these are unnormalized numbers.

This is a bit of tangent, but I couldn't help wondering if the spike in deaths in the late 19th and early 20th centuries isn't related to the increasing ability of people to travel (and whether WWI and WWII didn't mask a lot of disease-related deaths as well?).

If I had to guess at one possible reason for the rather precipitous drop in fatalities in the 21st century, I rather like Tex's point (on the other post) about rich societies being a very recent phenomenon. Similar to the dramatic increase in severely injured vets surviving what - just a few decades ago - would have been fatal wounds.

War didn't get less dangerous. We just got better at treating combat wounds.

Posted by: Cass at October 29, 2014 10:16 AM

A hundred million is about four times the top-level estimate for that plague that I've heard, and that one achieves 25 million by reaching all the way out to the end of the 6th century. (See here.) Not sure where they're getting that, but I'd guess it's by grabbing other Bubonic plague outbreaks in even more distant centuries.

If you drop it to 25M, your chart looks less skewed.

Posted by: Grim at October 29, 2014 11:51 AM

As far as I know, by the way, the only plague to have plausibly killed 100,000,000 people in a few decades is Communism.

Posted by: Grim at October 29, 2014 11:52 AM

It would seem a possible explanation is that the survivors of earlier plague (viruses) have imparted to their survivors some genetic trait or traits that has increased the inherent ability to survive.

There was a terrible plague in Athens during the Pelopponesian War (437-403 BC) and some now think this was ....Measles. There was just no resistance to it and it tore through the population lethally.

Today we have vaccines, but even now, it is unusual to hear of children dying from measles, in epidemic quantities.

We have really just learned about retro-viruses (HIV being the driver here), but as knowledge has increased, it seems as though retro-viruses have been with us for quite a long time. It is now believed that some cases of Type 1 Diabetes is a retrovirus that killed the cells producing insulin in the pancreas.

We all may have increase natural immunity to viruses that has been inherited, which MIGHT explain (in part) your graph.

We also don't really know how viruses come about; that is, why do they exist? There is plenty of explanations for bacteria and bacteriological diseases, but viruses are really strange bits of material (RNA) that are difficult to explain with respect to their existence.

Posted by: Don Brouhaha at October 29, 2014 12:34 PM

And one of the reasons the death toll for HIV is so large is that it's a 50 year total while many of the other epidemics are just a handful of years.

Annualized % of population is likely the best metric for analysis.

Posted by: Yu-Ain Gonnano at October 29, 2014 12:45 PM

We have really just learned about retro-viruses (HIV being the driver here), but as knowledge has increased, it seems as though retro-viruses have been with us for quite a long time. It is now believed that some cases of Type 1 Diabetes is a retrovirus that killed the cells producing insulin in the pancreas.

Don, I read a while back about a theory that it's a virus that switches leukemia on (please excuse my complete and utter scientific illiteracy - that's probably the wrong term for it). But I was fascinated b/c my nephew died from a particularly virulent type of leukemia, and the onset coincided with all three kids in the family suffering from some kind of virus.

I think it's quite possible that we're going to find that viruses cause all kinds of problems we didn't think of as communicable. And maybe they aren't, strictly speaking (the virus is, but the changes you'll get the disease too are just heightened - not "high").

Isn't schizophrenia another example of a malady that is possibly related to viruses?

YAG, there are several really looooooong epidemics in that list. That struck me as odd too.

Posted by: Cass at October 29, 2014 04:16 PM

There are lots of links between viruses and various cancers, not surprisingly considering that viruses work by commandeering your DNA. Anything that messes with DNA--whether it's a toxin, ionizing radiation, or a virus--is liable to spur cancer-like mistakes eventually.

Speaking of long epidemics, you get a funny idea of the impact of HIV if you think of it "happening" in 1961. Technically it may have appeared on the scene early, but to get to 39 million you have to imagine it between the mid-1980s or so and the present.

Posted by: Texan99 at October 29, 2014 04:27 PM

Anything "alive" is driven to reproduce itself.

Viruses are basically pieces of RNA, that insert themselves into the host DNA, and use the cell of the host to produce more viral RNA, to infect more cells, etc. The host cells usually rupture at some critical time to release more viral RNA, which will, over time, wreck your body (if your immune system cannot overcome the virus). There is a relationship between how infectious the virus is and how lethal it is. Very lethal viruses (such as HIV retrovirus) are actually not very communicable, and cannot actually survive outside the human body.
Measles, etc are very communicable, but rarely deadly to the host.
Polio was crippling and at times was an epidemic, but was usually not immediately fatal to the host. It might however shorten the life of the host.
If the virus is very deadly, it usually cannot infect many people prior to the death of the host body, and thus cannot perpetuate itself.

It may be that viruses have arrived at this point in an evolutionary sense, as a symbiote to many life forms to pass on certain bits of RNA (transcripted back to host DNA), and that the dangerous viruses are symbiotes gone wild or mutated to be terribly wrong/ unhealthy.

It may be also that people in the US that have had various viral immunizations may also have somewhat improved resistance to Ebola, for example, than the natives of Sierra Leone or Liberia, that may not have had all the general immunizations.

Every virus is different, and it take time for the human body to "learn" how to identify and resist a viral infection. It is also likely that viruses in your body are never completely irradicated, and as you age and your body immune system weakens, you are more vulnerable to the viruses that are latent in your own body, such as herpes zoster, which appears as Shingles to people that have already been infected with Chicken Pox as a child (same virus).

Posted by: Don Brouhaha at October 29, 2014 06:08 PM

Viruses are basically pieces of RNA, that insert themselves into the host DNA, and use the cell of the host to produce more viral RNA, to infect more cells, etc.

Don, this sounds suspiciously like a description of my day job :p

Seriously, thanks for the explanation. I love learning things, and this is a topic I know fairly little about. Most of what I do know is old news and probably long since disproved.

Posted by: Cass at October 29, 2014 06:41 PM

Old news? Hell, I just found out that the "London Plague" isn't a reference to Yoko Ono.

Posted by: spd rdr at October 29, 2014 08:43 PM

Wow, when you make a correction you really make a correction!

I had never heard of the Plague of Justinian. Now I'll have to go look it up.

Posted by: Tom at November 1, 2014 12:45 AM