Selective reporting
Selective reporting refers to rare events which are widely reported, thereby altering the perception of how common they actually are. This is common amongst practitioners of bullshit, where they — and their customers — selectively report their successes but make no mention of their numerous failures. It contrasts to cherry picking in that selective reporting is often unintentional, and concentrates on the reporting and memory of events, while cherry picking is more specific to selecting evidence and actively ignoring evidence that isn't favourable. Both are mechanisms of confirmation bias and are reasons why anecdotal evidence, no matter how "convincing", is not accepted as evidence in science and rationalism.
| You gotta spin it to win it Media |
 |
| Stop the presses! |
| We want pictures of Spider-Man! |
|
| Extra! Extra! |
v - t - e |
“”‘Aye,’ asked he again, ‘but where are they painted that were drowned after their vows?’ And such is the way of all superstition, whether in astrology, dreams, omens, divine judgments, or the like; wherein men, having a delight in such vanities, mark the events where they are fulfilled, but where they fail, though this happens much oftener, neglect and pass them by. |
| —Francis Bacon |
Examples
The lottery
Selective reporting is best shown with examples. The easiest example is with a lottery, such as the The National Lottery in the UK, its even grander and larger Europe-wide counter-part, EuroMillions, or the numerous state lotteries in the United States. It's clear to anyone with a grasp of maths that it's rare to win the lottery; indeed if you work out the odds of dying from a heart attack and divide this through until you get the odds of having one in the next hour, you have a higher chance of having a heart attack in the hour leading up to the Lottery than you do of actually winning it. However, the millions of people who wasted their time, energy and money get zero coverage, but the winners often get paraded around; "look, you really can win the lottery, it happened to me!". This alters people's perceptions to think that the lottery may well be worth playing, when frankly, the money you have to spend just doesn't add up in a rational way.
The rational analysis of gambling doesn't alter the thrill of gambling, and the thought that you might just win. This feeling is actively supported by the selective reporting of winners, reinforcing the "might just" win idea. As such, playing the lottery may not be a wise investment, but it can be seen as good (if expensive) entertainment.
School shootings
The Columbine massacre terrified people all over the US, leading to ridiculous over-compensation such as metal detectors at school entrances to protect our babies. Statistically, the shooting was irrelevant — despite widely-reported shootings in schools and colleges and their presence in modern culture, they are in fact very rare events. Dying in such an event is even rarer. Even though any death in such a circumstance is a tragedy, the number of people killed in school shootings only infrequently go into double figures and they are almost invariably isolated and completely unpredictable incidents.[1] At least one RW editor wrote a lame song about this phenomenon.
Psychics
Because people have difficulty in processing genuine probabilities, they can often selectively report events to themselves. For example, some people claim "psychic connections" with friends and can tell when they are about to call. This idea is reinforced every time they think of a friend and they do call, however much time later, but it is not refuted every time they think of a friend and they don't call. Similarly, cold-reading mediums will happily cite the time that they were correct, that the universe conspired to beat the odds that one time. Indeed, if they're having no luck with someone, they'll stop, switch over to another topic that might work (or even more deceitfully, to a planted member of the audience) and repeatedly go on about this thing that worked, rather than even acknowledge the failure.
Ghosts and hauntings have a similar effect. No one cares about the hundreds of people who visited the "most haunted pub in England" and saw nothing, but the local press and storytellers will happily harp on, at great length, about the two or three who did.
Nuclear incidents
Among industrial accidents which do not directly affect the public, only nuclear incidents, especially at nuclear power plants, are considered general interest. A nuclear incident that causes the death of one worker will be reported worldwide, but similar events at other industrial facilities — such as coal power plants, oil refineries, chemical processing plants, mines and shipyards — don't tend to get beyond the local media. The fact that nuclear incidents have far stricter reporting requirements than any other kind of workplace accident may contribute to this, thus making a failure of such stringent protocols more worthy of media attention.
Note that "nuclear incident" is generally a low impact event, and a different classification than "nuclear accident", which involves at least some impact to the public. Nuclear accidents aren't supposed to happen.
In hypothesis testing
Selective reporting is a strong bias that prevents correct conclusions arising from testing hypotheses — it's a specific form of selection bias whereby only interesting or relevant examples are cited. The Francis Bacon quote heading this article is in reference to glorifying those who survived a shipwreck by praying first. By citing those individuals, the power of prayer is almost self-evident. However, how effective would this be for testing the hypothesis "saying your vows will save you from a shipwreck"? Even without Bacon's prompting it should be immediately obvious to ask what about the ones who prayed and didn't survive? Priests wishing to espouse how important prayer is have a vested interest in only talking about those who survive and said their prayers and not any of the other three possible groups; those who survived and didn't pray, and those who died despite doing either.
Specifically, the act of talking only about survivors is an observation selection effect, that is; the ones who didn't survive aren't around to be observed! We have, at least for the purposes of the illustration, no way of knowing the answer to the question of those who don't survive and what they did before. So we can only report the survivors, and even then only the ones who claimed to pray will be of interest. Because the examples are selectively reported only to confirm the hypothesis, the ability for this evidence to assess the hypothesis is, in fact, severely limited.
In medicine
This is unfortunately a very serious application of the use of selective reporting, particularly during health scares.
Vaccine complications
Vaccines and their side-effects are commonly selectively reported, and facts and citations are often cherry picked too. Side-effects are inevitable with all medical interventions, but they're usually very rare, reasonably mild and much less risky than leaving an illness well alone. However, in the hands of the media, and the right publicity people pushing for it, the minor incidents of side effects can be happily blown out of all proportion and appear as real, commonplace risks. For example, when one girl died after receiving the HPV vaccine in September 2009, it was widely reported in the national media.[2] Almost no coverage was given to the 1.4 million girls who received the vaccine, had no reaction to it and will now go through their lives with a much reduced risk of cervical cancer — "Girl Gets Vaccine; Doesn't Fall Ill" doesn't make for an impressive headline. Selective reporting in these cases serves nothing but the self-interests of newspapers and magazines that want to sell more copies by dramatising a story. Reports like this take exceptionally rare events and thrust it into the public eye so hard that people have almost no choice but to think "but it could happen to me" and panic.
Recreational drug deaths
Selective reporting of drug deaths is very common. Deaths from improper use of legal over-the-counter drugs are often ignored, while deaths from illegal recreational drugs receive a lot of publicity. The resulting perception is that recreational drugs are very dangerous. A review of 10 years of media reports about drug deaths in Scotland showed that 1 in every 250 deaths from paracetamol was reported, for benzodiazepines — 1 in 50, for amphetamines — 1 in 3, and for ecstasy — every single death was reported.[3]
See also
References
- See the Wikipedia article on List of school-related attacks. (Also note that this is not a new phenomenon.)
- For instance, here's a Guardian article, although it does at least mention the girls who got it safely.
- Forsyth, AJM (2001) Distorted? A quantitative exploration of drug fatality reports in the popular press. Int. J. Drug Policy 12: 435–453.