Skip to main content

Evaluating information & data: Lies and statistics

Lies and statistics

Many students love using statistics in their papers. Statistics can lend your story more weight and give the reader a feeling of having insight into or a clear view of the material, which would not be have been possible without the figures.

However, this feeling is also very well-known by organizations that present statistics. Statistics can be misused to give a feeling of insight and weightiness. It is therefore important to have insight into the techniques that are used to manipulate figures.

Watch the video Lies, damned lies and statistics (about TEDTalks), in which Sebastian Wernicke applies the instruments of statistical analysis to TED talks. There's also a version with Dutch subtitles.

Tips:

  • Do your own thinking. Ignore the statistical information before you for a moment. Then think of what you yourself would consider a representative statistic for the research that you are doing. What elements would you like to see in the statistics? Which research group would you yourself consider representative?
  • Pay attention to the intention or aim! Look carefully into the interest of the organization presenting the figures. Is there a commercial or political bias? The chance that the figures have been manipulated is greater in such cases than when your figures are supplied by a recognized agency, such as the CBS (Dutch National Statistics Bureau).
  • Search for an unbiased source. If you're looking for statistics, it's better to first find a reliable and authoritative research agency, and then search within its website for the figures that you need. If you search for these figures using only a general search engine, you will often end up in commercial websites where you will have to pay for statistical information or on sites that present unreliable statistics.

Examples

Some examples of why you should be careful with numbers and figures you find on the Internet.

  • Made-up statistics

81 percent of all published statistics are made up on the spot, the remaining 19 percent are heavily manipulated’. Sound credible? Guess again, these numbers were made up on the spot to make the point.

  • Non-representative target group

Since statistics rarely come from research of the whole population, you just have to hope that the research group is representative of the group that it is supposed to represent.

  • Manipulative survey questions

Sometimes survey questions are worded in such a way that they lead the respondent in a certain direction. ‘Do you think it just that thousands of animals suffer their entire lives so that we can eat them?’ sounds a lot different than ‘Are you for or against the bio-industry?’

  • Misleading use of figures

Even figures that are factual can be manipulated in the presentation or conclusion of a report, by, for example, fiddling with the chosen starting and ending points, times or seasons.

  • Misleading use of categories

‘Brand X shampoo was chosen by 80 percent of the target group as the best shampoo in its category.’ What is the target group? What is the category?

  • Use of conditions/qualifications

’65 % of all carnivorous mammals are threatened with extinction.’ So the conditions are that the animal is both carnivorous and a mammal. The percentage of animals in general that are being threatened with extinction may be far lower.

  • Absolute figures versus percentages

One can manipulate a result by choosing absolute figures or even relative figures. ’20 percent of the employees of company A were fired’ sounds more impressive than ‘one out of every five employees of company A was fired’.

*Based on: Korn, Bob. (rkorn@nc.rr.com) ‘Responsible thinking’, http://www.truthpizza.org/ last updated on 04/2015

Related