"Scientific" "Studies"

A BBC headline blares "Beef diet 'damages sons' sperm'" and the subheadline makes the link to legislative change: "Europe was right to ban the beef industry from using growth promoters to increase yield." The "study" (at least, as presented in the story) purports to demonstrate that intake of beef by pregnant mothers leads to lower sperm counts of their male offspring. And not just any beef: beef that has been produced with "growth promoters" such as sex hormones.

Scary stuff. But did the study really show a causal relationship? In paragraph eleven, the BS-radar is first alerted:

[Lead researcher Professor Shanna Swan] admitted the research team had no specific data on which chemicals the meat contained, and conceded other possible causes, such as exposure to pesticides, or lifestyle factors could not be ruled out. 

Wow. So basically, they did not control for anything (maybe meat eaters smoke or drink more, etc, etc), and did not even know the specific content of the meat, other than that American mothers consumed it. But surely there was a control group eating non-hormone beef for comparison, just in case, say, beef in itself is bad for sperm. Actually, there wasn't:

Professor Swan said to pin down the role of growth promoters the study would have to be repeated in men born in Europe after 1988.

Wow again. Then in the very last paragraph of the article, the BS-radar is just going crazy:

One weakness of the study was that it relied on women being able accurately to recall how much beef they had eaten during pregnancy - something which may have happened many years earlier. 

Um. So this "study" consisted of asking women about beef consumption during their pregnancy (as far back as 1949!), then regressing it on sperm counts of their male children. And not controlling for anything! Allan Pacey, a senior lecturer in andrology (that's the study of male reproduction), stated in 2004 that:

The only way to answer whether semen quality of adult males may be declining is to undertake in-depth long-term prospective studies that investigate the reproductive health of men in advance of their attempts to conceive. 

One would be hard pressed to call the study the BBC reported on an "in-depth long-term prospective" study. So it is quite curious that the legislative "money quotation" of the story, "[The study] confirms that Europe was justified to ban the use of these hormones when they did," comes from the same Dr. Allan Pacey! The quotation appears three times in this twenty paragraph story: the subheadline, a block quotation right below a nice picture of beef, and in the narrative.

The trouble is, most newspaper readers will not get much past the headline or subheadline of the story (documentation, anyone?). If they do, they are probably have zero training in how scientific studies are produced, and will have no idea that the study proves little if anything. Also curious is that the lead researcher seems to be quite aware of the study's limits. But that didn't stop The Beeb from painting the story in stark terms.