On Scientific Bias: When the Reported Truth is Not the Whole Truth
Posted: 7 May 2008
Date Written: May 6, 2008
In the current issue of Newsweek, Sharon Begley discusses a new book on Toxic Chemicals, one based on the premise that industry-sponsored studies often obscure positive results due to ease of manipulation of data along with financial incentive.
Ms Begley is indeed correct in reporting that It's quite easy to take a positive result [showing positive effects], and turn it falsely negative. The problem with the ensuing conclusion: that industry studies are suspect, is that the converse of the principle is also true: It's quite easy to take a negative result [showing no effect or a helpful one] and turn it falsely positive. The tricks are basically the same, along with added ones such as choosing flawed controls groups. And this feature is often used by academics in creating articles of interest to publishers.
Since most journals are generally uninterested in reporting negative results, academic researcher's whose career and tenure status is dependent on publishing are susceptible to the temptation to skew results toward the positive. Remember the coffee causes pancreatic cancer study, anyone? The study, performed by a leading epidemiologist at a premier research center was later retracted when it could not be replicated. Nevertheless, the study - flawed on its face - initially found it's way into the New England Journal of Medicine - satisfying peer reviewers and editors alike of its scientific integrity. And while industry might have a financial motive to distort study results, so do academics whose funding revolves around keeping the funding agency in business, by verifying a problem exists.
The regular peer reviewed press is governed by established researchers - i.e. those with reputations, grants and egos to protect. Research at odds with established principles of science of the day (discovered, administered or supported by these reviewers, their students, or colleagues), usually find publication doors slammed in their faces. This phenomena, as noted by Thomas Kuhn, occurs when research produces paradigm shifts that upset the status quo, thereby also impeding the advancement of science.
In sum, all peer-reviewed research is suspect, a fact recognized in 1993 by the Supreme Court in Daubert v. Merrell Dow, reversing a 70 year history of making publication by peer review the sine qua non of admissible science. The fact that a study may be sponsored by a federal grant is no more assurance of its validity than one financed by industry. Author bias is an ancillary fact to be considered. However, it is not, per se, reflective of the caliber of the researcher or the research. A similar analogy might be used to describe the author of the book under review. While the medical community has taken upon itself the obligation of disclosing associations and financial ties of authors, this ethical advance has not yet found it's way to the legal community or the lay press. And thus, while he is a respected researcher at a first class institution, it still would have been useful for your readers to know of the relationship of Dr. Michaels, the book's author, with the plaintiff's bar.
Keywords: data manipulation, integrity in research, scientific evidence, researcher bias, industry manipulation
JEL Classification: C80, C99, C42, C49, K41, K32, D83, D84
Suggested Citation: Suggested Citation