Discovery and the Evidentiary Foundations of Implicit Bias
31 Pages Posted: 13 Feb 2015
Date Written: February 1, 2015
Experts who opine on implicit, or unconscious, bias as a source of discrimination pose unusual challenges for attorneys who must challenge their testimony. Implicit bias is difficult to measure using even sensitive instruments in controlled testing environments, and to date courts have not ordered employees of defendant companies to submit to testing for implicit bias. Thus, unlike other experts who commonly appear in employment cases, experts who testify regarding implicit bias base their testimony on general social science research that has no demonstrated connection to the present case but which the experts contend is helpful in understanding the claims or defenses at issue in the case. For example, Dr. William Bielby’s report in Dukes v. Wal-Mart Stores, Inc. contains over 100 citations to the social-psychological literature, but fails to analyze any decisions by Wal-Mart that are alleged to be discriminatory. In contrast, the report in that same case by the plaintiffs’ statistician, Dr. Richard Drogin, contains no citations to academic research but describes his analysis of Wal-Mart’s data on pay and promotions. The attack on Dr. Drogin’s testimony focused on his own statistical analysis, which was rebutted with the statistical studies of Wal-Mart’s own expert.
On the other hand, the attack on Dr. Bielby focused primarily on the attenuated connection between the research he cited, which was not specific to Wal-Mart, and the employment decisions at issue in the case; the numerous studies on which he based his testimony went largely unexamined.
Dr. Drogin’s testimony was based on Wal-Mart’s own data. During the course of discovery, Wal-Mart was able to replicate his statistical work and expose what it alleged to be its weaknesses and shortcomings. As a result, not only was Dr. Drogin’s credibility as a witness in play, but so were the many judgments he made as a statistician. His decision to include or exclude various segments of Wal-Mart’s workforce in his data, which promotions he deemed worthy of study, how he defined a promotion, any distinctions he drew between full and part time employment, the estimation techniques he used to obtain his statistical results, and his interpretation of those results all were potential areas of inquiry. Obviously, the same opportunity was afforded the plaintiffs, who took aim at Wal-Mart’s statistical expert. In contrast, because Dr. Bielby conducted no empirical study of Wal- Mart, but based his opinion largely on the research of social scientists who studied other businesses or laboratory subjects, the data under- lying this substantial body of research was beyond the scope of the litigation.
This article advocates discovery of data underlying academic research relied on by testifying experts. We refer to such data as “secondary data.” Absent discovery, the data underlying the testifying expert’s opinion will be unexamined. Although a party’s own expert may critique the research literature cited by an opposing expert, the evidentiary foundation of studies crucial to expert opinions will evade scrutiny when the secondary data is not available for review. The research studies may have been peer reviewed, but reviewers typically recommend studies for publication without reviewing the underlying data. Confronting the research relied on by an expert solely by considering published details and competing published results is unsatisfactory because key details will remain unknown and flaws in collecting or analyzing data will not be exposed.
We argue that parties should request secondary data, that courts should not only permit but encourage discovery of secondary data to ensure the reliability of expert opinions, and that courts should exclude testimony premised on data that remain secret. First, we document the extent to which expert opinions regarding implicit bias rely on research that evades careful scrutiny by either the academic journals or the courts that admit the expert’s testimony. Next, we discuss the arguments that shield the data underlying research from discovery. Finally, we argue for discovery of secondary data notwithstanding the arguments against disclosure, and argue for excluding expert testimony that relies on data beyond the reach of the opposing party.
Suggested Citation: Suggested Citation