When Old Habits Die Hard: A Comment on Sander and Steinbuch's "Mismatch and Bar Passage"

Abstract

In the companion cases, Students for Fair Admissions Inc. v. President and Fellows of Harvard College and Students for Fair Admissions Inc. v. University of North Carolina et al. (SFFA), the U.S. Supreme Court held that race-conscious affirmative action violated the Equal Protection Clause of the Fourteenth Amendment. In their respective concurring and dissenting opinions, Justices Clarence Thomas and Sonia Sotomayor debated the empirical validity of “mismatch theory,” which posits that race-conscious admissions policies harm racial minorities by admitting them into challenging schools where they cannot succeed. Both Justices referenced social scientific evidence submitted to the Court through amicus briefs to support their positions. The Court’s rulings in SFFA did not make the debate over mismatch obsolete. In fact, understanding student performance across various academic settings, and how that performance relates to post-graduation outcomes, remains highly important as universities are reevaluating their admissions systems. Standardized test scores present a sizable obstacle to underrepresented minority students—especially Black and Latinx students—obtaining admission to the country’s elite colleges and universities. The continued reliance on these tests requires a strong empirical justification. Law schools have been the site for a disproportionate amount of the debate over mismatch theory, in no small part because of the availability of nationally representative datasets on law school performance, bar exam performance, and post-law school employment. Richard Sander and Richard Steinbuch’s (S&S) article, published in the Journal of Legal Education, claims that mismatch accounts for racial differences in bar exam performance, and an earlier draft was cited favorably by Justice Thomas in SFFA. The present Article, which was included in the amicus brief cited by Justice Sotomayor in SFFA, is the first comprehensive critique of S&S’s study. Numerous conceptual and methodological mistakes in S&S’s study are identified, and these errors undermine S&S’s conclusion that “the mismatch effects in their models can therefore account for the large disparities in bar passage across racial lines.” Empirical research examining the possible impact of mismatch across a range of outcomes must adhere to well-established rules of statistical inference in order to meaningfully inform academic institutions on how to best conduct their admissions practices. The recurring and unacknowledged flaws identified in studies claiming support for mismatch underscore that careful vetting of empirical research on mismatch is essential if science claims to be self-correcting. This assessment of S&S’s study lays out both the challenges to mismatch theory and, potentially, fruitful avenues of future work.

Keywords: Affirmative Action, Bar Exam, Legal Education, Mismatch, Race

JEL Classification: K00, I23, J15