Computer-Supported Peer Review in a Law School Context

Kevin Ashley

University of Pittsburgh - School of Law

Ilya Goldin

Carnegie Mellon University - Human-Computer Interaction Institute

September 12, 2012

U. of Pittsburgh Legal Studies Research Paper No. 2012-24

Legal instructors have been urged to incorporate peer reviewing into law school courses as a way to provide students much needed feedback. Peer review can benefit legal education, but only if law school instructors adopt peer review on a large scale, and for that, computer-supported peer review systems are crucial. These web-based systems orchestrate the mechanics of students submitting written assignments on-line and distributing them to other students for anonymous review, making it considerably easier for instructors to manage.

Beyond the problem of orchestrating mechanics, however, a deeper obstacle to widespread acceptance of peer review in legal education is the question of how well student peer feedback focuses on the legal analytical aspects of the student-authored texts. Can students receive useful feedback on substantive legal issues from peers who, after all, are law students taking the same course and learning the material for the first time, too?

This article reports an experiment assessing the pedagogical utility of computer-supported peer review in a legal educational context. We compared two kinds of review rubrics, suites of instructor-provided prompts focusing student reviewers on a set of pedagogically relevant reviewing criteria: legal domain-related versus problem-specific rubrics. In particular, we compared peer ratings elicited with each of these rubrics to independently assigned instructor scores.

Our results show that peer-generated review scores using either type of rubric are correlated with an instructor’s independently assigned grades and support the belief that peer review can serve as an additional source of useful feedback, including on substantive legal issues. In addition, analysis of the data can inform instructors about how well the writing exercise met their instructional goals and can assist instructors in adopting more pedagogically effective review criteria. The article concludes by explaining how legal instructors can administer a computer-supported peer-review writing exercise in their own courses using a web-based program.

Number of Pages in PDF File: 33

Keywords: computer-supported peer review systems, peer review, peer editing, peer feedback, legal writing, legal instruction, law school, teaching, education, Web-based peer review systems, writing assessment, validity, reliability

Open PDF in Browser Download This Paper

Date posted: September 16, 2012  

Suggested Citation

Ashley, Kevin and Goldin, Ilya, Computer-Supported Peer Review in a Law School Context (September 12, 2012). U. of Pittsburgh Legal Studies Research Paper No. 2012-24. Available at SSRN: https://ssrn.com/abstract=2145570 or http://dx.doi.org/10.2139/ssrn.2145570

Contact Information

Kevin Ashley (Contact Author)
University of Pittsburgh - School of Law ( email )
3900 Forbes Ave.
Pittsburgh, PA 15260
United States

Ilya Goldin
Carnegie Mellon University - Human-Computer Interaction Institute ( email )
5000 Forbes Avenue
Pittsburgh, PA 15213
United States
Feedback to SSRN

Paper statistics
Abstract Views: 445
Downloads: 50