Ethical Implementation of an Automated Essay Scoring (AES) System: A Case Study of Student and Instructor Use, Satisfaction, and Perceptions of AES In a Business Law Course

13 Pages Posted: 1 Nov 2015

See all articles by John Lewis

John Lewis

United States Coast Guard Academy; Salve Regina University; University of Maryland University College (UMUC) - Graduate School

Date Written: 2013

Abstract

A pilot study of a vendor provided automated grading system was conducted in a Business Law class of 27 students. Students answered a business law fact pattern question which was reviewed and graded by the textbook vendor utilizing artificial intelligence software. Students were surveyed on their use, satisfaction, perceptions and technical issues utilizing the Write Experience automated essay scoring (AES) software. The instructor also chronicles the adoption, set up and use of an AES. Also detailed are the advantages and disadvantages of utilizing such software in an undergraduate course environment where some students may not be technologically adept or may lack motivation to experiment with a new testing procedure.

Automated grading of student assignments is part of the next wave of textbook enhancements that vendors will be providing to instructors in the near future. Several vendors are conducting beta testing with instructors in hope of offering automated grading as part of their textbook and course support. Of course, such services will be an additional cost to the text. Exactly what will be charged for such services remains to be seen.

The vast majority of previous research in the area of AES has been limited to its use in grading assignments in the STEM fields. Computer science instructors have been experimenting with self-created automated grading software for several decades. Recently automated grading software has been implemented by vendors to score essay questions in online tests such as the Graduate Management Admission Test (GMAT).

The experience of one business law class in using automated grading software as part of such a beta test can give valuable insight into the needs and expectations of the three stakeholders in this situation – instructor, student and vendor. Implementation of an AES raises numerous issues. Should an instructor be willing to relinquish control of the grading process to an outside entity? Are the student needs for feedback and grading fairness being met? Is the technology advanced enough to replace the human element always present in grading assignments? What will be the economic impact on students if such software is adopted?

The implications for the various stakeholders are discussed and addressed. The author comes to conclusions about the pedagogical usefulness of AES systems and offers suggestions for best practices to be employed by instructors interested in implementing such software in their courses.

Keywords: Automated Essay Scoring, AES, summative assessment, formative assessment, essay questions, essay grading, computer ethics, Write Experience

JEL Classification: K2, M1

Suggested Citation

Lewis, John, Ethical Implementation of an Automated Essay Scoring (AES) System: A Case Study of Student and Instructor Use, Satisfaction, and Perceptions of AES In a Business Law Course (2013). Available at SSRN: https://ssrn.com/abstract=2684803 or http://dx.doi.org/10.2139/ssrn.2684803

John Lewis (Contact Author)

United States Coast Guard Academy ( email )

31 Mohegan Ave Pkwy
New London, CT 06320
United States

Salve Regina University ( email )

Newport, RI 02840-4192
United States

University of Maryland University College (UMUC) - Graduate School ( email )

United States

Here is the Coronavirus
related research on SSRN

Paper statistics

Downloads
57
Abstract Views
622
rank
406,385
PlumX Metrics