Human Detection of Deepfakes: An Investigation of Confidence and Accuracy

24 Pages Posted: 11 Apr 2023

See all articles by Klaire Somoray

Klaire Somoray

James Cook University - Department of Psychology

Daniel Miller

James Cook University

Abstract

Deepfake videos are becoming more pervasive. In this pre-registered online experiment, participants (N = 454, Mage = 37.19, SDage = 13.25, males = 57.5%) categorize a series of 20 videos as either real or deepfake. All participants saw 10 real and 10 deepfake videos. Participants were randomly assigned to receive a list of strategies for detecting deepfakes based on visual cues (e.g., looking for common artifacts such as skin smoothness) or to act as a control group. Participants were also asked how confident they were that they categorized each video correctly (per video confidence) and to estimate how many videos they correctly categorized out of 20 (overall confidence). The sample performed above chance on the detection activity, correctly categorizing 60.7% of videos on average (SD = 13). The detection strategies intervention did not impact detection accuracy or detection confidence, with the intervention and control groups performing similarly on the detection activity and showing similar levels of confidence. Inconsistent with previous research, the study did not find that participants had a bias toward categorizing videos as real. Participants overestimated their ability to detect deepfakes at the individual video level. However, they tended to underestimate their abilities on the overall confidence question.

Keywords: deepfake, human detection, deepfake detection strategies, accuracy, confidence

Suggested Citation

Somoray, Klaire and Miller, Daniel, Human Detection of Deepfakes: An Investigation of Confidence and Accuracy. Available at SSRN: https://ssrn.com/abstract=4412638 or http://dx.doi.org/10.2139/ssrn.4412638

Klaire Somoray (Contact Author)

James Cook University - Department of Psychology ( email )

Daniel Miller

James Cook University ( email )

Cairns, 4878
Australia

Do you have negative results from your research you’d like to share?

Paper statistics

Downloads
62
Abstract Views
297
Rank
636,652
PlumX Metrics