Effects of Automated and Participative Decision Support in Computer-Aided Credibility Assessment

Abstract

Historically, inaccurate credibility assessments have resulted in tremendous costs to businesses and to society. Recent research offers unobtrusive credibility assessment aids as a solution; however, the accuracy of these decision aids is inadequate, and users often resist accepting the aids’ recommendations. We follow the principles of signal detection theory to improve the accuracy of recommendations in computer-aided credibility assessment by combining automated and participatory decision support.

We also leverage participation in decision-making theory to explain and predict an increased acceptance of assessment aid recommendations when perceptual cues are elicited from users. Based on these two theories, we design and test a hybrid decision aid to perform automated linguistic analysis and to elicit and analyze perceptual cues from an observer. Results from a laboratory experiment indicate that decision aids that use linguistic and perceptual cues offer more accurate recommendations than aids that use nly one type of cue. Automatic analysis of linguistic cues improved both the decision aid’s recommendations and the users’ credibility assessment accuracy.

Challenging the generalizability of past findings, the elicitation of perceptual cues did not improve the decision aid’s recommendations or the users’ assessment accuracy. Elicitation of perceptual cues, however, did improve user acceptance of the decision aid’s recommendations. These findings provide guidance for future development of credibility assessment decision aids.

Keywords: credibility assessment, decision support systems, indirect cues elicitation, linguistic analysis, signal detection theory