Audience Evaluations of News Videos Made with Various Levels of Automation: A Population-Based Survey Experiment

Abstract

The use of automation in news content creation is expanding from the written to the audio-visual medium with news organizations including Reuters turning to video automation services provided by companies such as Wibbitz. Although researchers have explored audience perceptions of text-based news automation, to date no published study has examined how news consumers perceive automated news videos. We conducted a between-subjects online survey experiment to compare how a socio-demographically representative sample (n=4,200) of online news consumers in the UK perceived human-made, partly automated, and highly automated short-form online news videos (n=42) on 14 different story topics. Our findings show that human-made videos received on average more favourable responses on some evaluation variables, although the differences were not large. We also found that there can be significant differences in the relative evaluation of automated and human-made news videos across different individual stories. For practitioners our results suggest partially automated news videos with post-automation human editing can be well received. For researchers our results show the need to use reasonably large sets of experimental stimuli, and suggest that maintaining socio-demographic variation within samples of respondents is worthwhile.

Keywords: AI, audience, automated journalism, news videos, perception, reception study, survey experiment, video automation