The Effects of Verbal and Visual Marketing Content in Social Media Settings: A Deep Learning Approach

Abstract

Due to the relentless development of social media marketing, firms increasingly rely on a combination of verbal and visual elements to communicate with consumers and attract their attention. The present research investigates how the semantic relationship between text and image information affects consumer engagement (forwards and comments). Leveraging a large-scale dataset of firm-generated messages, we develop a novel end-to-end scalable deep learning model to quantify each text-image message with a well-established, two-dimensional text-image incongruency (relevancy and expectancy). We find that the interaction of relevancy and expectancy, two distinct dimensions of text-image incongruency at the cognitive level, plays a predominant role in affecting consumer engagement on social media. High-relevancy-high-expectancy (HRHE) content and low-relevancy-low-expectancy (LRLE) content are the most effective strategies, whereas high-relevancy-low-expectancy (HRLE) and low-relevancy-high-expectancy (LRHE) contents do not work so well. Furthermore, this paper also shows different antecedents of different types of consumer engagement in social media contexts, including forwards and comments. In particular, HRHE offers an exclusive benefit of boosting forwards while the two strategies are equally effective in eliciting comments. This research contributes to the literature on consumer engagement and social media marketing by addressing the importance of multi-dimensional text-image incongruency and generates important managerial implications.

Keywords: consumer engagement, social media, text-image incongruency, deep learning, topic models

JEL Classification: M31