Leveraging Consensus Effect to Optimize Ranking in Online Discussion Boards

Abstract

Online discussion platforms (often referred to as discussion boards) are designed for facilitating remote discussions between users. To stimulate engagement (e.g., participation in the discussion), these platforms offer arriving users a ranked list of existing discussion comments. In this paper, we formalize the level of consensus in the discussion and study its impact on engagement, and how it could be leveraged by ranking algorithms to increase engagement along the discussion path. We collaborate with a leading online discussion board for education settings. Analyzing data from online discussions, we identify the level of consensus in the discussion as a new engagement driver. The presence of the consensus effect suggests that ranking algorithms should consider not only comments that would induce engagement in the present period, but also ones that would maximize future engagement by managing the desired level of consensus. Based on this insight, we propose a new dynamic model for ranking optimization, and a class of intuitive algorithms that, among other factors, account for the level of consensus when prescribing rankings that maximize engagement using a limited lookahead. In a randomized experiment consisting of 100 discussions held in an education setting, our proposed algorithm outperformed the approach used in current practice (that does not actively manage the level of consensus). Our study proposes consensus as an essential factor in user engagement and in the design of user interface in online platforms, and demonstrates the performance improvement that is achievable by leveraging it in the design of ranking algorithms in discussion boards. In doing so, our study suggests that online platforms may often benefit from rankings that build debate rather than an “echo chamber” of consensus.

Keywords: Ranking, Online Platforms, Education Platforms, Discussion Boards, Dynamic Optimization, Data-Driven Optimization