Longitudinal Impact of Preference Biases on Recommender Systems' Performance

67 Pages Posted: 9 Mar 2021 Last revised: 1 Jun 2022

See all articles by Meizi Zhou

Meizi Zhou

University of Minnesota - Twin Cities - Carlson School of Management

Jingjng Zhang

Indiana University - Kelley School of Business - Department of Operation & Decision Technologies

Gediminas Adomavicius

University of Minnesota - Twin Cities - Carlson School of Management

Date Written: March 7, 2021

Abstract

Research studies have shown that recommender systems' predictions that are observed by users can cause biases in users' post-consumption preference ratings. Because users' preference ratings are typically fed back to the system as training data for future predictions, this process is likely to influence the performance of the system in the long run. We use a simulation approach to study the longitudinal impact of preference biases (and their magnitude) on the dynamics of recommender systems' performance. We look at the influence of preference biases in two conditions: (i) during the normal system use, where biases are typically caused by the system's inherent prediction errors, and (ii) in the presence of external (deliberate) recommendation perturbations. Our simulation results show that preference biases significantly impair the system's prediction performance (i.e., prediction accuracy) as well as users' consumption outcomes (i.e., consumption relevance and diversity) over time. The impact is non-linear to the size of the bias, i.e., large bias causes disproportionately large negative effects. Also, items that are less popular and less distinctive (in terms of their content) are affected more by preference biases. Additionally, intentional recommendation perturbations, even on a small number of items for a short time, substantially amplify the negative impact of preference bias on a system's longitudinal dynamics and causes long-lasting effects on users' consumption. Our findings provide important implications for the design of recommender systems.

Keywords: Recommender systems, longitudinal dynamics, preference biases, system performance, agent-based simulation.

Suggested Citation

Zhou, Meizi and Zhang, Jingjng and Adomavicius, Gediminas, Longitudinal Impact of Preference Biases on Recommender Systems' Performance (March 7, 2021). Kelley School of Business Research Paper No. 2021-10, Available at SSRN: https://ssrn.com/abstract=3799525 or http://dx.doi.org/10.2139/ssrn.3799525

Meizi Zhou (Contact Author)

University of Minnesota - Twin Cities - Carlson School of Management ( email )

19th Avenue South
Minneapolis, MN 55455
United States

Jingjng Zhang

Indiana University - Kelley School of Business - Department of Operation & Decision Technologies ( email )

1309 E. Tenth Street
HH4143
Bloomington, IN 47401
United States

Gediminas Adomavicius

University of Minnesota - Twin Cities - Carlson School of Management ( email )

19th Avenue South
Minneapolis, MN 55455
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
210
Abstract Views
617
rank
205,616
PlumX Metrics