A Bitter Pill to Swallow? The Consequences of Patient Evaluation in Online Health Q&A Platforms
49 Pages Posted: 3 Jun 2019
Date Written: March 26, 2019
Online Health Q&A platforms (OHQPs), where patients post health-related questions, evaluate advice from multiple doctors and reward a bounty to their most preferred answer, have become a prominent channel for patients to seek and receive medical advice in China. These platforms exhibit common design features, including bounty-motivated problem solving, limited search functionality, and lack of peer-assessment, but it is unclear if these platforms promote good medical advice. We analyzed data on patients’ evaluation of 497k answers to 114k questions on one of the most popular OHQPs over a 3-month period. We assembled a panel of independent (offline) physicians and instructed them to professionally evaluate the quality of 13k answers. We found that patients chose suboptimal advice in over two thirds of all cases. The medical accuracy of patient evaluation was even worse in critical categories (cancer, heart and liver disease) and for vulnerable subpopulations (pediatrics). Given that millions of patients seek medical advice from OHQPs in China annually, the social and economic implications of this finding are troubling. To understand why, we leveraged natural language processing techniques to construct a rich set of answer features and estimated a deep learning model to determine how patients responded to features. Importantly, we identified the extent to which patients respond positively or negatively to different heurist cue features. We found strong evidence that the platform reputation metric effectively encourages patients to select suboptimal medical advice, increases the moral hazard for physicians on the platform to provide advice that is less accurate but more agreeable to patients in exchange for bounty, and that OHQPs enable or exacerbate care avoidance. We discuss several potential policy changes to address these shortcomings.
Keywords: online health, patient evaluation, care avoidance, health advice
Suggested Citation: Suggested Citation