An app a day will (probably not) keep the doctor away: an evidence audit of health and medical apps available on the Apple App Store

Abstract

Background: There are more than 350,000 health apps available in public app stores. The extolled benefits of health apps are numerous and well documented. However, there are also concerns that poor-quality apps, marketed directly to consumers, threaten the tenets of evidence-based medicine and expose individuals to the risk of harm. This study addresses this issue by assessing the overall quality of evidence publicly available to support the effectiveness claims of health apps marketed directly to consumers.

Methodology: To assess the quality of evidence available to the public to support the effectiveness claims of health apps marketed directly to consumers, an audit was conducted of a purposive sample of apps available on the Apple App Store.

Results: We find the quality of evidence available to support the effectiveness claims of health apps marketed directly to consumers to be poor. Less than half of the 220 apps (44%) we audited state that they have evidence to support their claims of effectiveness and, of these allegedly evidence-based apps, more than 70% rely on either very low or low-quality evidence. For the minority of app developers that do publish studies, significant methodological limitations are commonplace. Finally, there is a pronounced tendency for apps – particularly mental health and diagnostic apps – to either borrow evidence generated in other (typically offline) contexts or to rely exclusively on unsubstantiated, unpublished user metrics as evidence to support their effectiveness claims. 

Conclusions: Health apps represent a significant opportunity for individual consumers and healthcare systems. Nevertheless, this opportunity will be missed if the health apps market continues to be flooded by poor quality, poorly evidenced, and potentially unsafe apps. It must be accepted that a continuing lag in generating high-quality evidence of app effectiveness and safety is not inevitable: it is a choice. Just because it will be challenging to raise the quality of the evidence base available to support the claims of health apps, this does not mean that the bar for evidence quality should be lowered. Innovation for innovation’s sake must not be prioritized over public health and safety.