Study evaluates quality indicator for health apps

A study by researchers from the School of Computing at the University of Ulster, carried out in partnership with the Organisation for Healthcare Application Analysis (ORCHA), sought to assess the need for a more in-depth understanding of the quality of digital health apps.

With the curious title “Don’t Judge a Book or Health App by its Cover“, the research analysed 880 health apps for Android and concluded that user ratings and download rates are not reliable indicators of quality. The researchers argue that in an era when digital health apps are increasingly used to manage various aspects of health and well-being, the disconnect between popularity and safety and quality metrics needs to be revealed, with important implications for app developers, healthcare providers and legislators.

“Our research emphasises the importance of looking beyond metrics such as user opinion and number of downloads when evaluating digital health apps. While these metrics can influence user perceptions, our research shows that they don’t necessarily reflect the quality, effectiveness, safety or reliability of these health apps,” says Raymond Bond, professor of human computer systems at the University of Ulster and head of the PhD research project.

The professor emphasises that the focus needs to shift from popularity to quality, as users deserve access to high-quality digital health interventions that are evidence-based, easy to use and designed to improve health and wellbeing outcomes. “Relying solely on user ratings and download counts can lead to misguided decisions and eventually undermine public confidence in digital health technologies,” Bond points out.

According to the researchers, by recognising the limitations of traditional metrics and prioritising more rigorous evaluation processes, the various stakeholders, including healthcare professionals who recommend apps to patients, can ensure that these apps meet high quality standards and contribute significantly to patient care and well-being.

The results suggested that there is no significant correlation between users’ subjective evaluations, the number of downloads and objective measures of the quality of health apps. They also indicate the need for wider application of methodologies that can accurately determine the quality, safety and compliance of health apps. More measures should be taken to enable users to recognise high-quality apps, including digital health education and the provision of nationally approved “libraries”.

According to a 2021 report, there were more than 350,000 health apps available in the iOS and Android shops, with around 250 health apps added every day. In addition, also increased during the Covid-19 pandemic.

Conducted as part of a PhD programme between 2021 and 2024, the University of Ulster research examined a unique dataset of more than 880 Android health apps provided by ORCHA, using an evaluation tool that rated the apps on a 300-point scale in three domains: user experience, data privacy and professional/clinical assurance. The study was published in the journal PLOS ONE.

Mental health

One category of digital apps that has been gaining ground since the Covid-19 pandemic and therefore needs more attention, especially when it comes to quality, is mental health. This market was valued at $5.2 billion in 2022 and is expected to reach $23.8 billion by 2032, with a compound annual growth rate (CAGR) of 16.9 per cent between 2023 and 2032, according to figures from Market.us. The main factors driving growth are the rising rates of mental disorders such as depression and anxiety, as well as the convenience and accessibility offered by digital applications.

It is predicted that the Apple iOS system will dominate the mental health app market and that depression and anxiety management will hold the largest share of revenue.

Last month, Fortune magazine presented a list of the best mental health apps for 2024, according to a ranking based on research by a former therapist. Meanwhile, the Mobile App Daily website brought together 100 health app developers recognised for creating innovative, safe and easy-to-use solutions adapted to meet the growing needs of the medical field.