How to generate evidence for digital health solutions

Digital solutions in healthcare have grown rapidly recently, but developers still face considerable challenges in presenting evidence in terms of safety, efficacy, and value generation. Presenting evidence for medicines and treatments in traditional medicine is routine, but how should it be approached in the digital arena? There is still a lack of widespread consensus on the types and levels of evidence that health technology solutions should generate.

Roche, a Swiss multinational pharmaceutical and diagnostic company, has sponsored a paper that explores challenges and possible approaches to generating robust evidence for digital solutions to help drive large-scale innovation for clinicians, healthcare professionals and patients.

The research was based on interviews and discussions with experts in the US, EU, and UK and included representatives from industry, clinical practices, academia, policymakers, and regulators. Several questions were raised, including what is meant by “evidence generation” in digital health and why this is important. It also addressed the main challenges and how to help startups and innovators generate evidence and conducted a survey of 144 healthcare leaders from around the world, asking about the importance of different types of evidence in their decisions to purchase digital solutions.

According to the study, there are several means of generating evidence that developers can adopt. This can be evidence based on literature or real data to quantify a clinical problem, validate a concept and inform product development. Subsequently, evaluations can be conducted to verify the safety, efficacy, and value generation of solutions. Although many types of evidence can be generated at any stage of the development cycle, some are more strongly associated with certain stages.

As the generation of evidence in digital health is still in its infancy, there is a lack of consensus around terminology. For this reason, the study highlights that it is critical to have common ground with important definitions to facilitate communication between stakeholders, including patients, healthcare professionals, software engineers, academics, manufacturers, regulators and policymakers.

Another warning made by the study is that traditional methods considered the best tool for evaluating the effectiveness of drugs and other interventions may be unsuitable for digital health solutions because they are incompatible with the typically fast pace and iterative nature of software development. In addition, they can require a lot of time, money and other resources.

Therefore, new evaluation approaches are needed for new technologies. Software is fundamentally different from drugs and medical devices and therefore should not be treated in the same way, reinforces the study. So-called Software as a Medical Device (SaMD) is still in its infancy and lacks specific validation methods.

Generating evidence under the microscope

There is no single, exhaustive way to categorize evidence generation activities. However, many experts have divided the process into three broad fields: technical, clinical and economic.

In addition, most types of evidence can be generated at various stages of the product life cycle, but some may be particularly associated with certain phases. For example, clinical results that demonstrate the safety and efficacy of the solution are essential for regulatory approval, while economic analyses are important to ensure reimbursement. Already in the early stages of ideation, technical professionals can generate evidence to validate a concept by conducting user surveys and gathering descriptive studies.

The main challenges to generating strong evidence for digital health solutions highlighted in the research are:

• Lack of knowledge: In many cases, there is no clear information about the evidence requirements demanded, for example, by regulatory bodies. In many countries, we are already seeing progress in this area, with evidence standards being established and communicated.
• Financial cost: Generating evidence can be an expensive process. There is no incentive to invest in developing robust evidence. Reimbursement routes should be established.
• Data availability: Developers often lack access to data to aid the evidence generation process. Poor interoperability, isolated data architectures and inconsistent data policies across regions limit progress in this area.
• Methodological challenges: Traditional research methodologies used in drug evaluation and other procedures are inappropriate for many digital health services. In addition, the rapid software update cycle makes traditional methods redundant in some situations.
• Digital health literacy: Low levels of digital health literacy among both patients and health professionals is a considerable barrier to evidence generation and wider adoption of these solutions.

New approaches

According to the Roche study, in the early phase of product development, there is scope to employ a variety of methods for generating evidence, such as qualitative studies, secondary research (e.g. analysing clinical treatment lines or the healthcare system) and usability testing. However, these methods produce weaker evidence than traditional methods.

A more pragmatic approach might include clinical simulations, observational studies using real data, and platform trials. This option is a new type of study that may be useful for evaluating rapidly evolving digital health solutions, as trials are created, designed to be adaptive, allowing interventions to be modified or changed over time.

Recommendations for the parties involved

Developers and innovators should work closely with policymakers, regulators, clinicians and patients to ensure that the type and level of evidence expected is robust, realistic, reasonable and formalized. It is necessary to understand how to relate the methods of evidence generation to the stage of development of solutions.

There is also a need to promote greater awareness among healthcare professionals about digital solutions to enable the appropriate prescription of these resources. Health professional bodies and medical societies can play a key role in education and guidance in this regard.

On the other hand, innovators should be aware of the benefits of evidence generation, including accelerated product development. Public and regulatory bodies should formally establish the use of real evidence in decision-making processes, and provide clear guidance on how this evidence can be used to support decisions.

Health systems should also establish clear reimbursement pathways for digital health solutions. They should be accompanied by guidance on what evidence is required for inclusion in the procedure list.