Guidance

Criteria for health app assessment

Published 9 October 2017

Evidence of effectiveness

All apps must work and must be clear about their purpose, their benefits to patients and medical practitioners, and the outcome they want to achieve.

They must also be grounded in the best and most up-to-date knowledge, derived from research, clinical experience and patient preferences.

You must provide evidence that the app:

• improves outcomes for patients and users
• provides value for money
• meets user needs
• is stable and simple to use, and that people actually use it

Evidence based on independent research will score highly on an assessment.

App developers will need to demonstrate a high level of clinical effectiveness for the app to be considered for ‘NICE evaluated’ status. This will represent the gold standard for apps.

Apps must show that they meet criteria covering:

• clarity of purpose and intended use
• their evidence basis
• the data that forms the basis their evidence and findings
• any published academic studies

Read the NICE commissioned Evidence Generation Guide; Evidence Guide for App Developers; and Digital Health Evidence Case Studies produced by the York Health Economics Consortium.

Regulatory approval

The regulation of health apps provides patients and healthcare professionals the assurance that apps are high quality, safe and ethical.

If you’re building an app, there may be regulations that you need to conform to before being considered for the app assessment process. The main 2 types of regulation are:

• medical device regulation
• Care Quality Commission (CQC) registration

Medical device regulation

Medical devices must be registered with the Medicines and Healthcare Products Regulatory Agency (MHRA) and have a CE mark before continuing in the app assessment process.

If your app meets the definition of a medical device then it will need to be regulated by the MHRA.

CQC registration

If your app provides a health or social care service that fits in one of the 14 regulated activities, you’ll be required to register with the CQC before continuing in the app assessment process.

Clinical safety

We can’t endorse any app that can cause harm, for instance by miscalculating a drug dose or giving incorrect medical advice to a consumer or patient. This would also be the case if the app is unstable, and crashes midway through a diagnosis.

All apps must be clinically safe: this means that they they’re safe for people to use - not just for healthcare professionals and patients but everyone involved in creation, testing and approval of the apps.

You must outline:

• plans and policies to limit and mitigate risk for all apps
• any risks that the app could pose to people’s health if it crashes or is used incorrectly

Apps must meet the requirements of the NHS clinical risk management standards.

It must be shown that they meet criteria covering:

• mandatory NHS information standard DCB0129
• health and safety
• risk assessment and mitigation
• identification of potential adverse events

Privacy and confidentiality

Apps must capture and handle personal data legally and securely, and must make sure that the end user understands what the app will do with any data they provide. The user must be able to give ‘informed consent’ to the use of their personal data.

In effect, this means that the app - and any back-end systems it links to - must ensure that all data relating to a user is kept private and secure.

It must also explain clearly to users exactly what will happen to their data: who it will be shared with and whether it will be anonymised. The same goes where the app will pull in patient record information from NHS systems. As a minimum, it must comply with NHS Information Governance requirements.

Apps must meet criteria covering:

• the collection and processing of user data
• the ability for users to make an informed decision about whether they’re happy to use the app
• who the data will be shared with, and how long it will be retained

The Information Commissioner’s Office (ICO) Guide to data protection outlines how app developers should meet the Data Protection Act.

The Records Management Code of Practice for Health and Social Care 2016 sets out what you need to do to manage records correctly and how long you need to keep records.

The Information Security Management: NHS Code of Practice is a guide to the management of information security for people who work in or with NHS organisations in England and will advise you on the process and use of NHS information.

Security

You must ensure that user data is collected, transmitted and stored safely. You need to consider:

• the technologies you use
• your policies
• your practices

Check that your app is built to the required standards and test it for completeness and consistency with the OWASP Mobile Security Testing Guide (MSTG).

The mobile security standards include a number of checks to show that your app’s processes and architecture are secure. This applies to the collection, transmission and storage of user data.

You’ll need to demonstrate that you’ve addressed all security concerns and vulnerabilities, and explain how you’ve done this.

All apps must meet criteria covering:

• data storage and privacy
• authentication and session management
• network communication

Usability and accessibility

All apps need to meet the needs of a diverse set of users, including people with disabilities or those with limited technical knowledge.

Your app must be centred around users’ needs. This must include:

• the way you write
• the navigation you adopt
• the types of content you include in your app

You’ll also need to show that you’ve followed the Web Content Accessibility Guidelines (WCAG) and have evaluated your app with users during all stages of app development and deployment.

You’ll also need to show how you will make continuous improvements to your app following user feedback.

Apps must meet the criteria and be:

• easy to understand
• easy to operate
• informative

Interoperability

If your app needs to communicate with clinical systems to share data, you will need to ensure it complies with the relevant technical standards. For example, you’ll need to do this if your app writes clinical information to records held by GPs, or allows users to access their own records.

You will need to demonstrate that the app - and its back-end systems - will share data seamlessly with other clinical systems and software. There are rules around how you capture, present and store data, as well as the protocols you use to share this data with other systems.

As a minimum you should follow globally-accepted standards in the work you do. You’ll need to follow the standards set out in the NHS Interoperability Toolkit.

All apps must meet criteria covering:

• data sharing
• service level agreements for your services and APIs
• reliance on third-party services

Technical stability

All apps must be robust and stable.

It’s not enough to show that you have fixed all issues prior to launch. You should also provide a plan that explains how your app will continue to be developed and managed, and what resources are in place to test and monitor it for technical faults during its lifetime and when a new version is released.

You must also prove that your app has the ability to rollback to previous version should you encounter any significant problems following an update, and will limit and mitigate any risk to patient data and, more importantly, patient health.

All apps must show that they meet criteria covering:

• quality assurance
• service management
• product development