Health app reviewers: the assessment process
How health app content should be reviewed by subject matter experts.
Each app must be reviewed by subject matter experts (SMEs) to check that any health-related content:
- is accurate
- meets national guidelines
- can provide evidence for any health claims it makes
How the process works
Developers will have completed a self-assessment to ensure their app meets the required level of technical and clinical best practice.
Once an app has been approved via self-assessment, SMEs will act as an additional panel of reviewers to assess the app’s content from a number of more specialised angles.
This broadly covers 2 areas:
- health content
- behaviour change theory
Before you review the app you may be asked to sign a non-disclosure agreement (NDA) to protect the intellectual property (IP) of app developers. You’ll also need to declare any potential conflicts of interest.
What validation means
The validation stage aims to:
- verify the answers given in the developer’s self-assessment
- identify and address any concerns and issues with the answers provided by the developers
As an app reviewer, you will:
- use an objective process (based on data and evidence) to evaluate any app health or behaviour change content against a set of criteria deemed appropriate for the topic area by the relevant Public Health England team
- question developers and ask for documented proof of any elements of the app which you feel are not sufficiently covered - such as clinical evidence of impact
You can also provide feedback on improvements and changes to the app.
Self-assessment validation
Overall, the aim is to give an app a ‘pass’ or ‘fail’ score.
If the developer doesn’t provide enough information to back up their claims, ask them for good quality evidence to support the statements. If they still fail, you will need to provide feedback on improvements and changes.
Make sure you give developers a deadline for making changes and resubmitting evidence. We don’t suggest a cap to the number of feedback and change rounds: this is at the discretion of your app assessing body. Ensure your feedback is clear and specific.
Apps that complete this stage can then be used in confidence by public health bodies, and may be listed on the NHS Digital Tools Library.
Developers must provide good quality evidence to show that their app delivers the results that they claim. Their app will not be considered without good quality evidence. Evidence provided by an independent evaluating body will carry more weight than the developer’s own findings. All evidence should demonstrate the app’s clinical effectiveness and its benefits to the healthcare system.
Learn more about how to provide evidence from these guides commissioned by the National Institute For Health and Clinical Excellence (NICE).
Contact
Contact digital@phe.gov.uk for more information.