Guidance

Guidance for manufacturers on reporting adverse incidents involving Software as a Medical Device under the vigilance system

Published 15 May 2023

What to report

Any event which meets the three reporting criteria (MEDDEV 2.12/1 rev 8, 5.1.1) is considered an adverse incident and must be reported to the MHRA.

For software as a medical device (SaMD), indirect harm is the most probable outcome of adverse incidents and may occur as a consequence of the medical decision, action taken/not taken by healthcare professionals and/ or patients and the public based on information or result(s) provided by the SaMD.

Examples of indirect harm include:

• misdiagnosis
• delayed diagnosis
• delayed treatment
• inappropriate treatment
• absence of treatment or
• transfusion of inappropriate materials.

Examples of what indirect harm may be caused by:

• imprecise results
• inadequate quality controls
• inadequate calibration
• false positive or
• false negative results.

For self-testing devices, a medical decision may be made by the USER of the device who is also the patient.

Examples of types of adverse incidents that may be reportable

Performance issues

Adverse incidents resulting from the failure to perform according to the performance characteristics specified by the manufacturer. Examples might include, but are not limited to the following:

• Mental health tool fails to alert healthcare professional as per the instructions for use when user’s mental health status reaches a predetermined clinical threshold.
• Compatibility issues arising from operating system updates for Continuous Glucose Monitoring (CGM) apps such as loss of ability to see real time data leading to inappropriate or absence of treatment or a delay in diagnosis.
• Remote monitoring of vital signs in mental health hospitals and care homes could lead to delay to diagnosis and treatment if it fails to detect the correct physiological parameters e.g. pulse rate, respiratory rate and oxygen saturation.
• Activity tracker tools which fail to alert health professionals when a resident in a care home is deteriorating, or is at risk of physical deterioration, could cause delay to diagnosis and treatment.
• Artificial Intelligence (AI) diagnostic and triage tools failing to identify clinically relevant brain image findings related to acute stroke leading to an incorrect, delayed or missed diagnosis.
• AI tools intended to identify “normal” x-rays misses an abnormality leading to an incorrect, delayed or missed diagnosis.
• Accelerated MRI software that degrades the appearance of anatomical and pathological structures leading to an incorrect, delayed or missed diagnosis.
• Software tools that read the result from a lateral flow test incorrectly resulting in an infected individual not isolating and therefore spreading a disease.

Diagnostic accuracy issue

Adverse incidents resulting from the failure of diagnostic tools. Examples of reportable adverse incidents of devices such as symptom checkers (Guidance Appendix 1 – Symptom checkers) which are intended to be used by lay users.

• Symptom checkers used to analyse inputted symptoms and possibly identifying an incorrect medical condition or failing to identify a correct condition leading to a misdiagnosis, false reassurance or delay to treatment.
• Dermatology apps used for melanoma detection providing inaccurate results which could stop an individual from seeking an expert clinical opinion from a healthcare practitioner.

Decision support software resulting in harm

Examples of reportable adverse incidents of devices such as clinical calculators (Guidance Appendix 2 – Clinical Calculators) which are intended to be used by healthcare professionals.

• Overestimated / underestimated scores in predictive algorithm used to support medical practitioners to help assess the potential risk of cardiovascular disease in patients. As a result, may cause inappropriate or absence of treatment.
• Medication error occurring from incorrect calculations of dose calculators.
• Patients receiving inappropriate treatment for radiotherapy due to incorrect dose calculations.

Issue with connected hardware/software

Any harm caused by the device that runs the software should also be reported. Examples might include, but are not limited to the following:

• A multipurpose watch running a SaMD monitoring app causes skin burns.
• Use of non-compatible antivirus software causes malfunction of the SaMD.

Human-Device Interface Problem

Examples of adverse incidents which result from failures in the user interface.

• Failure of keyboard or touch screen to allow accurate input of data.
• Failure to account for colour blindness leads to a missed warning in the interface.
• Unclear user interface instructions or usability resulting in variable understanding of required input data or inaccurate data entry. For example, when a user fills out a yes or no questionnaire, if the position of the yes or no option changes midway through the questionnaire, this may cause the user to select an incorrect option leading to inaccurate data entry. Another example is when a user must select an incorrect option to proceed to the next question as the correct option is not available.

Use error resulting in harm.

Adverse incidents resulting from use error are considered reportable if there is actual harm or if there is a significant change in the trend/pattern of issues that could result in harm (guidelines on a medical device vigilance system, MEDDEV 2.12-1 - 5.1.5.1). For example:

• Triage tool used off-label (against disclaimer) results in a missed or incorrect diagnosis.
• User makes mistake entering values into contraception app leading to user relying on incorrect output and consequently gets pregnant.
• User instals an anti-virus tool not recommended by the SaMD manufacturer that causes performance issues with the SaMD.

Inadequate labelling and instructions for use

Any incident which has resulted from inadequate or misleading labelling is reportable. Additionally, any adverse incident such as the example below resulting from confusing or inadequate instructions for use is reportable.

• Inadequate or missing cleaning instructions for a tablet based SaMD leads to patient cross contamination. Manufacturers shall consider the risks from the device system. For software this will include hardware equipment.

Computer system security problem

Examples of reportable adverse incident impacting dosing, functions, user interface, safety information and patient status due to systems identified as having vulnerabilities to a cyber-attack.

• During active patient monitoring, this could cause loss of monitoring and/ or loss of alarm.
• Corruption of transferred patient data leading to incorrect, delayed or misdiagnosis.

How it should be reported

General guidance on vigilance is provided here: https://www.gov.uk/government/collections/medical-devices-guidance-for-manufacturers-on-vigilance

Report as individual events

Adverse incidents, such as the examples in any of the above categories, should be reported to MHRA. Manufacturers are advised to monitor customer feedback on social media channels and UK app stores and report where appropriate.

Each initial report must lead to a final report unless the initial and the final report are combined into one report. The final vigilance report should include the manufacturer’s final assessment, including root cause and details of any similar incidents and corrective or preventative actions with timescales as appropriate.

Report as periodic summary reports (PSR)

Some adverse incidents are appropriate for periodic summary reporting. Details of the timing and content of periodic summary reports should be arranged on an individual basis with the MHRA (MEDDEV 2.12/1 rev 8, 5.1.2).

Report at time adverse trend is identified

A trend report should be reported to MHRA if there is a significant increase in incidents that are usually excluded from individual reporting. The manufacturer is expected to have suitable systems in place to enable proactive scrutiny of trends occurring with their devices. A trend report is required where this is a significant increase in the rate of already reportable incidents, incidents that are usually exempt from reporting and events that are usually not reportable.

Report Information

Individual reports are to be submitted via the MORE portal. To use the MORE portal, you must register with us.

The MHRA would expect hardware and operating system model and version numbers, if relevant, to be included in section 2.6 of the MIR form.

If you are interested in setting up an API to submit your reports directly from your internal IT systems to ours, please contact us for further information. See the MORE production API guidance document for more information about how to set up your API.