Consultation outcome

Consultation document: MHRA draft guidance on randomised controlled trials generating real-world evidence to support regulatory decisions

Updated 16 December 2021

1 - This guidance is aimed at Sponsors who are planning to conduct clinical research to support a regulatory decision.

2 - For information about how patient data is collected and used, why research using patient data is essential for healthcare and benefits patients and the NHS, and the safeguards in place to protect patient privacy, visit Understanding Patient Data.

1. Introduction

3 - There are vast amounts of data routinely being collected on patients, for example, in electronic health records (EHR), and disease and patient registries. Such data is commonly called real-world data (RWD), reflecting that it is routinely collected while patients go about their regular lives, as opposed to being specifically collected in a clinical trial. When such data is analysed to make inferences about the effects of different treatments, the information produced is similarly called real-world evidence (RWE). While extensively used for monitoring the performance of drugs and devices after approval, RWE is utilised much less frequently when it comes to demonstrating the efficacy or effectiveness of an intervention to gain an initial approval or an extension of an indication for an existing product.

4 - Use of such pre-existing data sources has the potential for increasing the speed and reducing the cost of development programmes, which would see effective medications being approved more quickly, or even programmes which were previously thought to be unfeasible becoming feasible, with the consequent benefit to public health. There is also a view that RWE can be more representative of the true effects of a treatment in the community and more generalisable than data from the standardised setting of a conventional clinical trial.

5 - There is a growing level of activity and planning around the use of RWE in this arena globally^{[footnote 1][footnote 2][footnote 3][footnote 4]}. The MHRA has expertise in using RWD for pharmacovigilance based on UK primary care data, for example supplied via one of its centres, CPRD. Capabilities for delivering EHR-based trials for the purposes of comparative effectiveness have also been developed enabling greater insights into the uses of RWD.

6 - The MHRA encourages Sponsors to explore the opportunities presented by the utilisation of RWD and applicants with proposals in mind are invited to discuss these via the MHRA Innovation Office and or MHRA scientific advice meetings.

7 - To further support the clinical development programmes of Sponsors, the MHRA is producing a series of guidance documents to provide general points to consider when planning a trial to generate RWE, of which this is one. The guidelines will each be based on a different type of trial design which could be employed. This document discusses the points to consider when planning a prospective randomised trial using RWD sources. The document considers aspects related to clinical trial authorisation, clinical trial design including choice of endpoints and safety data requirements, and requirements in terms of database quality and inspection.

8 - The trials covered in this document all include randomisation of patients and therefore will be considered to be Clinical Trials of Investigational Medicinal Products (CTIMPs) requiring a clinical trial authorisation (CTA) from the MHRA. As such, there will be other guidance that may need to be considered when considering the use of RWD, including guidance applicable to all CTIMPs. Sponsors are directed to the clinical trials pages on the MHRA website and to seek assistance where required using the associated contact details.

9 - Running a randomised trial using a RWD source can generate RWE to support a regulatory decision. RWE from such trials is not generally considered of more or less value for regulatory decision making than evidence from conventional RCTs provided the data quality is robust and the trial well designed.

10 - Randomised trials using a RWD source are most likely to be considered for label changes for already approved products, including drug repurposing, but nothing is completely ruled out on principle, including the investigation of new products.

2. Scope and definitions

11 - This guidance provides points to consider when running a randomised clinical trial using a RWD source with the intention of using the trial to support a regulatory decision. Guidance is also given on points to consider if applying for approval to run such a trial in the UK.

12 - The guidance does not cover other types of studies that could be run using RWD. It also does not discuss wider issues regarding randomised clinical trials that are not specifically related to the use of RWD, either general points on randomised trials or disease and product specific requirements.

13 - The guidance in this document relates to interventional clinical trials of investigational medicinal products (“CTIMPs”) as defined in the Medicines for Human Use (Clinical Trials) Regulations 2004.

14 - A “clinical trial” means any investigation in human subjects, other than a non-interventional trial, intended to

1. discover or verify the clinical, pharmacological or other pharmacodynamic effects of one or more medicinal products
2. identify any adverse reactions to one or more such products, or
3. study absorption, distribution, metabolism and excretion of one or more such products, with the object of ascertaining the safety or efficacy of those products.

15 - A “non-interventional trial” means a study of one or more medicinal products which have a marketing authorisation, where the following conditions are met:

• The products are prescribed in the usual manner in accordance with the terms of that authorisation
• The assignment of any patient involved in the study to a particular therapeutic strategy is not decided in advance by a protocol but falls within current practice
• The decision to prescribe a particular medicinal product is clearly separated from the decision to include the patient in the study
• No diagnostic or monitoring procedures are applied to the patients included in the study, other than those which are ordinarily applied in the course of the particular therapeutic strategy in question, and epidemiological methods are to be used for the analysis of the data arising from the study
• An algorithm is available to assist in the determination of whether a study is considered non-interventional or not (see Appendix 2).

16 - A “Type A” clinical trial is one where the potential risk associated with the investigational medicinal product (IMP) is considered no higher than that of standard medical care. Examples are trials involving medicinal products licensed in any EU/EEA Member State if they relate to the licensed range of indications, dosage and form or, they involve off-label use (such as in paediatrics and in oncology etc.) if this off-label use is established practice and supported by sufficient published evidence and/or guidelines. See Appendix 3 for a description of the different types of clinical trial as defined in “Risk-adapted Approaches to the Management of Clinical Trials of Investigational Medicinal Products”.

17 - For the purposes of this document RWD is defined as routinely collected data relating to patient health status or delivery of health care. Sources of RWD include electronic healthcare records (EHR) defined as structured, digital collections of clinician-recorded patient level medical data. These include, but are not limited to, primary and secondary care records, disease registries, and administrative data on births and deaths. Other sources of RWD include patient reported data via wearable devices.

18 - RWE is defined as evidence derived from RWD.

19 - For the purposes of this document the main difference of interest between a conventional clinical trial and one producing RWE is that RWE is derived from data that is being routinely collected independent of a patient’s inclusion in the trial (such as the DECIDE study), whereas in a conventional trial the data are collected specifically for the trial. A conventional trial would also generally involve study specific visits which require attendance at a centre.

20 - Another commonly stated distinction between conventional clinical trials and those producing RWE is that in the conventional trial a very specific study population is pre-defined. This statement seems to reflect the objectives of particular trials rather than being an intrinsic property of the different study types. A conventional clinical trial could be run pre-defining a very broad study population and make efforts to recruit across the full range of that population, while a trial using a RWD source could choose to specify a narrow study population.

21 - A hybrid trial (such as the Salford Lung Study) is a trial where some of the data collected are RWD, and some are collected specifically for the trial outside of the RWD source. Such a trial produces RWE and is in the scope of this guidance.

22 - Pragmatic trials are trials which aim to investigate the effectiveness of a treatment in routine clinical practice. They contrast with explanatory trials, which investigate whether an intervention can have an effect in a carefully controlled situation. The terms explanatory and pragmatic are the two extremes of a continuum. Randomised trials using RWD sources can fall anywhere along this continuum. The pragmatic/explanatory terminology will not be generally used in this guidance, as it is not considered an important distinction for the purposes of regulatory decision making.

3. Types of RWD trials and points to consider

3.1 Simple trials

23 - In its simplest form a randomised, controlled trial using a RWD source involves patients in the database being randomised to one of a choice of interventions which could include standard clinical care alone. They are then followed as is routine practice for the database concerned. The interventions to be compared could be the intervention of interest added to standard of care, compared to standard of care alone. It could also be an active controlled comparison of an intervention of interest to another intervention.

24 - The protocol for the study should be of the same standard that would be expected for a conventional RCT including pre-specification of the objectives, data to be collected, primary and secondary endpoints and analysis methods.

25 - In a trial that is purely in the real-world setting no additional data is collected on patients aside from that which is routinely collected in the RWD source, and patients are not blinded to the treatment allocation. The triallists are given access to the specific data from the database required to complete the investigation described in the protocol. Aside from the act of randomisation and consent, the patient experience would not be altered by being in the trial. Patient consent is required before enrolment. Such a trial is likely to be classed as a Type A trial.

26 - Data from such a trial would provide RWE that from an MHRA perspective is equally acceptable (provided the data is robust and the trial well designed) for demonstrating the safety and efficacy of an intervention as data from a conventional open-label RCT investigating the same population using equivalent endpoints and comparators.

27 - Running a trial that would be acceptable for regulatory purposes in this way would be possible if the key endpoints necessary to make the regulatory decision are routinely collected in the database and are sufficiently objective such that they would not be subject to meaningful bias from the knowledge of treatment allocation in an open-label setting.

28 - If the endpoints are not sufficiently objective the trial would need to be blinded, unless otherwise justified. This could be particularly important if investigation of comparative safety criteria is a key aspect of the study. This would represent an additional burden to patients above their routine care, as specific trial medication would need to be received rather than an open prescription, but it can be done while avoiding the need for any additional deviations from routine.

3.2 Hybrid trials

29 - If additional endpoints are needed, a hybrid trial can be run, where as well as the routinely collected information, patients and/or healthcare professionals also provide trial specific data such as patient reported outcomes or additional clinical assessments. This could be done remotely via electronic data capture but may require specific study visits.

3.3 Safety monitoring

30 - Participants in a clinical trial should be monitored appropriately, with regular safety monitoring and mitigation steps based on the identified safety profile and risk-benefit review. This includes when licensed products are used in a trial and when all or some of the monitoring is through real world data collection activities. There is legal requirement in all clinical trials to report serious adverse events (SAEs), and this will apply to RWD trials. In particular there is a requirement that all suspected unexpected serious adverse reactions (SUSARs) will be reported as per the required timelines.

31 - A risk-proportionate approach can be taken to Type A trials. The UK has published guidance on Risk-adapted Approaches to the Management of Clinical Trials of Investigational Medicinal Products (Appendix 2). For example, the nature and extent of patient safety monitoring should be based on the assessment of the risks of the trial intervention(s) relative to standard care and the extent of knowledge about the IMPs being tested. Any proposals should be clearly outlined in the trial protocol but can be discussed in advance with MHRA.

32 - If products are being used outside of their licensed indications, then more frequent contact with patients is likely to be required to collect information, such as for adverse event reporting outside of routine care. The extent of safety monitoring needed would be considered on a case by case basis, but a minimum standard would still be required.

3.4 Regulatory acceptability of RWD-based trials

33 - The more involvement outside of routine care that is required, the smaller the advantage of running a RWD based trial becomes, and not all the evidence generated by the trial would be classified as RWE.

34 - However, from a regulatory perspective, whether the randomised trial is classified as generating RWE or as providing a hybrid dataset (or if it is a conventional clinical trial), is not critical. The important thing is that the trial is designed in a way which allows it to provide the evidence required to answer the regulatory question. The considerations regarding which endpoints are required, whether blinding is necessary, and any of the many other decisions made when designing, conducting and analysing a clinical trial depend upon the questions the trial seeks to answer and not upon the source of data. Existing regulatory guidance remains relevant to these requirements.

4. Clinical trial authorisation considerations

35 - Given randomisation is involved, all trials described in the scope of this document are Clinical Trials of Investigational Medicinal Products (CTIMPs) requiring authorisation by the MHRA.

36 - The regulatory situation will impact the Adverse Event (AE) reporting requirements. The starting point would always be full recording and reporting as per the Medicines for Human Use (Clinical Trials) Regulations 2004, and that reporting of Suspected Unexpected Serious Adverse Reactions (SUSARs) are key.

37 - However, there is some flexibility around the requirements for safety reporting, with more or less flexibility depending on the proposed population and expected AE profile compared to what is already known about the safety profile, and this would be considered on a case by case basis. For example, for type A trials risk-adapted approaches can be taken in line with the guidance contained in the MRC/DH/MHRA joint project document “Risk-adapted Approaches to the Management of Clinical Trials of Investigational Medicinal Products”.

38 - Detailed adverse event reporting necessitates more frequent contact with patients and the collection of information outside of routine care. The more involvement outside of routine care that is required, the smaller the advantages of running a RWD based trial become. Therefore, noting points the two paragraphs above, randomised trials using a RWD source are most likely to be considered for label changes for already approved products, for example:

• use in a new population (different age group, different disease severity, etc. to what is already licensed) where appropriate
• change in dose, or route of administration
• adding a new indication (repurposing of existing medications)

This is not a comprehensive list of possible situations, and nothing is completely ruled out on principle, including the investigation of new drugs.

39 - Otherwise the same requirements for clinical trial approval that would apply to conventional RCTs are relevant. 

5. Quality of the data

40 - The data from the database (or other data source) being used must be demonstrated to be of sufficient quality. Published guidances for good database selection from other fields, including pharmacoepidemiology, are applicable. Areas of consideration include:

• Will the database be used as the source population for recruitment, or be used to supplement other data sources? Will it include an appropriate population in terms of size, coverage and representativeness?

• Are important baseline characteristics measured regularly and likely to be up to date?

• Will the database capture the interventions, outcomes and other study variables in sufficient detail, consistently and without bias, with the level of frequency and completeness that is required for the trial? If not, how will data from the database be combined with additional data sources for the trial? It is recommended that a feasibility study is conducted to assess the capture of study variables prior to undertaking a RWD trial.

• How will changes in data collection during the study period be handled, both for individual patients, e.g. those who move between healthcare professionals or the whole population, e.g. changes in coding systems? This could be a particular issue for studies including paediatric patients transitioning to adult care during the course of the study
• Is the time between the occurrence of events and availability of the data to the trial team suitable for the usage of the data in the trial? How soon will data be able to be analysed after last patient, last visit? If used for monitoring adverse events, what impact would the availability of data have on the suitability of the database?
• What method will be used to link the database to additional data sources for the trial? Is reliable linkage likely to be possible for the trial population?
• Which privacy and security policies apply to the use of the database? What restrictions apply on the transfer, storage, use, publication and retention of the data?
• What data quality checks are undertaken by the data custodian? Which additional trial-specific checks will be needed?

41 - Sponsors should be able to provide a description of the tools and methods for selection, extraction, transfer and handling of data and how they have been validated. It is essential that processes are established to ensure the integrity of the data from acquisition through to archiving and sufficient detail captured to allow for the verification of these activities.

42 - The majority of MHRA GCP inspections are carried out under the risk-based compliance programme. These can be either systems-based or trial specific. Inspectors work in conjunction with assessors to identify trials for inspection based on a wide variety of factors including areas of interest e.g. ePROs, novel interventions or based on regulatory triggers.

43 - Inspection of the systems and processes used for the oversight of RWD suppliers and the subsequent onward management of RWD data may also occur in conjunction with an inspection of a Sponsor or CRO. Areas of particular interest for review would include randomisation methods, data management, IMP management, safety reporting, Sponsor oversight and where externally provided databases are used to collect trial data. Processes used to assure the integrity of the reported data form a golden thread throughout any inspection of a licensing application and are a common focus during inspection.

44 - Acceptable processes would need to be in place for any additional data collected outside of the main source database.

45 - General Principles:

• As for all trials, data quality and assurance and appropriate data management oversight is essential
• Data quality processes must be detailed in the trial protocol and these must be appropriately validated to ensure that any process is fit for its intended use
• Data quality assurance checks should be conducted with the ability to audit all data values to comply with GCP.
• Data should meet agreed specifications and requirements to ensure that any received data contains only expected data files and that all data elements are structured correctly as per the agreed specification

6. Examples

46 - Noting the points above, an example of a straightforward situation for a RWD based trial would be:

• A trial embedded in routine practice with a well-established EHR database
• An objective endpoint routinely and consistently collected in the EHR database for the patient population considered for the trial, for example, all-cause mortality
• The intervention is an existing licensed product with a well-established safety profile, to be added to standard of care, and compared to standard of care
• The intervention is already licensed for patients with a severe version of the condition and this trial aims to explore the benefit in patients with a milder form of the same condition

47 - For this example, the safety data collection requirements would be minimal as the product has a known safety profile and it would not be expected that significant additional knowledge would be gained on safety from the new trial. Also, as the safety profile is known to be well-established, patients would not be considered to be at risk in this trial such that close monitoring is needed. Given that the product is already known to be efficacious in the same indication for another population, it would not be considered necessary to have a lot of supportive secondary endpoints and no additional data outside of that collected in routine care would be needed. As the endpoints are objective an open-label trial is acceptable.

48 - The endpoint mentioned in the example, all-cause mortality, is particularly suitable for a RWD based trial with no blinding as it is objective and also does not require a patient to take any action to have the endpoint recorded. It is also not based on any time schedule and has a consistent method of being recorded and interpreted. If the event occurs it will be recorded in the EHR database, and if it does not occur no action needs to be taken. All the data needed to capture the incidence of events after a particular duration, or the hazard ratio for comparing the time-to-event can be obtained with no additional patient burden.

49 - The completeness and validity of the recording of potential endpoints in EHR should not be assumed. For example, acute myocardial infarction, which may be perceived to be well recorded within primary care, was shown to be recorded with poor completeness in primary care, secondary care admissions data, and a disease specific registry, with each data source missing 25-50% of events . As mentioned previously, it is recommended that a feasibility study is conducted to assess the capture of study variables prior to undertaking a RWD trial, and data linkage or supplementary data collection considered as required.

50 - Other measures which are used as endpoints, such as blood pressure recordings, while still objective and forming part of the EHR database, depend upon conscious actions to be recorded. If it is desired to understand the effect of treatment on blood pressure after a particular time interval, for example 6 months, it is necessary that a blood pressure measurement is taken at that time for all patients, as a large amount of missing data would undermine the quality of the dataset. Using the value for the patient that is closest to 6 months or another imputation technique is not an adequate replacement if a value is not generally recorded at 6 months. Unless all patients are scheduled to have blood pressure measured at regular intervals as part of their routine care in such a way that an assessment 6 months after randomisation is expected, and it can be seen that this is reliably achieved, it would be necessary to at least perform a hybrid trial by scheduling blood pressure measurements outside of the patient’s routine care at the particular time-points of interest. Digital health technologies (see below) have the potential to be a source of RWD for such endpoints. Irrespective of the source of RWD, data quality remains of critical consideration and any technology used must be validated.

51 - ‘Digital health technologies’ can be used to gather health related data. They include sensors, wearables and other digital health technologies, such as ingestible devices and implantables. Such devices and apps might be used to collect RWD as part of routine clinical care for patient reported outcomes or home-based measurements. For example, completing an activities of daily living questionnaire online before attending an appointment, or using an oxygen saturation probe to monitor a patient with congenital cardiac disease awaiting intervention. Other sources of RWD outside routine care might utilise devices to understand patient or caregiver experience, patient preferences, and indicators of function (for example daily step count). These might be used as supportive evidence of safety or efficacy of a treatment. It is important that both the device and any tool (such as a questionnaire) are suitability validated for the measurements required, and regulatory approval sought where required. The relevance, objectivity, and practicality of measurements should be considered, taking into account the disease, age, and potential functional abilities of the user.

52 - Returning to the example at the start of this section, if the new population is significantly removed from the existing licensed population such that there is minimal confidence the existing knowledge of the safety profile could be carried across, additional safety monitoring and reporting outside of routine practice could be required.

53 - If the clinical situation necessitates that many additional requirements outside of routine practice be incorporated into the trial then there may be few advantages over a conventional RCT, or a conventional RCT may become necessary. However, there is a large middle-ground before this becomes the case. The MHRA is keen to meet with Sponsors to discuss their trial designs for any specific situation.

54 - An area of great interest is the repurposing of existing licensed drugs for new indications. This can often not be viable if a conventional RCT is required, but could be viable in a RWD setting and Sponsors are encouraged to approach the MHRA to discuss their plans.

7. Advice

55 - This guidance lays out general principles and points to consider for Sponsors considering running a trial to generate RWE but cannot be comprehensive. If advice beyond what is contained in this guidance would be of interest, please contact the MHRA Innovation Office or request a scientific advice meeting. A scientific advice meeting to discuss your plans can include experts from Licencing division including the CTU, VRMM, Devices, IE&S, and the MHRA specialist research data services centre, CPRD, as applicable. See Appendix 1 for a description of the MHRA divisions and centres.

1. FDA. (2018) Framework for FDA’s Real-World Evidence Program. ↩

2. HMA/EMA. (2019) HMA-EMA Joint Big Data Taskforce Phase II report: ‘Evolving Data-Driven Regulation’ ↩

3. Innovative Medicines Initiative. (2018) GetReal Initiative. Accessed at: www.getreal-initiative.eu ↩

4. The Academy of Medical Sciences. (2018) Next Steps for Using real World Evidence ↩