Guidance

Oversight and monitoring activities

Published 28 January 2022

Oversight and monitoring activities can include a broad range of activities, for example;

• the use of committees to manage the trial or review the emerging safety data
• central review of clinical trial data
• documents and reports
• feedback from questionnaires sent to investigators
• data management processes
• statistical review of the data from the trial
• pharmacovigilance signal detection
• regular review meetings (e.g. sponsor with Contract Research Organisation (CRO)/Chief Investigator).

Audits and visits to the investigator site by a trial monitor and auditors to assess the conduct of the trial is considered to be part of oversight.

Oversight

Oversight is important where the sponsor has delegated functions to other parties, either within the same organisation (for example, to a Chief Investigator within the Trust) or to an external CRO. The sponsor’s project management or governance should have sufficient processes in place to verify that the functions are being conducted appropriately. The sponsor should be approving documents and processes implemented to carry out the delegated functions such as (not exhaustive):

• protocols,
• case report forms (CRFs),
• standard operating procedures (SOPs),
• analysis plans,
• data management plans.

The delegated parties would typically be implementing some or all of the detailed processes decided from the monitoring strategy. Oversight would usually involve assessment of the processes that are to be used by those delegated parties, regular review meetings with personnel, review and approval of specific documentation (such as site visit report, regulatory green light documentation) or perhaps visits or co-visits to investigator sites and audit.

Monitoring

Monitoring is one of the key mechanisms whereby the sponsor can be assured that it is in compliance with the legislations, and the trial protocol/procedures. Effective monitoring may also provide an opportunity for feedback to the sponsor for process improvement. Traditionally, “monitoring” has tended to focus on monitors visiting sites as part of quality control and this approach has been extensively used by commercial sponsors consisting of regular on-site visits covering the activities outlined in ICH GCP guidance. Non commercial trials have primarily taken a more centralised approach to monitoring activities with less reliance on on-site visits.

Audit

Audit activities are sample based and may occur during or after the trial is completed, or for cause in response to issues detected by monitoring activities. The auditor would also assess the effectiveness of the monitoring activities and compliance with the processes outlined in the protocol/monitoring plan/SOPs.

Important factors in determining the oversight and monitoring strategy

Monitoring may aim to assess compliance with every detail of the protocol and trial procedures and conduct checks of every data point for consistency with source documents and validity. Such an approach has been used traditionally and is extremely resource intensive. The key aim is to ensure the rights, safety and wellbeing of the trial participants are protected and that the final results of the trial are reliable.

It is recommended that a proportionate approach to the management and monitoring of the trial is undertaken based on the trial risk assessment that identifies the areas that matter to achieving the above key aim, i.e. those activities/data that, if incorrect, would have a negative impact on participant safety and trial results. A thorough risk assessment of the trial should be conducted to identify the risks and then determine the strategies and procedures to mitigate them. This risk-based approach is consistent with ICH GCP where the “sponsor should determine the extent and nature of the monitoring” .

It is not possible to provide an exhaustive list, but the strategy, as considered by risk assessment, is likely to depend upon:

• Trial categorisation as per the MRC/DHSC (Department of Health and Social Care)/MHRA risk adapted approach (A, B or C)
• Complexity of the trial protocol and procedures, for example some trials can have complicated protocols involving several different objectives and collection of extensive data
• Novelty of the trial, for example, the use of new methods/equipment
• Endpoint measurements, for example, could be directly collected/transferred electronically, for example if laboratory based
• Size, multicentre, multinational, number of participants, data quantity
• Use of vulnerable participants, for example consent process involving parents, carers, legal representatives
• Type of investigator site and experience of site staff
• Whether electronic data capture is being used – an eCRF allows additional information about who and when data were entered etc. (from the audit trail)
• Use of commercial or academic contract research organisations, which would involve their assessment
• Complexity and ease of use of the Case Report Form (CRF)
• Training needs of monitoring staff, investigators and research teams
• Type and effectiveness of central monitoring approaches available – e.g. if it is a single site trial there will not be the opportunity to compare data across a number of sites

Documentation of the oversight and monitoring strategy

The risk assessment should document the necessary oversight/monitoring actions put in place to mitigate risks and further documentation to implement these would usually be prepared. How this is done is decided by the sponsor. This may be contained in one document, for example the protocol, an SOP, or a monitoring plan which clearly outlines for a particular trial how the trial would be overseen, managed and monitored to ensure compliance with the regulations.

In many organisations, the oversight and monitoring strategy is contained in a range of documents, for example:

• SOPs,
• protocol,
• monitoring plan,
• committee charters,
• data management plan,
• data validation plan,
• medical monitoring plan,
• pharmacovigilance plan etc.

It is recommended that there is clarity on how these link together to implement the oversight and monitoring strategy and an overriding document to explain this would be recommended. Such a document would be particularly useful where there is delegation of functions from the sponsor, for example, to CROs in different countries, as it will facilitate consistency. Where these types of documents/plans are being used extensively for trials within an organisation, there should be formal procedures to control the content, version control and maintenance of them.

It is recommended that where the sponsor is undertaking many trials, there is a procedure in place for the development of the documentation that implements the oversight and monitoring strategy. For example, templates could be developed to ensure all necessary content is addressed for all. Items for consideration for inclusion in documentation could include the following examples (not intended as an exhaustive list):

• There ought to be a clear link to the risk assessment to identify the areas that matter to the results reliability, participant safety and rights and compliance with the legislation
• How sponsor oversight of (Chief) Investigator (Non-commercial), vendors etc will be undertaken
• Type of monitoring to be utilised (central, remote and/or on-site)
• The standards and any SOPs to be followed and any document templates to be used
• Cross-references to other relevant documents within the suite of documentation implementing the oversight and monitoring strategy (e.g. medical monitoring plan, safety plan, data management plan, committee charters etc.)
• Site assessment and initiation procedures
• Training of site staff plus training of sponsor and CRO personnel involved in monitoring, audit, data management etc.
• Frequency of monitoring activities, e.g. site visits, database interrogation reports
• Communication activities with vendors and site(s)
• Monitor site capacity (i.e. maximum number of sites per monitor in relation to resource/skills)
• Where site visits take place, which departments will be visited (i.e. in addition to the investigator, any ancillary departments, such as pharmacy, laboratories, imaging etc.)
• What data will be reviewed at site (% of SDV needed and on what data)
• Data validation specifications
• Data collection and transfer methods
• Computer systems validation/study build validation – e.g. for data transfers, eCRF, IRT use
• Procedures for use and monitoring of central laboratories, specialist services (imaging, ECGs etc.)
• Considerations for unblinded monitors and reviewers (if necessary)
• Expectations for availability of the principal investigator (PI) during monitoring visits
• How non-compliance will be recorded and resolved
• Oversight of the investigator and ancillary department site files
• Escalation process:
  • For central monitoring, what triggers will be utilised for escalating to on-site monitoring visits, or triggers from site visits for audit etc.
  • For both central and on-site monitoring, how unresolved issues or serious non-compliances are handled that the monitor cannot address
• Supplies management and monitoring processes (IMP and ancillary materials, subcontractors, QP certification etc.)
• SAE reporting processes and associated monitoring responsibilities
• AE and SAE review, signal detection, medical monitoring
• Query Management:
  • Data queries from data management activities and found by monitoring at site
  • Protocol queries to medical monitor
  • IMP (including temperature excursion management)
• Documentation and review of monitoring activities:
  • Types of reports (monitoring visit report, central monitoring metrics, statistical monitoring etc.)
  • Format of the reports
  • Responsibility for reviewing the reports
  • Timelines for preparing and reviewing reports
• Role and procedures of adjudication, data monitoring, management and steering committees
• Processes for interim analysis (data gathering, analysis and reporting control)
• Sponsor Trial Master File management
• Regulatory and research ethic committee (REC) approval maintenance (process for amendments, urgent safety measures, serious breach evaluation and reporting)
• Contract and insurance review and maintenance
• Audit plans
• Checks on randomisation processes

The review of the documentation should be undertaken by appropriately qualified individuals within the sponsor organisation. Where there are several documents prepared by different areas, it is recommended that there is a cross-functional review to ensure consistency and compliance with the overall risk-based strategy.

There is no requirement to send additional documentation outlining the oversight and monitoring strategy to the MHRA or the REC as part of the approval process, as there is no requirement for this to be approved by the MHRA or REC, and there are currently no plans for this to be introduced in the future. The protocol must, however, according to the principles of GCP that must be adhered to, contain the monitoring policy though this may not contain the level of detail that other sponsor documentation (plans/SOPs etc.) contains.

For multi-country trials, global documentation for the trial is acceptable, but it may be necessary to include specific procedures to mitigate any country-specific risks that were identified from the risk assessment – for example, differences in clinical practice, local regulations etc. It may be necessary to include some site-specific actions, for example, there may be additional monitoring checks to be undertaken at the chief investigator’s site (e.g. if the sponsor has delegated numerous functions) or the site may be responsible for undertaking a specific trial activity or where a site-specific risk may have been identified.

The monitoring strategy must be followed, together with any procedures described in the protocol (for example setting up of data monitoring committee etc.) and/or standard operating procedures. Any deviations should be documented, justified and any potential impact on the trial quality identified.

There will need to be documented evidence of the activities outlined in the monitoring strategy which would provide evidence to support compliance. Whilst site visit reports are well-established monitoring evidence, and take a similar format across organisations, activity reports from central and statistical monitoring are not established in the same way, so all the documentation and checks undertaken to demonstrate the planned central monitoring activities must be retained. It is of particular importance that documentation shows that any non-compliance issues were effectively dealt with in a timely manner including documentation of any escalation actions.

Adaption of oversight and monitoring strategies

The adaptation of aspects of the conduct of the trial based on a trial categorisation to Type A/B/C and trial specific risk assessment is a broad approach that impacts on all areas of the trial conduct, but beneath this, the activities outlined as the monitoring strategy could include a further flexible or adaptive approach. This ensures that the monitoring tasks undertaken are also risk-based and reflect accumulating information about the compliance of the investigator site(s) or from data/information from outside the trial (e.g. new legislation).

The document(s) comprising the monitoring strategy (e.g. protocol, SOPs, data management plan, medical monitoring plan, safety plan, monitoring plan etc.) are recommended to have been based on the vulnerabilities identified in the risk assessment, but are also recommended to contain further risk based flexibility within. For example,

• the intensity of monitoring/oversight and/or visits is high initially and then decreases when compliance is acceptable,
• high intensity (visits, contact etc) for unknown/new sites with little trial experience compared with sites used many times by the sponsor,
• triggering an on-site visit, if central monitoring reveals concerns at a particular site (e.g. a serious breach has occurred).

There should always be a process and resources made available to escalate the monitoring activities when necessary. Such decisions could be delegated to the monitor to decide the appropriate level and frequency of monitoring based on guidance in the monitoring strategy together with the outcome of monitoring activities and actively overseen by the project manager/chief investigator/trial steering group. The monitoring strategy is recommended to define how adaptations will be decided and the rationale/justification for the change should be documented. In some cases, the monitoring strategy documents may need to be amended if the planned adaptation is beyond the scope of flexibility built into the original strategy.

It is also recommended that the strategy is reviewed when other important documents are amended, for example, the protocol, the risk assessment or the investigator’s brochure or when significant trial events occur (e.g. the addition of extra sites).

The sponsor must ensure that the trial is conducted in accordance with the legislation and principles of GCP. In some trials categorised as Type A, it may be possible that the sponsor oversight can be reduced substantially, but some activity will still be needed; however central or on-site monitoring activities may not be deemed necessary as part of the risk assessment of the trial. This would be very rare and the trial would need to be very low risk for no such monitoring to be justified. The rationale for the oversight and monitoring strategy would be contained in the risk assessment. For example, this may be limited to sponsor oversight of the initiation process and then a detailed questionnaire returned from the investigator as a self-assessment process, perhaps with addition of an audit.

Trials that are very small, investigator-led, and primarily related to scientific investigation of the mode of action or scientific other aspects of a licensed investigational medicinal product (IMP) rather than general clinical efficacy or safety are more likely to lead to a conclusion that no on-site and no extensive central monitoring is necessary when risk assessed. It is recommended that there is a contingency plan to implement monitoring if this was considered necessary by the oversight activities results.

Intensity and focus of the oversight and monitoring activities

It is appropriate for the monitoring to focus or have more detailed review (intensity) on different areas within a trial or in different trials. The following examples illustrate this:

Type A (where the IMP is used as per normal clinical practice) involving vulnerable participants or emergency research:

• 100% source data verification (SDV) of consent may be indicated
• But no recording of IMP accountability is done beyond normal clinical practice resulting in limited or no monitoring of this aspect

In a trial where pharmacokinetic analysis of samples is the primary or an important trial objective

• A detailed review of the processing should be done to ensure that it had been completed in accordance with the trial protocol/procedures. This ensures their integrity compared with routine samples for haematology/biochemistry or for exploratory analysis, particularly if the routine or exploratory samples were taken as per the normal hospital practice.

Unlicensed IMP in a trial categorised as Type C

• Monitoring of the accountability should be carried out in more detail than would be expected for IMP in a Type A trial (where the IMP is used as per normal clinical practice).
• Additionally, if the IMP needed refrigerated or frozen storage conditions this would potentially increase the monitoring focus, though still may not necessitate on-site visits. This could be the case for example if there had been a detailed assessment and approval of the facilities and regular provision of remote storage data to demonstrate compliance.

The Role of Oversight committees in monitoring a clinical trial

In the conduct of a trial, it may be appropriate, if considered as mitigation from the risk assessment, to set up committees as part of the oversight/monitoring strategy. Such committees could be set up on national, regional or global basis dependent on the trial and the remit of the committee.

One type would be a committee that would oversee the trial conduct, i.e. a Trial Management Committee (TMC), which may be expanded to also involve members independent of the trial, to then form a Trial Steering Committee (TSC). These committees would typically oversee that the trial is appropriately managed to ensure compliance and deal with any issues identified during its conduct. One of their functions would be monitoring that appropriate systems and procedures were in place, the trial was adequately resourced, and that progress of the trial was on track. They would be involved in reviewing the monitoring strategy and may make the decisions on any escalation activities that have been identified as being necessary to address non-compliance or any adaptations to the monitoring plan. The TSC may be acting in a more strategic manner than the TMC who would be more operationally focussed. Such committees may not be necessary in the case of a small trial conducted at one site under the direct control of an investigator or organisations may have a project manager and project team that is essentially acting as a TMC.

An independent trial data monitoring committee (DMC) may also be set up primarily to review the safety data for a trial at regular intervals to assess whether the trial is to continue or not or whether amendments need to be made. They may review un-blinded data and also be involved in reviewing efficacy data for sequential trials.

On-site monitoring visits

Investigator site assessment visit

There should be some form of assessment of the trial investigator site undertaken and this together with the decision to proceed with use of the site should be documented, but this does not necessarily mean conducting an actual visit. An assessment could be either a feasibility or information-gathering exercise (e.g. of facilities, resources, staff [CVs], accreditation status for example, as part of a research network, SOPs etc, recommendations from CROs etc) and perhaps via telephone or video-conference to establish the suitability of the site. It may be necessary, however, that after this process, a visit is needed to confirm the suitability for the trial. The investigator site may be well-known to the sponsor and may have been used for similar previous trials. In such situations, there could be a review of existing information about the site, for example previous monitoring or audit reports to supplement a feasibility review to confirm the continuing suitability of the investigator site. The complexity of the trial may affect the need for a site assessment visit, for example, a trial categorised as Type A may be no different to normal clinical practice and thus the need for an assessment visit would be reduced. It is recommended the risk assessment for the trial considers the investigator site selection process.

Investigator site initiation visit

The site research team must be suitably trained to undertake the trial activities. There should be a documented process to demonstrate that the sponsor has ensured that the investigator site research team is suitably trained, set up and ready to conduct the trial. Whilst this is often done with a specific initiation visit and sometimes a meeting of investigators, this may not always be necessary. Specific training could be done centrally via telephone or videoconferencing or using web-based approaches. Training and initiation packages could be sent to the site containing manuals/instructions and checklists to complete and return. The level of training needed would be dependent upon the experience of the site staff and the complexity of the trial. The level of instruction given to the trial site staff at this stage and during the initial stages of conducting the trial is likely to influence the compliance level of the site. Remote training may have risks as it will be reduced in length and may be harder for the trainer to identify any lack of understanding, particularly if it is done without any interaction (e.g. web presentation). The sponsor is recommended to retain evidence of the training given and ideally, evidence of the effectiveness, i.e. the understanding of the site staff, of the training provided. Again, it is recommended that the risk assessment considers the need for site initiation and training. It is also recommended that there is the means to address the training of new staff or re-training of existing staff if the need arises.

On-site monitoring visits

The UK legislation does not require regular on-site visits. The MHRA GCP Inspectorate believes that ICH GCP underestimates the role that central monitoring can take as it states that monitoring without on-site visits should only be in “exceptional circumstances” (although ICH E6 R(2) does revise this to state that the sponsor may choose centralised monitoring only, where justified). If the risk assessment undertaken has not identified the need for on site visits to mitigate vulnerabilities and assure that the trial would be undertaken to the requirements of the legislation and GCP, participants are protected and the results would be reliable; then this would provide suitable justification for an “exceptional circumstance”. The ICH GCP statement that, “in general there is a need for on-site monitoring before, during and after the trial” is not recommended to be interpreted that on-site visits for assessment, initiation and on-going monitoring and closeout must always be performed. Reports from such activities, however, must be retained. A need for on-site visit(s) may arise from information gained during the trial conduct. Where routine on-site visits are not part of the monitoring strategy, it is recommended that provision to undertake a triggered or “for cause” on-site visit is in place if circumstances arise via central monitoring activities that necessitate such an escalation activity to undertake corrective and preventative actions to maintain compliance. If only one planned monitoring visit is to be undertaken per site post initiation, it is recommended that this is done soon after the trial commences so that preventative actions can be implemented at an early stage.

Centralised or remote monitoring techniques may cover many of the tasks that a monitor would undertake at site provided the appropriate and genuine documentation has been provided by the investigator. There are, however, some key benefits of undertaking an on-site visit which include:

• Reviewing patient medical records (e.g. when central remote electronic review is not possible by the sponsor) to conduct SDV, verify the existence of a participant verifying the existence and quality of source documents and detect unreported adverse events
• Meeting, training and motivating the research team and the investigator
• Interviewing staff face to face, which builds rapport and can determine the actual processes used at the site (and any issues associated with that) rather than only conducting a review of the outcome (documentation) that may not present the whole picture
• Reviewing facilities and equipment
• Establishing the role of the investigator in the trial conduct (e.g. monitoring the delegation of duties and the investigator’s involvement and oversight of the trial)
• Providing an overall impression of the quality of the conduct of the trial at the site, which may enable action to be taken to improve quality at an earlier stage
• Reviewing IMP storage areas and performing any necessary accountability checks on the actual IMP (this may only be a sample (initially), if considered appropriate in the risk assessment)
• Reviewing investigators’ site file (Investigator’s TMF) and archive facilities
• Mentoring new staff/investigators
• Witnessing participant visits – e.g. consenting process

If such considerations are identified as vulnerabilities in the trial risk assessment, then this would impact on the decision to undertake on-site monitoring and make it more likely to be needed. For example, for a trial categorised as a Type C, with little safety information in patients, on-site visits to check for adverse events in the notes is more important than in a trial categorised as a Type A where the IMP is well known and used in accordance with standard clinical practice. It should be noted that on-site monitoring does not necessarily need frequent regular visits (e.g. traditionally once every 4-8 weeks), the interval between visits could be set at an appropriate level based on risk assessment, for example, the sites could be visited once, shortly after starting the trial or on a fairly infrequent basis, with a more targeted approach to monitoring activities based on the outcomes seen at previous visits or from central monitoring.

Investigator site close out visit

The close out visit by a monitor would primarily be to ensure all the trial documentation and source data is appropriately filed and stored for future retrieval if necessary, to ensure that all trial materials and IMP is accounted for and returned/destroyed, that all the necessary trial data and documentation have been collected and that there are no outstanding monitoring issues or data queries. Some or all of this could be undertaken remotely, with the investigator site staff undertaking tasks under instructions from the sponsor, for example completing checklists for the contents of the investigator file. If the site has been visited before, then the archiving storage should have been assessed and no further visit needed or alternatively, the archiving could be arranged at an archiving vendor by the sponsor. Again, it is recommended that the risk assessment considers the need for close down visits and how the investigator site file and source data will be available for the required retention time, as this must be available for at least 5 years.

Central monitoring of a clinical trial

Central monitoring activities, focussing on the areas that matter as identified in the monitoring strategy could be undertaken by numerous roles and across various departments, for example clinical operations, study management, data management, statistics, medical monitoring etc. with close co operation where appropriate. It is recommended that where the sponsor is undertaking many trials, some generic central monitoring processes are contained in SOPs, rather than trial specific documents.

Any relevant/important communication or contact relating to the conduct of the trial by the sponsor would be part of monitoring the trial and this oversight, be it telephone calls, emails or letters should be documented and must be retained and this would also demonstrate regular contact with the investigator(s).

There will be many documents received from the investigator sites as per the monitoring procedures, these could include:

• consent/eligibility documents
• status reports (recruitment/withdrawals etc)
• self assessment questionnaires/checklists
• pharmacy records

The sponsor should ensure that processes are in place to ensure blinding of the trial is maintained if documentation that could potentially unblind the trial is to be handled in central monitoring activities, for example records sent from pharmacy. Any generated evidence that receipt and review of such information has taken place must be retained and as such this could prevent the need to retain the actual copies of the documents received. Some non-commercial trials use a form of central monitoring that involves sending in information from the investigator site to a central data centre. It is important that if information that identifies a participant is sent, for example copy of consent form, for monitoring purposes, that the participant has given consent for this and is aware of who will have access to their data and this process is recommended to be included in the protocol. A formal system should be in place at the receiving site to restrict access to confidential participant data with respect to compliance with the GDPR. Compliance any local country-specific requirements will also be required.

Additionally, the investigator site will submit the clinical data CRFs either using eCRF or faxing/mailing/emailing paper CRFs. Any generated evidence of central monitoring activities regarding this data must be retained, these could include reports generated from interrogating the database, for example,

• looking at data submission timeliness
• audit trails of the eCRF to examine times of completion
• data query rates and response timeliness
• SAE reporting rate
• comparisons/reconciliations with other databases (e.g. IRT, Central Labs) and data from other sites.

The investigator must retain the original source document or a certified copy and the sponsor should not have exclusive control of a source document.

The data validation activities, usually done by data management, would be expected to be documented and so must be retained; this would include evidence of manual checks of CRFs and outputs of inconsistencies detected from data validation systems. It is recommended that the data validation activities are recommended to be focussed on the data that is critical to the reliability of the trial results as identified by the risk assessment rather than excessive resource spent on raising data queries whose resolution makes little or no impact on the quality of the trial, the safety of the participants and reliability of the results. This is similar to the approach taken for proportionate source data verification (SDV).

There should be a formal process for dealing with issues and data queries identified during central monitoring and data management activities, including an escalation process. Any generated evidence of identification of the issue(s), review and discussion and subsequent actions must be retained. The monitoring strategy and procedures must be followed and there should be documentary evidence of this.

The use of central monitoring transfers some additional activities to the investigator site and may impact on site resources; however, the investigator and research team must conduct the trial in accordance with the principles of GCP and have some specific responsibilities they must undertake according to the legislation. Some of the additional activities needed for central monitoring may assist in ensuring that this occurs (e.g. completing a checklist for review of investigator site file contents).

Statistical monitoring

Statistical monitoring is an aspect of central monitoring. Sometimes this term can be used in relation to a sequential trial design with respect to analysis for stopping rules, but this is not what is being considered here, which relates to monitoring the conduct of the trial. It is where the accumulating data, either clinical data from the CRF, for example SAE event rates or performance data, for example eCRF completion times obtained from audit trail, data query levels/response timeliness and CRF return times are examined using statistical approaches or modelling across the trial. This allows comparisons of sites to occur and this trending, modelling and process control has the aim of identifying any unusual or extraordinary patterns/variance/distributions within the data and in particular if any sites appear to be “outliers”. Some statistical monitoring can use multivariate methods whereby the data from many variables can be used simultaneously to identify outlying sites. The aim of the methodology is to use the information to decide where to target potential increased monitoring such as on-site visits, telephone calls, training etc and there may be predefined criteria or tolerance limits in the derived parameters which would trigger further monitoring activities or corrective/preventative actions. Outputs from the statistical monitoring usually take the form of graphs.

The use of all the data in this manner can potentially reveal issues that on-site monitoring might not detect, for example potential fraud. Statisticians have traditionally been involved in trial design and data analysis, but an expansion in use of central monitoring is likely to develop new roles in this area for statisticians. It will also increase the necessary interaction between monitoring, data management and others with statistics personnel.

It is understood that many organisations are developing quality metrics or key risk/performance indicators to assist in determining which investigator site to visit, often this is part of the audit function rather than monitoring, but there does not appear to be a list of accepted or validated metrics for sponsors to use. The methodology would include ranking investigator sites with respect to the metrics to identifying outlying sites. Some organisations are also using multivariate statistical methodologies enabling the use of several metrics simultaneously. It is recommended that the methods and metrics to be used are documented in the monitoring procedures. Some sponsors have used the following metrics:

• recruitment rate
• screen failure rates
• CRF submission/completion times against actual patient’s progress in the trial
• query rates
• time to queries resolution vs number of active queries (site level)
• SAEs reported
• numbers of missed or late visits/data
• number of Subject withdrawals/dropouts
• numbers of protocol/GCP non-compliances recorded/reported
• eCRF – audit trail information on completion times in relation to visits or expected timescales

Example of Statistical Monitoring

The examples are not intended to be definitive approaches. They are not endorsed or recommended by the MHRA. They have been reviewed by the MHRA and are provided to illustrate and share methodologies that are being used to support a risk-based monitoring approach.

Example 1: CluePoints’ SMART™ engine, is a statistical software solution that examines if data collected are consistent, and if not, pinpoints those investigator sites that differ substantially from the others involved in a trial. In this regard, it is powerful in detecting outlying sites, but would be limited in detecting a systematic issue affecting all sites in the trial because no site would be an outlier. It is a multivariate approach and analyses are performed on all available data and this generates thousands of p-values as test results (a single p-value being the probability of an observed result arising by chance).

The scoring algorithm summarizes the information from the tests into a summary indicator or Data Inconsistency Score (DIS) that pinpoints atypical sites and data subsets. All the calculated individual p-values are used to process a score per site and rank all of them accordingly. The most significant the DIS for a particular site, the most different statistically the site is. As the chance to identify outliers increases with the number of sites in the study, the SMART™ engine also offers users an adjustable FDR (False Discovery Rate %) that takes multiplicity into account. The FDR sets an upper limit to the probability of detecting a center as an outlier when it is not one. The smaller the FDR, the more confident one can be that a center is truly an outlier. The output is displayed on charts to identify the outlying investigator sites and this can then be investigated further, perhaps triggering audit or site visit.

View an Example Where Centralised Statistical Monitoring detected mis-calibrated thermometers in a clinical trial.

References:

L. Desmet, D. Venet, E. Doffagne, T. Burzykowski, C. Legrand and M. Buyse Signal detection and power in central statistical monitoring of multicenter trials. Statist. Med. 2010; 00: 1-16.

D. Venet, E.Doffagne, T. Burzykowski, F. Beckers, Y. Tellier, E. Genevois-Marlin, U. Becker, V. Bee, V. Wilson, C. Legrand and M. Buyse A statistical approach to central monitoring of data quality in clinical trials. Clinical Trials 2012; 9: 705-713.

Dealing with non-compliance

There should be a formal process to identify, assess and document any non-compliance, i.e. failure to comply with the protocol, GCP or the legislation, which is identified through the monitoring activities. This is to protect the rights and well-being of the trial participants and the integrity of the trial results. Ability to do this is also necessary to comply with the requirements of serious breach reporting, as some non-compliances must to be reported to the MHRA.

Actions could include:

• increasing site monitoring intensity/frequency,
• visits to site by sponsor senior management, conducting an audit,
• holding or terminating recruitment at the site until the non-compliance issue is resolved (such temporary halting of recruitment at a site may constitute an Urgent Safety Measure) e.g. by retraining, additional resources etc.

The corrective and preventative actions to deal with the non-compliance should be documented and they should be followed up to ensure that they are completed as per a formalised escalation process and in a timely manner. In many MHRA GCP inspections it has been found that where non compliance issues have been found at investigator sites by the monitor there has been a failure to address them in a timely manner and to an appropriate resolution. There should be a mechanism to ensure that all the non-compliance is documented such that it can be reviewed as part of the analysis of the data for impact on the trial results and details provided in the clinical study report/publication.

The importance of accuracy of the clinical trial data

It is not the accuracy of the individual trial data that is important, but the reliability and robustness of the trial results. The aim of the management, monitoring and data management activities is recommended to focus on the data and activities that are critical to the reliability of the trial results, for example, the endpoint for the primary objective of the trial or key design aspects (e.g. randomisation). These would be identified during a risk assessment of the trial. It is recommended to aim for a high level of accuracy in these areas identified and potentially accept some degree of error in other areas. Consideration for defining such acceptability in terms of tolerance limits is recommended

Concern may be raised by sponsors about the reliability of the data where no SDV is conducted, for example, if the primary endpoint has not been verified or if there has been no review of patient notes to detect unreported SAEs or if the data submitted in the CRF is found to be incorrect. These concerns arise from the traditional monitoring approach that all the data has to be accurate and that any error is unacceptable. The design of the trial can assist in reducing or mitigating the impact of missing or incorrect data, for example, the results of large blinded, randomised trials with high power are unlikely to be affected by increased variability/omissions of the data, particularly as the errors/omissions would not be differential on a treatment basis (biased).

Small blinded and randomised trials may suffer from reduced power with increased data variability/omissions and there is potential to increase the risk of a false negative result. Open trials are more at risk from bias, as errors and omissions could be potentially differential for the treatment groups. This issue is recommended to be evaluated as part of the risk assessment to determine what level of SDV (and other monitoring checks) is needed to mitigate any concerns about the reliability of the trial results. The monitoring plan may take a conservative approach initially, then reduce the monitoring intensity if the concerns are not realised. The data accuracy and proper conduct of the trial can be influenced not only by the monitor detecting and the investigator correcting errors retrospectively (where possible), but by prevention of such errors in the first place, for example, by appropriate trial design, training, communication and systems that facilitate the conduct of the trial.

Source data verification (SDV)

It is recommended that any SDV is focussed on the data that matters to the reliability of the trial results, for example, focus may be on consent, eligibility, data for outcome measures related to the primary objective, safety reporting, non-compliance, or IMP accountability rather than 100% SDV of all the data. Also, if the monitor is doing 100% SDV then they generally have little time for anything else, and important issues can be missed if the monitoring focus is this narrow. The key data could be identified during the risk assessment and the trial protocol. It is also recommended to define what the source data is at a particular investigator site.

For trials categorised as Type C, due to the exploratory nature of the trial and the uncertainty, it is likely that a higher level of SDV would be needed compared to a trial categorised as Type A, where the IMP is well known and used as per normal clinical practice. This illustrates the risk-based approach. There may be situations where no or very limited SDV is acceptable by the risk assessment. The necessary SDV checks are recommended to be contained in the protocol, SOPs or monitoring strategy documents for the trial. The sponsor may wish to share these planned SDV checks with the investigator, because if the investigator is informed of which data is of importance for the trial results, this may improve the quality of this data. Where there is not 100% SDV of the identified critical data, but a sampling mechanism is used, then there should be some procedure for escalation if the sample verified reveals problems with data quality.

GCP inspectors may perform some SDV at investigator site inspections, but organisations are not recommended to undertake 100% SDV using this as a rationale. The inspectors will always consider the risk assessment (where available) and the plans and procedures which will define the sponsor’s plans for the SDV of the trial data. Inspectors accept that discrepancies may be found if they examine data where SDV is not planned to be undertaken (or the planned SDV had not yet been undertaken at the time of the inspection, for example data collected since the previous monitoring SDV). Also, if the SDV plan is risk-based, focussing on potential discrepancies that would impact on the trial results, then similarly, as the inspectors will use a risk-based approach to selecting data to check, then inspection examination of the data that is not planned to have SDV is unlikely to occur anyway. If discrepancies are found in data that has been documented to have been verified by the monitor, and the selection of this data for SDV by the monitor was risk-based, then this may result in a significant inspection finding as there is obviously a potential to impact on the trial results (which is why it was selected to be verified).

It is likely that trials will continue to be increasingly electronically based, with the potential for data to be copied from source data electronically and transferred to the sponsor. This occurs already for laboratory data and examples have been seen where patient electronic records have been accessed to obtain data. Such an increase in this activity is likely to reduce the need for source data verification with more emphasis on computer system validation to ensure that the systems used function correctly and obtain the correct data.

Finally, some source data can be accessed remotely and verified, for example, there are registries to confirm the date of and reason for death that could be used in long-term survival trials. Access to such information may be restricted to sponsors who are authorised to do this, where participants have explicitly consented to this and where the relevant participant’s personal data (e.g. NHS number) that is needed for such access has been provided and is kept confidential (complying with GDPR), for example, sponsors who are NHS Trusts.