Guidance

Producing post-implementation reviews: principles of best practice

Updated 10 May 2024

1. Introduction

1.1 This document was produced by Department for Business and Trade (DBT), Better Regulation team to assist colleagues across government in completing Post Implementation Reviews (PIRs).

1.2 The Better Regulation team is providing this guidance to assist colleagues across government in completing PIRs and in explaining the interactions between PIRs and the Better Regulation Framework (the Framework). This document is therefore not a collection of binding rules and should be distinguished from the statutory guidance that is discussed under paragraph 1.5. Instead, it provides overarching principles of best practice to be used in conjunction with departments’ own guidance when completing PIRs.

1.3 Additionally, colleagues should refer to the guidelines set out in HM Treasury’s Green Book, Aqua Book and Magenta Book. The Regulatory Policy Committee’s (RPC) Proportionality guidance also provides analytical guidance for conducting PIRs.

1.4 Further guidance on the framework, including information on the process for developing policy for regulatory measures, engaging with the RPC, seeking collective agreement, and laying legislation in Parliament is provided in the Better Regulation framework guidance

1.5 For more information on the legislative requirements for undertaking PIRs, departments should consult sections 28 to 31 of the Small Business, Enterprise and Employment Act 2015 (SBEE Act 2015) and the PIR statutory guidance, which sets out the factors that should be considered in determining whether it is appropriate to include a review provision in secondary legislation.

1.6 Departments should also consider how the evidence collected in the PIR could be used for other purposes, such as for post-legislative scrutiny.

1.7 Your department’s Better Regulation Unit (BRU) and-or central evaluation team should, in the first instance, be approached for advice and information on preparing a PIR, particularly in terms of proportionality and other areas of the PIR process mentioned in this guidance. RPC proportionality guidance can be found under the Proportionality in regulatory submissions guidance on GOV.UK. There is also a PIR template on GOV.UK.

2. Overview of PIRs

What is a PIR?

2.1 A PIR is a process to assess the effectiveness of a measure after it has been implemented and in operation for a period of time. PIRs aim to review regulations at timely intervals to assess their effectiveness, whether they are still necessary and whether they are having the intended effects following implementation and during their operation.

PIRs in scope of the framework

2.2. Departments should ensure that a record is kept of any PIRs undertaken for  any regulatory provisions (as defined in the Framework guidance), including any PIRs of regulatory provisions implemented  through primary and/or secondary legislation.

The PIR rationale and process

2.3. A PIR provides evidence that can inform the various stages of the policy development cycle, known as the ROAMEF cycle (rationale, objective, appraisal, monitoring, evaluation, feedback) as they assess the effectiveness of a regulatory measure after it has been implemented and enable an assessment of whether the actual impacts are similar to those expected when the policy was implemented.

The ROAMEF cycle

Rationale

• why is the government intervening?

• what is the problem that government is trying to solve?

• what does the evidence say about this problem?

Objective

• what would success from the intervention look like?

• what metrics can we use to measure success?

Appraisal

• what are the options for intervening?

• what is the evidence on the likely effectiveness and cost-effectiveness of these options?

Monitoring

Data collection to answer the questions:

• did we do what we said we would do?

• how are our success metrics changing over time?

Evaluation

Research and analysis to answer the questions:

• did the intervention work as expected?

• what was the impact, on who, and why?

• was it cost-effective?

Feedback

• What have we learnt?

• how will we use these results in future?

2.4 This is the expected lifecycle for all policies. More information on the ROAMEF cycle can be found in the HM Treasury Green Book. A key element of the ROAMEF policy cycle is evaluation, so that lessons can be learned, and subsequent legislation be made more effective. It is important however that none of the stages within the ROAMEF cycle are considered in isolation by departments. As such, monitoring and evaluation should inform thinking throughout the ROAMEF cycle - before, during and after implementation - which maximises the potential for learning to inform future policy interventions. Where practicable, a PIR should draw upon evidence generated by such evaluation work, so that duplication of effort is avoided.

When a PIR is required

Duty to include statutory review provisions

2.5 In the UK, it is a legal requirement for secondary legislation that creates or amends Regulatory Provisions to include a statutory review provision where certain conditions are met (see sections 28 to 32 of the SBEE Act 2015).

2.6. Departments should consult the legislative requirements for ministers to include statutory review provisions in secondary legislation which are set out under sections 28 to 32 of the SBEE Act 2015. Under section 28 of the SBEE Act 2015, ministers must either include a review provision or publish a statement that it is not appropriate in the circumstances to do so.  The legislative requirements to include a statutory review provision are set out in the business regulation statutory guidance.

2.7 Where secondary legislation does not include a statutory review, the explanatory memorandum, or other ministerial statement may reference other arrangements, for example, an independent review, commitments for a non-statutory review, monitoring or evaluation.

2.8 The duty to include a review provision requires departments to carry out a review of the relevant legislation and publish a report setting out the conclusions of the review within five years of commencement. This should be repeated at least every five years, or more frequently if appropriate for the legislation.

2.9 Section 30(4) of the SBEE Act 2015 sets out that at a minimum, a report must:

• set out the objectives intended to be achieved by the regulatory provision in question
• assess the extent to which those objectives are achieved
• assess whether those objectives remain appropriate
• if those objectives remain appropriate, assess the extent to which they could be achieved in another way which involves less onerous regulatory provision (for example, consideration of alternatives to regulation)

2.10 Ministers must have regard to the PIR statutory guidance under section 31 of the SBEE Act 2015 when determining whether it is appropriate to include a review provision in secondary legislation. Where a statement is made that it is not appropriate to include a review provision, it should be included in the explanatory memorandum together with a brief explanation of the reasons why this is the case.

2.11 The PIR statutory guidance states that if the Regulatory Provision has an impact below +/- £10 million equivalent annual net direct cost to business (EANDCB) then this should be a strong indication that a statutory review provision may not be appropriate.^{[footnote 1]} It is important to note that this recommendation for deciding whether a statutory review provision may be appropriate is different and should be distinguished from the threshold for whether a PIR requires RPC scrutiny. The review provision threshold ensures that statutory review provisions are focused on measures with significant impacts on business.

2.12 It is important to note that primary legislation is sometimes subjected to post-legislative scrutiny, which is separate to the duty to review secondary legislation. For further information, departments should consult Chapter 40 of the Cabinet Office ‘Guide to making legislation.

2.13 The duty to include a statutory review provision in secondary legislation does not apply to regulations made under the Retained EU Law (Revocation and Reform) Act 2023 (the REUL Act). Departments are expected to follow best practice in line with the Framework, including carrying out proportionate monitoring and evaluation of their REUL Act reforms, in particular prioritising those statutory instruments which make substantive changes or are revoked.

Early consideration of statutory review provisions

2.13 Departments should consider the duty to include statutory review provisions during the pre-implementation stage of regulatory policy making. This is demonstrated under Chapter 3 of this guidance. Departments are strongly encouraged to seek legal advice when determining whether it is appropriate to make provision for review in secondary legislation.

Proportionate monitoring

2.15 Regardless of the presence, or otherwise, of a review provision in secondary legislation, Regulatory Provisions may where appropriate be subject to proportionate monitoring, evaluation and/or non-statutory review.

2.16 It is therefore important that departments ensure that resources assigned to monitoring and evaluation are deployed effectively from the outset to ensure a proportionate approach to evaluation is taken. This is key to ensuring departments are allocating their policy and analytical resources to best achieve ministerial priorities and commitments.

2.17 Departments should refer to the guidelines set out in the HM Treasury’s Green Book, Aqua Book and Magenta Book. The Green Book provides guidance on economic principles that should be applied to both appraisal and evaluation. Whereas the Magenta Book is the principal source of guidance for policy makers and analysts in central government on conducting an evaluation. It includes supplementary guidance specifically aimed at supporting analysts in preparing a PIR and includes advice on how to assess proportionality.

2.18 Departments may also find it useful to refer to the RPC’s case histories guidance for Conducting Regulatory Post Implementation Reviews, which provides analytical guidance for conducting PIRs.

Teams to consult when planning a PIR

2.19 To ensure that there is a proportionate review of the policy context and analytical approaches, planning a PIR is often multi-disciplinary. Departments may wish to consult the following principal groups and professionals, which are typical but not exclusive groups (often their work is collaborative), when completing a PIR:

• legal advisers – departments should liaise with legal advisers at the outset to understand which parts of the legislation the review provision applies to and ensure that these are covered in the review and, that any legal requirements which may mandate the review are complied with
• policy professionals – can provide context for specific regulations and its anticipated impacts (departments should consider logging key information regarding the policy and where appropriate, key contact details in the event policy officials leave)
• social researchers and central departmental evaluation teams - are skilled in evaluation strategies and can help advise on data collection and analysis
• regulators - can provide a practical perspective on the delivery and effectiveness of the regulation
• economists - can advise on how to measure impacts that were previously estimated in the regulatory impact assessment
• other departmental analysts – such as statisticians who may advise on the statistical robustness of data collection designs and evidence

PIR template

2.20 The PIR template provides suggested headings for the findings of a PIR and can be used for both statutory and non-statutory reviews. The use of the template is not mandatory but is considered best practice, particularly as it ensures consistency in the presentation of evaluative findings.

PIR bundling

2.21 Where proportionate and feasible, departments may consider combining the reviews of related regulations to create a ‘PIR bundle.’ Under this approach, departments must make abundantly clear which parts of the PIR relate to which statutory instrument to meet the terms of statutory review provisions. There is no guarantee that bundled PIRs will pass RPC clearance. Departments are strongly encouraged to seek the advice of their BRU and the RPC Secretariat at the earliest opportunity when considering whether to bundle together a group of PIRs.

3. Monitoring and evaluation plans

The importance of monitoring and evaluation plans

3.1 The development of an effective monitoring and evaluation (M&E) plan at the initial stage of designing a regulatory intervention is crucial when the time comes to complete a PIR on a specific policy and/or measure. This is because M&E plans go towards ensuring that:

• evaluation delivers meaningful feedback to those that are designing, implementing, and continually monitoring the effectiveness of policies
• evaluation is of appropriate quality
• monitoring data is collected in the most efficient and cost-effective manner possible

3.2 Departments should briefly outline their M&E plans within the explanatory memorandum which accompanies the legislation, or in the relevant section of the options assessment and/or final regulatory impact assessment, whichever is appropriate.

How an M&E plan should be developed

3.3 M&E plans should be designed to inform policymakers about the effects and impacts of their policies as they are being implemented in ‘real time’ and not merely as ways of measuring the effects that feed into PIRs. For example, there may be developments in new technologies and market structures of which departments should  take account . As such, M&E plans should be viewed as ‘live documents’ which set out intentions and commitments but are adaptable and can be amended as necessary to capture different outcomes or unexpected consequences should they take place.

3.4 M&E plans should specify a timeline for the review to be carried out, which as a minimum, should align with the legislative requirements set out within section 28 of the SBEE Act 2015 (a review must be carried out and a report must be published within 5 years).

3.5 Wherever practicable, work on the PIR should be built into any wider evaluation work for a policy so that all relevant evidence can be considered, and any duplication of effort avoided, particularly if there are several related policy measures. This will be important if the measure is integral to wider departmental policy, in which case there will be a strong case for producing the PIR to coincide with outputs of the wider evaluation. Any relationship between the measure and the wider policy, and the proposed next steps for each, should be explained in the PIR.

3.6 Departments should refer to the options assessment template and the regulatory impact assessment template for further steers on what to consider for M&E plans.

The regulatory policy development process

3.7 The appropriate level of PIR and M&E data collection planning for the 3 stages of the regulatory policy development process is set out below.

Initial policy development stage

3.8 This refers to the initial development of policy proposals, prior to formal or informal consultations. At this stage, policy teams should follow the HMT Green Book policy appraisal processes. Departments are strongly encouraged to develop a M&E plan for each regulatory measure at the very outset of the policy cycle and before the measure is implemented.

3.9 Where regulation is already in place, the production of a PIR may be necessary at this stage for the purposes of proportionate monitoring and evaluation.

Pre-implementation stage

3.10 During the pre-implementation stage, policy teams should consider the duty to include statutory review provisions in legislation.

3.11 Once the regulatory impact assessment has been finalised under either the previous or reformed Framework, thorough planning of the PIR should begin. The PIR strategy, data collection plans, and budgets should be finalised and the systems for collecting post-implementation monitoring data should be in place.

Reformed framework

3.12 The pre-implementation stage refers to the stage where a Regulatory Provision has been identified as a preferred option and will need to be logged with the RPC Secretariat, and consideration given to the work needed to comply with the Framework.

3.13 This will include the need to consider whether an options assessment and/or regulatory impact assessment is needed and whether they require RPC scrutiny. An initial M&E plan for a proposed regulatory provision should be produced as part of the options assessment. As part of the initial M&E plan, departments should consider when they intend to carry out a PIR of the policy, although it is expected that a department’s M&E plan will not be fully developed at this point as departments are looking to develop their preferred policy options.

3.14 The initial M&E plan should be proportionate to the level of evidence that is available at this early stage. As a guide, they should closely mirror the expectations for plans produced as part of a consultation stage impact assessment. This includes a decision on the scale of evaluation that might be required, the estimated timeline for carrying out a PIR, the likely scale of data requirements for evaluation, and an initial estimate of budgets and resources required. The M&E plan should then be refined as part of the process of developing the final regulatory impact assessment.

Previous framework

3.15 The pre-implementation stage refers to the stage following initial consultation and before the laying of legislation, where consideration should be given to whether a Regulatory Impact Assessment is needed and requires RPC scrutiny. An M&E plan will also be required to be produced as part of the regulatory impact assessment.

Post Implementation stage

3.16 Once the proposals have come into force, this stage provides the opportunity to review whether the regulation has met the intended objectives of the legislation and whether the policy could be improved. Policy teams may need to conduct a PIR and seek RPC scrutiny of a PIR.

3.17 Once the proposals have come into force, data that has been monitored should be collected and the PIR strategy and data collection plans should be implemented.

RPC scrutiny of PIRs

3.18 A PIR should be scrutinised by the RPC if the ‘original Regulatory Provision required scrutiny’. The scale of the impact in question is at the point that the measure was developed/ first introduced:

• under the previous framework, ‘if the original Regulatory Provision required scrutiny’ is referring to independent scrutiny of the final regulatory impact assessment stage
• under the reformed framework, ‘if the original regulatory provision required scrutiny’ is referring to independent scrutiny at any prior stage, such as at options assessment stage or final regulatory impact assessment stage

3.19 Where there will usually be no RPC scrutiny, departments may wish to voluntarily submit a PIR for scrutiny. If so, they should discuss the merits of a submission with both their departmental BRU and the RPC Secretariat.

4. Considering impacts in the PIR

4.1 As set out in the principles of cost benefit analysis within HMT’s Green Book, the role of appraisal and evaluation is to provide objective analysis that supports decision making.

4.2. Evaluation is the systematic assessment of an intervention’s design, implementation and outcomes and tests how far an intervention is working and whether there were significant unexpected consequences.

4.3 The scope and proportionality of the PIR should take into consideration the expected impacts as set out in the regulatory impact assessment, although evidence should not be restricted to these expected impacts of the measure but should also be capable of capturing unintended effects and consequences.

Determining whether the policy has been successful in delivering objectives

4.5 When determining whether the policy has been successful in delivering objectives, departments should consider the following list of questions:

• to what extent is the existing regulation working and has the policy achieved its objectives?
• is government intervention and the regulation itself still required?
• is the existing form of government regulation still the most appropriate approach?
• if this regulation is still required, what refinement could be made, if any?
• if this regulation is not required, but government intervention in some form is, what other regulation or alternatives to regulation would be appropriate?

4.6 In order to answer whether a policy has met or delivered upon its objectives, departments should draw upon metrics and evidence gathered through their monitoring activities. This monitoring work should reflect the objective identification that departments should have undertaken as part of the original regulatory impact assessment development.

Non-exhaustive list of impacts to consider in the PIR

4.6 Whilst not the primary focus of a PIR, it is important that each PIR should look to assess what a given measure’s actual effects have been and where possible, comparing these impacts (including key underlying assumptions) with those discussed (or perhaps excluded from) the original regulatory impact assessment.

Direct and indirect impacts^{[footnote 2]}

Both are of potential concern for the PIR. For smaller interventions however, when conducting the initial proportionate analysis, it may not have been judged to be proportionate to attempt to assess indirect impacts. The PIR should therefore consider significant wider impacts, even if not included in the initial proportionate analysis.

Small and micro businesses

Given that the burden of regulation can fall disproportionately on small and micro businesses, the impacts on them should be specifically considered by departments. In the international context, consideration of these impacts on small and micro-sized businesses is required where PIRs are appropriate under the UK-EU Trade Co-operation Agreement.

Assimilated law

Where assimilated law contains a statutory PIR commitment, the legal requirement to undertake a PIR will continue. This will also apply to non-statutory commitments. Departments should contact their departmental BRU in the first instance for further advice.

Unintended effects

It is important for the PIR to both capture the size of the unintended effect, and assess the reasons why they have occurred, insofar as possible. Examples of unintended effects and consequences include:

• unusual costs borne by the policy target which are not part of the intended policy, such as unexpected administrative costs, or policy costs very different to what had been expected
• indirect effects (also known as second order, or ‘knock-on’ effects) which were not anticipated in the original policy design. This could be impacts on other groups (such as businesses, sectors, people, parts of society) which were not the intended policy target
• externalities which are unexpected or unaccounted for, such as air or marine pollution, degradation of the natural environment, greenhouse gas emissions
• innovation impacts - the PIR should include a proportionate assessment of whether the measure has enabled or restricted innovation. For example, if the measure has facilitated the creation of new processes or products or has locked-out innovation
• impacts on competition - for measures that affect a market where products and services are provided by private/ public sector organisations, a PIR should consider whether there is an impact on competition
• UK business impacts - business impacts that occur within the UK economy should be assessed as part of a PIR, as reflected in HMT’s Green Book guidance
• trade and investment impacts - for regulatory provisions with impacts on international trade and investment, departments should consider if the provision is compliant with the UK’s international obligations, evaluate the broader impacts to the UK’s regulatory environment, and impacts on the UK’s trade partners if diverging from existing regulatory approaches. If trade impacts exist, there may be World Trade Organization (WTO), free trade agreement (FTA) or broader international obligations applicable

Some key considerations may include:

• an assessment of whether the measure has inhibited or created additional barriers to trade for businesses that are exporting relevant products or services
• whether it has introduced different requirements for businesses in the UK compared to those from foreign countries
• how the measure is consistent with relevant international standards - this includes:
  • where possible, the UK’s international obligations for equal treatment
  • consideration should also be given to arrangements with Northern Ireland, where applicable

Departments may also refer to relevant guidance on trading under WTO rules.

Commitments under FTAs

Following EU Exit, the UK has entered into FTAs with a number of other countries and is negotiating Good Regulatory Practice and Regulatory Cooperation chapters (GRPRC) in other FTAs.

Agreed chapters include:

• the UK-EU and EAEC Trade and Co-operation Agreement
• the EEA EFTA-UK agreement
• the UK-Japan Comprehensive Economic Partnership
• the UK-Australia Free Trade Agreement
• the UK-New Zealand Free Trade Agreement

The UK has also joined the Comprehensive and Progressive Agreement for Trans-Pacific Partnership (CPTPP). Some of the FTAs have provisions on PIRs.

International considerations

The PIR should review whether the measure is consistent with other approaches internationally which seek to address the same policy issue. The International Regulatory Cooperation (IRC) toolkit can help when considering such impacts, which can include, for example, international instruments, standards, and the approaches of other countries. The SBEE Act 2015 (s30(3) requires a review of a regulatory provision that implements an international obligation of the UK to have regard to how the obligation is implemented in other countries (so far as is reasonable). The review should consider and analyse the costs (and benefits) of diverging from international practice and identify any unintended divergences in design and enforcement.

Assessment of compliance and enforcement

It is standard practice to assume at the regulatory impact assessment stage that there will be 100% compliance with the measures unless there is evidence to the contrary. At the PIR stage, it is important to test this assumption empirically where it is feasible and proportionate to do so. The effectiveness of any enforcement regime should be assessed against the Hampton Principles and the regulators’ code.

4.7 Under the reformed framework, there should also be a consideration of:

Household impacts

Household impacts (or impacts on individuals, employees, consumers, where appropriate to the policy) should be considered as part of a PIR. This should include costs or benefits which directly affect people’s budgets and could be related to wages, or other compliance costs faced by households such as the costs of obtaining documentation and information.

Environmental impacts

The potential effects on natural capital and Net Zero should be considered as part of a PIR. For potential effects on natural capital, the Green Book A1 provides the process to follow. More detailed guidance is available in the Defra guidance on Enabling a Natural Capital Approach. The Green Book provides guidance on valuing changes of energy usage and greenhouse gas emissions. A whole system approach should be taken when assessing the impact on emissions, including where there could be potential interactions with consumer or producer behaviour, or interdependencies with other government bodies. Departments should also have due regard to the environmental principles policy statement (EPPS) as part of their PIR, as one of many factors which they considered when making the policy and whether anything has changed since then.

4.8. Departments should consider the timing of the expected impacts of the regulation, and how reviews can be factored in to match these. Where feasible, co-ordinating the reviews of related regulations should be considered.

4.9 Departments could develop their own internal checklists to support meaningful peer reviews and aid policy officials and analysts in deciding that an appropriate and proportional PIR has been completed.

5. Drafting the PIR

5.1 For transparency and consistency of reporting, a summary of the findings, conclusions and recommendations of a review should be set out in the recommended PIR template. Ultimately It is up to departments to decide on the best way to present their findings. However, it is recommended as a minimum that all PIRs include a summary of the methodology and a critical review of the coverage and quality of the data and evidence that has informed the review, including the approach to economic, impact and process evaluation where applicable.

5.2. The process of finalising and publishing a PIR should follow the 5 steps outlined below:

Diagram 2 – steps for finalising and publishing a PIR

Step 1: PIR internal approval process

• Departments may find it useful to peer review their PIR. BRUs should be asked to provide further advice on the internal approval process in the first instance.
• The PIR should be signed off by the department’s Chief Economist and relevant Minister, or in line with the department’s own internal process.

• the PIR should be signed off by the department’s Chief Economist and relevant Minister, or in line with the department’s own internal process

Step 2: formal quality assessment by the RPC

• A PIR should be scrutinised by the RPC if the original Regulatory Provision required scrutiny.
• The RPC will undertake a review of the quality of the evidence base and analytical elements of the PIR. A “fit for purpose opinion” from the RPC should be obtained before moving to step 3.

Step 3: collective agreement

Government publications, including a PIR setting out the conclusions of a statutory review, may require collective agreement from Cabinet, or one of its committees. For further information and advice on clearance, please consult your BRU or speak to your departmental desk officer in the Cabinet Secretariat of the Cabinet Office.

Step 4: Publishing the PIR

• An opinion from the RPC on the PIR must be obtained before seeking clearance to publish a review for any measure/s of which independent scrutiny was required.
• For statutory PIRs, the PIR must be published by the deadline stated in the review provision of the legislation.
• The PIR should be published on legislation.gov.uk, linked to the original legislation, under the ‘more resources’ tab to which it relates. Policy teams should look at the PIR submission guide for further instructions on the process for submitting a PIR for publication on legislation.gov.uk. Policy teams should approach departmental BRUs for any further questions on publishing the PIR.
• Departments should notify the RPC of the publication of their reviews, especially where an opinion on a review was required.

Step 5: use and dissemination of the PIR findings

• After the publication of the PIR, policy teams should seek to undertake any recommendations made in the review.

• Chapter 6 of HMT’s Magenta Book covers the use and dissemination of evaluation findings.

• As outlined in the ROAMEF cycle, the findings of the review should be used as feedback and a reference point to inform new policies and ensure lessons learnt are applied.
• Another PIR will be required within 5 years if the statutory obligation has not been removed.

1. Important to flag that this +/- £10 million EANDCB figure included in the statutory guidance regarding the inclusion of statutory review provisions only applies to novel regulatory provisions. Where the legislation has already been given effect, departments will be required to comply with the existing legal requirements for that legislation. ↩

2. For further information on what constitutes a direct as opposed to an indirect impact, see the case histories guidance published by the RPC Secretariat ↩