Guidance

Exercising Best Practice Guidance (HTML)

Published 30 September 2024

Introduction

Published in 2022, the Resilience Framework focuses on the foundational building blocks of resilience. The framework uses ‘resilience’ to refer to an ability to withstand or quickly recover from a difficult situation, but also to get ahead of those risks and tackle challenges before they manifest. It sets out the plan to 2030 to strengthen the systems and capabilities that underpin the UK’s resilience to all civil contingencies risks. Planning and preparation for risks cannot be considered reliable until they have been exercised and shown to be workable. This is why exercises play an essential role in strengthening resilience, enabling us to test and validate the structures, plans, procedures and skills required to deliver an effective response when needed^{[footnote 1]}. Importantly, there must also be a system in place to learn from exercises, to evidence and understand areas of good practice, or identify areas for refinement and improvement.

This highlights both the importance of the commitment to a reinvigorated National Exercising Programme (NEP), and of the many and varied exercises carried out within sectors and

local settings, to test preparedness throughout the resilience system^{[footnote 2]}. Exercising is a critical tool for building resilience, enhancing civil protection capabilities, and contributing to wider national strategic goals. The Cabinet Office, through the National Exercising Programme (NEP), directs and coordinates the delivery of coherent exercising activity to meet national resilience objectives.

This first publication of the Exercising Best Practice Guidance (2024) has been jointly developed by the Cabinet Office and its Emergency Planning College (EPC), primarily for all those working in the public sector on civil contingency risks and resilience.

This includes: those with duties and responsibilities under the Civil Contingencies Act (2004) whether national, regional, devolved or local; Government Departments and Arm’s Length Bodies; Local Resilience Forum partners and agencies; and other resilience players such as the Voluntary and Community Sector (VCS). For the purpose of this guidance we define an exercise as a process to train for, assess, practice and improve performance in an organisation.’^{[footnote 3]} This guidance is non-statutory and non-mandatory. It acknowledges that a range of existing exercise guidance is in use across Departments, Devolved Administrations and across the resilience community and it has been designed to complement it all.

To support that process this Exercising Best Practice Guidance has been split into three parts:

Part 1 covers the purpose and principles of exercising. This includes the fundamentals of exercise planning, types of exercises and an overview of the planning cycle. It aims to support all aspects of exercising, from framing activity in the wider context of resilience strategy, through to scoping, planning, designing, delivering and evaluating high quality exercises. It provides a categorisation of the types of exercise to provide coherence and understanding across all stakeholders.

Part 2 details the ‘common to all’ aspects of a comprehensive and effective exercise planning process. This includes an overview of different types of exercises used in resilience building, and helpful templates and examples to support those engaged in exercise work.

Part 3 focuses on the importance of exercise evaluation. It includes applicable methodologies, techniques, and processes that can be used across exercise types, to deliver effective and appropriate evaluation of the exercise cycle. Further guidance and tools for effective lesson capture and identification and management can be found in the Lessons Management Best Practice Guidance 2024.

Combined, these areas will not only inform the quality of our exercising in practice, but also help to ensure that our planning and preparedness for the risks we face are continually reviewed and strengthened.

Preface

Purpose

1. The purpose of the Exercising Best Practice Guidance is to provide a practical guide for individuals and teams who plan, prepare and deliver exercises in a civil contingency resilience setting. It is designed as a ‘hands on’ reference source for practitioners, and is not therefore intended to deliver an academic examination of the subject.

Context

2. This Guidance is written to support the commitment in the Resilience Framework (PDF, 6MB) to reinvigorate the ‘National Exercising Programme (NEP) to test plans, structures and skills’. It aligns with and reinforces the outcome and the good practice statements in ‘National Resilience Standard for Local Resilience Forums No 8: Exercising’.

Scope

3. This Guidance focuses directly on enabling individuals and teams to plan, prepare and deliver exercises across the whole resilience landscape and through the resilience cycle. It directly supports the implementation of the NEP objective to ‘inform the future of high-quality resilience training.’

4. The first part of the guide seeks to establish a shared understanding of the approach to exercising resilience in a national risk context. It provides a baseline for those tasked with or responsible for integrating an exercise or programme of exercises into their organisation’s resilience activity.

5. The second and third parts are aimed at practitioners who are tasked to plan, prepare and deliver specific types of exercising to meet the direction of the NEP, and in accordance with relevant standards (such as Local Resilience Forum (‘LRF’) National Standards).

Other Relevant Documents

6. The Exercising Best Practice Guidance is linked with the Resilience Framework (PDF, 6MB), the Managing Lessons Best Practice Guidance (2024) and the Ministry of Defence Red Teaming Handbook (PDF< 4MB) (2021).

Part One: Purpose and Principles of Exercising

The Purpose of Exercising

Principles of exercising

7. Exercises must be:

• well planned
• realistic
• challenging
• evaluated and used to identify lessons
• supportive of but not replace other forms of training

Exercises must be well planned

8. This is the fundamental, overarching principle. The remaining principles to a greater or lesser degree rely on a sound and well delivered planning process. Whilst many exercises are short and relatively simple to deliver, if they are not meticulously and comprehensively planned there will be a high risk of a failure to meet the agreed objectives and dissatisfaction amongst the training audience.

9. All exercises should be planned using an agreed process. There should be an experienced planning team appointed to plan the event. They must be given clear strategic direction at the outset of the planning cycle. The objectives must be set, and audience(s) identified. From there, developing the scenario and exercise main events will provide the required exercise design, and allow a suitable exercise control plan to be built. The exercise participants, both those being exercised (the training audience) and those delivering and enabling the exercise (exercise control or EXCON), should be properly briefed and where necessary given pre‑exercise training.

Exercising must be realistic

10. The realism of an exercise (both the scenario that is developed and the conditions under which participants operate) ensures practical skills are tested, that participants develop a deeper understanding of the subject matter and are better prepared to tackle challenges in real-world scenarios. It bridges the gap between theoretical knowledge and practical application, making the learning experience more meaningful and valuable. Ensuring realism also means treating the risk of overestimating the likelihood of experiencing positive events and underestimating the likelihood of experiencing negative events, a tendency labelled optimism bias.^{[footnote 4]}

Top Tip

Top Tips for avoiding this inclination may include any of the following: rational thinking with the understanding that every event has a probability of ending negatively or positively; accepting failures and losses and learning from others’ failures. Planners should make explicit, empirically based adjustments to the estimates of any planned activity to be exercised.

Exercises should support but not replace other forms of training

11. Exercising will often be a key element of a training programme both for individuals and for organisations and teams at multi-agency training events. It is important that exercising is not seen as an overall training solution.

12. It would be best practice if participants have attended specific role training and/or pre-briefing sessions prior to taking part in an exercise as this can prevent unnecessary distractions and questions as participants spend time understanding their role rather than undertaking the exercise.

Considerations

• Purpose must be established first. The single most important question that should be asked is: “what is the exercise for?” Discussing the objectives and knowing what the lead hopes to gain by doing the exercise is fundamental to the first meeting. The exercise objective will determine the ‘master question’. This summarises, in a single statement if possible, what problem the exercise is intending to solve. The master question sets the conditions for the activity, from which the method, analysis plan, scenario, facilitation plan and data capture method will derive.
• There should be improvement‑oriented goals. Exercises are held to understand how any proposed action, strategy, or policy might occur in the real world, but lowering the risks of finding out what might happen should it occur for real. Exercising is not about predicting the future in an exact way, but instead enabling the participants to better understand the consequences of their actions in an environment that allows them to make the necessary changes in time for its real-world equivalent.
• Scale, scope and methods vary. There is no one-size-fits-all approach to conducting an exercise, and a large variety of methods have emerged over the lifetime of the profession.
• Enable data capture and analysis. Although regularly developed in tandem, the method should be selected/developed first, followed by the analysis plan, then joint development of the scenario and facilitation plan, and finally the data capture plan.
• Record and implement lessons identified / implemented. Record any lessons identified, and implement improvements or at least track the effort to do so (even if not practicable to implement for whatever reason). If there are no lessons to implement then the exercise objectives should be revisited and consideration given to whether this was the correct exercise to design.
• Establish clear roles and responsibilities. Even though you might be designing an exercise, it is the established lead who owns the event, including the risk by doing an exercise. It is their responsibility to source a venue for the event, create a list of participants (and ensure they turn up) and get the buy-in from stakeholders to ensure delivery. Exercise Planner will seek advice from or work alongside the lead to identify an appropriate venue and a list of participants.
• Not achieving desired outcomes in exercises is okay; and can be essential. The best way of learning how to design better exercises is by doing more of them in a relatively safe-to-fail environment. No-one’s exercise is perfect, and each iteration is a learned opportunity to hone one’s craft, better understand leads’ needs, and experience new methods to apply elsewhere. An exercise that demonstrates a perfect plan suggests you wasted the opportunity to exercise something else that warranted one.
• Diversity is genuinely mission critical. Developing an exercise is a collaborative endeavour that works best with genuine diversity of viewpoints and experiences. The ultimate purpose of exercising is to learn through constructive challenge, and the best exercises are those that use a planning group that benefits from a range of experiences in understanding and responding to problems.

Facilitation plan:

• Facilitation of any scenario is an important skill, which can be developed to build resilience in your exercising team.
• Observing an exercise is a viable technique for learning facilitation skills, particularly if there is an opportunity to gain a copy of the facilitation plan or speak to the facilitator around the event.
• A facilitation plan should break the master question down into its supportive parts, and outline how the time allocated for the exercise will be used to address these points.
• This can form the basis for testing your own facilitation plans, with improving your skills being best achieved by creating simulated laydowns and either reviewing them with more experienced practitioners or finding opportunities to run your own exercises.

Scenario writing:

• Writing a scenario is about communicating a plausible context within which the exercise objectives can be achieved and the master question challenged.
• There is no single approach to writing a scenario, and most experienced authors will have their own technique.
• The scenario is the common core that provides the context within which the exercise occurs, and concludes with a ‘jumping off point’ that marks the transition from the scripted scenario to the start of an exercise.
• The key is to practise and peer review. Scenario writing is easier to practise than facilitation, with historical case studies being an area that authors can use to improve their methods.

Data capture:

• An exercise’s analysis plan needs to understand the connection between the master question and the exercise objectives, and needs to be carefully explained at multiple stages during its development. Exercises require:
  • the collation of action points, with the final report collating the key points of the discussion, the insights regarding gaps or weaknesses in current policy, and references to extant policy that might also be affected.
  • The product refers to pre-existing work and materials, with a focus on the ‘theory of success’ in addition to the insights collected on the topic from the specific exercise.
  • a form of data capture, with the purpose being to improve the quality of the learning experience.

Exercise Planning Fundamentals

13. To be successful, every type of exercise requires a well-defined, robust planning process. This section draws together existing doctrine and good practice from a range of sources and sets out a common best practice standard for resilience professionals.

Top Tip

Planning considerations

14. Every exercise should ensure the following fundamental planning considerations are addressed:

• defined purpose
• agreed aim (or outcome) and objectives
• specific roles and responsibilities
• identified training audience(s)
• realistic and credible scenario
• evaluation
• resourcing

Defined purpose

15. The purpose of the exercise states why it is necessary. It must be clear to every stakeholder why the exercise is being planned and delivered. The purpose of the exercise is different to its aim and objectives.

Example – A national level exercise rehearsing a response to an oil spillage:

‘The purpose of exercise [XXXXX] is to contribute to building national preparedness for major emergencies within the National Exercising Programme’. A well-defined purpose (along with clear objectives) will serve as a good checkpoint throughout the subsequent planning cycle.

Agreed aim (or outcome) and objectives

16. The agreement of objectives should be one of the very first activities undertaken by the exercise director planning team. Any exercise that does not have a clear and agreed set of objectives has a very high risk of failing to meet intended outcomes and should not form part of the exercise.

17. The Exercise Director (see roles and responsibilities at Table 3) will articulate an aim for the exercise. The ‘aim’ should describe the desired outcome of the exercise. This can be condensed into a single sentence, or constructed as a short paragraph (two or three sentences or bullet points). However long it is, the aim must be a clear statement of intent, that can be easily measured after the exercise to determine whether or not it has been met.

18. Objectives are attainable goals which provide the building blocks for exercise design and delivery. The exercise planning team should use the exercise aim to analyse what specific objectives are needed to meet this. The planners can begin to build a list of objectives developed from a variety of sources^{[footnote 5]} to provide the basis for the exercise design. This will determine the events and incidents that the exercise will simulate, in an effective and challenging way.

19. Clearly written and agreed objectives will help to design the exercise scenario and training audience(s).

• Planners are recommended to adopt a SMART approach to objective writing (see below) in order to provide clear and easily understood guidance to all exercise stakeholders. Once the exercise objectives have been drafted, these can be ‘read back’ to the exercise director for amendment (if required) and endorsement.
• The SMART objective is specific, measurable, achievable, relevant and timebound.

20. Table 1: Example: Exercise outcome and objectives

Exercise Outcome

The lead government department’s multi-agency crisis management concept of operations has been rehearsed and evaluated using a reasonable worst-case scenario developed from the National Security Risk Assessment. Potential lessons have been identified for further analysis.

Objective 1

Test the national crisis management cell’s ability to operate at full capacity for an extended period of 48 hours.

Objective 2

Rehearse Department Operation Centre (DOC) staff in national multi-agency crisis working.

Objective 3

Improve understanding of the likely impacts and consequences of a catastrophic environmental accident occurring within the United Kingdom’s borders.

Objective 4

Identify potential lessons for improvements to national crisis management response capabilities.

Specific roles and responsibilities

21. An efficient and effective planning cycle requires a well-defined and empowered organisational structure. Good governance practices will contribute to effective planning and ensure that national multi-agency level cooperation is achieved, when delivering the National Exercising Programme (NEP) across the 3 exercising tiers and exercising outside of the NEP for example, Ministerial tabletop exercise. Tier 1 involves a central response, including Cabinet Office Briefing Rooms (COBR) mechanism, Tier 2 is a departmental exercise and Tier 3 is a regional or local exercise.

Table 2: Possible NEP Tier categorisation and descriptors

Exercise Tier	COBR Unit involvement	Considerations
Tier 1	Cross Whitehall activation – COBR Unit, Situation Cell, Operational Response Cell,	1. Involves central response COBR Unit mechanism. 2. Involves relevant regional tier/devolved administrations and local tier responders. 3. Involves relevant industry engagement such as key businesses, voluntary and community organisations. 4. Key Government Departments will be identified by LGDs and the Cabinet Office. 5. Ministerial play is strongly encouraged. The presumption should be that key departments will fully participate in the exercise at senior official or Ministerial level. 6. One Tier 1 exercise per year may include an aspect of tactical or operational level live play. This may involve the deployment of specific police or military assets, or the engagement of a local health or industry response capability. 7. Involves Scientific Advisory Group for Emergencies (SAGE) if requires emergency scientific or technical advice fed into COBR.
Tier 2 – Departmental Exercises	Likely participation of COBR	1. LGD led response and usually involving key stakeholders (including other departments). 2. May take the form of Tabletop or Live Exercises or elements of both. 3. COBR may have some involvement of OGDs to be included at their discretion.
Tier 3 – Regional or Local Exercise	Possible participation of COBR	1. Usually led by the local government or the emergency services at regional or local level. 2. Key participants may include LRFs/The Association of Chief Police Officers of England, Wales and Northern Ireland Regions/ Health Authorities/ Police Forces/ and other departments and agencies as appropriate. 3. May take the form of Live or Tabletop exercise or elements of both. 4. COBR Unit may have some involvement and OGDs may attend to provide advice.

22.. The following table identifies and describes the key personnel roles and responsibilities, which should be identified and filled at the outset of planning. Please note that the roles below may not all be required for every exercise, and one person may fulfil several of them.

Table 3: Personnel Roles and responsibilities

Role	Responsibilities
Exercise director The director is the person accountable for delivering the exercise in accordance with its stated purpose. The director will not usually be involved in day-to-day planning, but will direct and endorse key aspects (such as approval of the exercise objectives and allocating resources).	- Directs work of the exercise planner and exercise controller. - Chairs the exercise initial planning conference (IPC). - Approves exercise aim and objectives. - Approves the exercise training audience(s). - Signs off the post exercise report including directing any urgent and important changes arising from the exercise.
Exercise planner The planner is the person responsible to the exercise director for the detailed design, planning and preparation of the exercise in accordance with its aim and objectives. The planner will hand over responsibility for delivery of the exercise to the exercise controller.	- Create or embed a governance structure that holds departments/teams to account and takes forward the responsibilities for the lessons identified. - Directs work of the exercise planning team responsible for design and preparation of the exercise. - Drafts exercise aim and objectives for exercise director’s approval. - Prepares a detailed exercise planning programme. - Ensures the exercise is planned in accordance with safety and welfare requirements. - Coordinates the work of specific exercise working groups required for detailed design and preparatory work. - Ensures the exercise is complete and ready to hand over to the exercise controller.
Exercise controller (facilitator) The controller is the person accountable for running the exercise in accordance with the exercise scenario and main events list or other detailed instructions. In some cases (smaller exercises usually) the exercise controller and exercise planner will be the same person. For smaller TTX events the controller will often be referred to as the ‘exercise facilitator’.	- Takes over responsibility for the exercise delivery from the exercise planner - Manages the exercise, ensuring it meets its objectives and runs according to the planned programme - Ensures the exercise is conducted in a safe manner - Directs the exercise control organisation in delivering the exercise in accordance with the main events list and scenario - Liaises with the exercise evaluation team on issues arising during the exercise - Briefs and hosts visitors and/or observers to the exercise - Briefs the exercise director and seeks decisions on issues arising during the exercise
Exercise planning team (EPT) The assembled team of qualified and experienced people who work with and to the exercise planner in designing and preparing the exercise. For many smaller exercises a team will not be required with a single planner calling in specific advice and assistance as required.	- Develop and write the component parts of the exercise under supervision and direction of the exercise planner: - Exercise the scenario - Main events and main injects lists - Evaluation plan - Exercise control structure and organisation - Provide administrative and logistic support to capture enough data in real time for the lessons capture - Liaison with all exercise participants including specialist working groups and preparatory briefings
Exercise control organisation (EXCON) The assembled team of qualified and experienced people who work to the exercise controller in delivering the exercise in accordance with the scenario and main events list or other detailed programme. In some exercises, selected elements of EXCON may also be designated as a secondary training audience and will gain value from delivering the EXCON role.	- Enable exercise play by delivering ‘injects’ in accordance with the exercise main events list or other programme - Record or otherwise capture exercise activity as required by the exercise programme
Exercise evaluator/mentor The evaluator is the person responsible to the exercise director for observing, recording and assessing the exercise (including participants, controllers and scenario outcomes as directed) in accordance with its aim and objectives. The evaluator will brief the exercise director during the exercise and may also provide planned or opportune ‘hot debriefs’ to the exercise participants.	- Agrees the required exercise evaluation plan and programme with the exercise planner - Directs the work of the exercise evaluation team during the exercise - Reviews performance on the exercise against the evaluation criteria and writes reports and assessments for the exercise director as part of the post-exercise report process
Exercise evaluation team The assembled team of qualified and experienced people who act as observers/evaluators for the exercise. They work to the exercise evaluator in assessing the exercise in accordance with the exercise evaluation plan.	- Capture observations and assessments to meet exercise evaluation requirements - Contribute to post exercise debriefings as required - Identify lessons, this can be found in the Lessons Management Guidance
Data Capturer/Observer Capture key learning, issues and identify any lessons arising from the exercise.	- Data capturers support the director and controller by freeing both to concentrate on delivering and directing the exercise - They must be an experienced and knowledgeable individual able to analyse, interpret and precise the key exercise findings with little oversight or guidance
Technical controller Assist both planner and controller in ensuring the exercise ICT and admin support is delivered.	- The technical controller is particularly important where the exercise is delivered partially or wholly on a virtual basis - They will be responsible for setting up, testing and then running the ICT and ensuring contingencies work etc.

Identified audience(s)

23. It is crucial to tailor the exercise to the specific needs and responsibilities of the selected organisations(s) and individuals being exercised. A standing recommendation is to get stakeholders involved early to mitigate risks of constant changes to the exercise plan.

24. Defining the training audience(s) is a crucial step in designing an effective exercise because it helps tailor the content and format to meet the specific needs and goals of all the participants, whilst recognising the main effort of the exercise rests with the primary training audience. The ‘primary audience’ is the organisation(s) or group(s) who are the focus of the exercise and without which the exercise has no purpose. The ‘secondary audience’ is one or more groups, who will take part in exercise. A well planned exercise will identify all relevant stakeholders who could be impacted by the scenario and who might add value to and gain benefit from the exercise. This may include those in cross-cutting functions such as communications teams or scientific and technical advisory bodies. They also are likely to extend beyond the public sector and include industry and academic players.

25. Consider exercise security classification to ensure the exercise content is appropriately aligned to the audience(s) and enables information sharing. Some exercises may have multiple classifications to allow for the needs of different audiences. There are pros and cons to each of these choices; over/under classifying an exercise can either hinder discussions or limit attendance, while accommodating for multiple classifications will require additional resources and careful planning.

Realistic and credible scenario

26. A realistic and credible scenario is crucial for exercises because it creates a simulation of the expected or experienced real-world conditions whilst creating the conditions to achieve the exercise objectives. It allows participants to rehearse, learn, experiment, and improve in a controlled environment.

27. The exercise scenario. There is a balance to be struck between developing the most realistic scenario or writing a credible scenario that best meets the exercise objectives. If the exercise aim is primarily to prepare for a specific risk (such as an aircraft accident at a specific airport) then the planners will need to ensure a realistic scenario, specific to that location, is a key design factor. However, if the exercise objective is to practice one or more organisations’ crisis response procedures to an aircraft crash, the scenario can be adapted to remain credible but better meet the stated exercise objective. Narrow recreation of real life is not the end goal; as long as the scenario is sensible enough that participants do not disagree, it is more important that the right decisions and behaviours are tested.

28. A credible scenario becomes vitally important where the exercise aim or key objective(s) is focused on a particular risk or event where examining the cause, effects and consequences is the primary focus.

29. An exercise scenario will be developed once the aims and objectives of the exercise have been set. It is the exercise planner’s job to use the objectives to drive the credible and realistic scenario, rather than try to fit objectives to a specific scenario. A realistic scenario will ensure that the audience responds to the simulated conditions of the exercise, and will play their part fully for the benefit of all. Exercises where scenarios have not been properly devised (and lack a sense of realism) will quickly lose the attention of the participants, whose actions and responses may not then closely represent how they would behave in a real situation.

Evaluation

30. Evaluation can be defined as: a systematic assessment of exercise objectives, design and delivery. The process should measure against recognised, predetermined criteria that enable the team to determine any discrepancies between intended and actual outcomes.^{[footnote 6] [footnote 7]}

31. The purpose of (and resulting plan for) evaluation of the exercise should be addressed at the outset of the planning phase. There must be clarity in the minds of the exercise planners as to:

• what is being evaluated and why
• who is doing it and how

32. At the outset of planning, the exercise director will identify any external demand for evaluation and ensure the exercise planner is clear on how that will impact exercise design. The requirement for assurance may be a significant driver of the exercise. In such cases that assurance requirement is likely to become an explicit exercise objective. The evaluation methodology and delivery may be defined and delivered by an external third party, requiring liaison and coordination with the exercise planning team. Key planning considerations must be given to how any external evaluation requirements shape and impact exercise design.

33. The requirement for an evaluation of the exercise will derive from one or more of:

• the exercise purpose. If the purpose is to rehearse people and teams in preparation for a specific risk, evaluation may assess the requirement for defined training objectives for individuals and teams delivering a particular capability.
• the opportunity for effective and efficient concurrent activity. The exercise design and planning process may unveil potential for gathering useful, relevant information, observations and data by stakeholders or third parties. For example, confirming whether actions implemented in response to a previously identified lesson have been successful in achieving positive changes or improvements, and that learning can now be evidenced and validated in practice.
• the requirement for continuous improvement. Exercises by their nature create the conditions to observe and capture potential lessons and examples of good practice. A coherent evaluation plan and process must be available to deliver this output ensuring that the write-ups are timely.

Further guidance on evaluation is given in Part 3 and full detail can be found in Lessons Management Best Practice Guidance 2024.

Adequate resourcing

34. How, and to what extent, an exercise should be resourced must be addressed early in the planning process. Adequate resources allow for the creation of an exercise environment that closely simulates real-world conditions. This realism is essential for preparing individuals and teams to handle actual situations effectively. Without sufficient resources, the exercise may fail at no notice, risking reputational damage in addition to time and money wasted.

35. The exercise director must be clear on the level of resourcing available to deliver the exercise and ensure that the exercise planner conducts a comprehensive assessment of what assets and funding will be needed. Resources to be considered are:

• Human resource. Many exercises will need subject matter expertise and/or sufficient planning and support staff to plan, prepare and deliver all aspects of the exercise. This is particularly true for larger exercises or complex ‘stress tests.’

Specific areas might include:

• scenario development including a ‘red teaming’^{[footnote 8]} capability
• engagement and communications
• information, communication and technology (ICT) specialists
• exercise control functions
• evaluation and assessment specialists

• role players (for example media/press, voluntary and community sector players, CNI operators).

• Infrastructure
  • IT hardware, such as functioning projectors, adaptors and dial-in equipment should be tested ahead of the exercise.
  • fully compatible software, including joint working applications and tools
  • physical exercise materials, including printed documents to mitigate IT failures
  • available buildings to re-create command and control locations
  • exercise control locations
• Information and communication technology
  • exercise specific intranet
  • secure communications
  • modelling data

36. Red teaming:

• Many exercises will require dynamic challenges in the form of ‘Red Teaming’, organised challenges to the actions being proposed by the main group. This challenge forces the consideration of ‘what could go wrong’ with the actions being proposed, what aspects of the plan might not be possible and what unintended consequences might manifest.
• Red teaming is a specific skill set that requires nurturing, including understanding of how government systems operate and are prone to failure, what preconditions and concurrencies exist within them.
• A subset of red teaming is ‘Playing Red’, the representation of an active opposition (of any kind) intended to heighten the challenge by resisting the policies and strategies of the main playing group.

Avoiding ambiguity:

• Risks in this category might occur when requirements are articulated late or are vague, are then changed, and/or micromanaged without oversight and continuity of ownership. Working in such an environment remains very difficult and detracts from the potential realisable benefits of exercising.

Senior versus working‑level exercises:

• Working-level exercises are about the details, the direction that the working group has from their seniors that contains the objective they want the exercise to achieve. Exercising with seniors is always more focused – you can incorporate detailed information but only in as much only to enable seniors to make a decision.

Exercise Planning: The Planning Cycle

37. Every exercise requires a competent, comprehensive approach to planning if it is to achieve its desired aim. This section suggests an approach which divides the process into 4 broadly sequential elements.

Plan the exercise

• Understand the aim and objectives
• Develop and refine the plan
• Identify stakeholders/ participants

Prepare the Exercise

• Brief participants
• Run rehearsals

Deliver the Exercise

• Run the exercise
• Capture learning
• Debrief participants

Evaluate the Exercise

• Review objectives
• Identify lessons
• Write post exercise report (PXR)

Plan the exercise

38. The planning phase starts with the exercise director appointing the exercise planner (see paragraph 46) and providing the initial key advice and guidance to get the process underway.

Scoping work

39. The exercise planner together with the assembled planning team will then work on fully understanding the exercise purpose, and then identify the options for the most effective and efficient solution. The key factors or considerations to work through will be:

Key factors of considerations

1. drafting the initial exercise objectives for the exercise director’s approval
2. identify the primary audience for the exercise and potential secondary audience(s) for consideration
3. determine the type of exercise and methodology required to deliver the objectives (for example a TTX or a stress test, as outlined in Annex A)
4. an outline exercise setting and scenario, including any options available for scenario variation
5. prepare an initial work plan for the exercise development to meet any deadline on delivery
6. any key resourcing (including budgetary), logistical or administrative requirements or issues

40. At this stage the exercise planner should provide an initial feedback brief to the exercise director and seek any further decisions required on key fundamental aspects for the exercise.

Initial planning conference

41. The exercise planner should then convene an initial planning conference (IPC). This is the first time that all exercise stakeholders will meet to discuss the exercise. The purpose is to provide all participants with the essential information and guidance they need to plan for and then prepare for the exercise.

42. The meeting will usually be quite short, and information will be high level and in outline. For example, the scenario may be described in the vaguest of terms where an exercise objective may be to rehearse readiness to respond, and it would be unrealistic for the audience to be fully aware of the risk to be played out.

43. There should be clarity on the aims and objectives and the primary training audience.

44. The IPC will also set out the key timings and milestones (including post exercise activity) required to ensure the exercise and its participants are fully prepared by the required start date.

45. Stakeholders with tasks to plan and prepare for the exercise will have an opportunity to ask questions and test any assumptions with the exercise director and the planning team, so that they leave the meeting ready to contribute to develop and refine the exercise plan.

A specimen IPC agenda is shown at Annex B.

Main planning phase

46. The main planning phase is where the exercise planner coordinates the exercise delivery plan with the exercise planning team and the exercise stakeholders.

47. This phase may see the formation of one or more small specialist working groups as part of the main exercise planning team to develop key components of the exercise. In many exercises a small scenario working group may be formed to develop the scenario and the exercise main event list. For a large exercise, it is likely that a small working group will be established to design, recruit, and train the exercise control organisation (which may be very large) and also the exercise evaluation and observation team.

48. This phase will probably require regular short review meetings for all parties to come together and for the exercise planner to assess progress against the plan, and identify any significant issues arising. The likely key areas for review may include:

• exercise scenario development
• main events list with injects
• primary training audience pre-training requirements
• exercise control organisation staffing and pre-training
• exercise evaluation, mentoring, staffing and pre-training
• exercise administration and coordination (for example, participant directory and contact lists/trackers)
• exercise location(s) preparation
• ICT and other technical requirements

49. The exercise planner may also issue a regular exercise update instruction to all stakeholders to include any new, changed or confirmed requirements.

50. Potentially this phase may see the convening of a concentration of all parties in a main planning conference which might last for one or more days, and is designed to efficiently work together to progress exercise planning to near completion. This option may be useful if time is short to deliver the exercise.

51. The shape and effort required in the main planning phase will look very different depending on the type of exercise selected and its scale. The main planning aspects for each type of exercise are discussed in Part 2.

52. This phase is where the exercise planner comes to the fore and their leadership, commitment and drive will be key. The availability of a good, well-practised exercise planning team will greatly improve the chances of the exercise’s success.

53. Once the main planning phase is substantially or fully completed the exercise planner will confirm to the exercise director that the planning cycle can move into the preparation phase, and a final confirmatory conference will be held. The planner may choose to issue a final confirmatory exercise instruction.

Final confirmatory conference

54. This should be held once all the key aspects and elements of the exercise are confirmed and locked in, and major planning activity is complete. The purpose of the meeting is to allow the exercise director to carry out a ‘conditions check’ that the exercise is ready in all respects, and to engage with the key participants and confirm they are fully engaged in the process.

55. It should be timed to be far enough away from the exercise to allow participants to complete the required preparation activity, but close enough that participants will be fully engaged with the required preparation. This will also see the exercise controller fully engaged with the preparation and delivery phases, and start to assume responsibilities from the exercise planner.

An example agenda is shown at Annex C.

Prepare the exercise

56. The preparatory activity is focused on the exercise audience and the key enabling staff required to deliver the exercise. It is about making sure that the exercise players arrive at the start of the exercise at the right time, properly prepared with the right training and equipment and with a good understanding of what will be required of them.

Exercise planners should consider the following preparatory tools/activities:

An initial brief

57. Once the initial planning conference has concluded and the aims and objectives are fixed, a short briefing note should be circulated. Its purpose is to ensure exercise participants can begin their own preparations in good time before the exercise starts. It should contain the following key information:

• a brief description of the type of exercise and its purpose (including date and timings of the exercise, if they are known)
• the exercise aim and objectives
• who will take part in the exercise
• details of the setting and/or an outline of the scenario or risk to be played
• any essential pre-training for exercise players
• date and time of the exercise participants’ briefing (if known)

Rehearsals

58. The exercise controller should consider arranging rehearsals for aspects of the exercise which may be complex or technically demanding. These should be conducted at a point allowing sufficient time for changes to be made and briefed to stakeholders. Examples of where rehearsals can prove useful are:

• use of new and/or multiple communications systems by players and the exercise control organisation
• where an exercise will be run virtually
• where the playing of events and injects requires close coordination by supporting exercise control elements

59. For smaller virtual TTX type events, combining the participants’ briefing with a rehearsal session to test connectivity and allow players to experience using the system, can be an efficient use of time.

60. For larger scale, more complex exercises, consider dedicating a session immediately prior to the start as a set up and rehearsal phase, to allow players to set up as appropriate and test communications.

Deliver the exercise

Exercise control (EXCON)

61. For TTX type events, the EXCON functions can often be filled by the exercise controller facilitating discussion and ensuring timekeeping is adhered to with assistance from one or more technical assistants (for example, to act as the host for a virtual exercise).

62. For larger stress tests, it is likely that a more comprehensive and well-staffed EXCON organisation will be required. This will ensure that the main events list and main injects list is properly managed and exercise play is monitored. When necessary EXCON can intervene to correct or stop any unwanted or unrealistic situations being played out. EXCON should normally be located away from exercise players to avoid interference with exercise play. There can be a balance to consider between over managing events and stifling learning or understanding and careful interventions, to ensure that the exercise progresses and achieves its objectives.

63. For live exercises EXCON must have well‑resourced health, safety, wellbeing, and risk mitigation controls in place. For example, if a LIVEX is taking place in areas where non-player access is allowed, a separate EXCON team will be needed to ensure exercise play and daily non-player activity do not come into unwanted and unintended contact. The exercise controller will be the key individual responsible throughout the exercise.

Exercise Delivery Process

64. Every exercise requires careful management and control from start to finish. Exercise planners should consider what exercise control functions and measures are required as part of the exercise design process.

Before Exercise	During Exercise	After Exercise
Exercise controller convenes all players and EXCON) to conduct a final ‘conditions check’. - all exercise players are present and ready - EXCON organisation is fully staffed and ready - all communications systems are working as required - all control measures for the exercise are in place - the exercise director is content	The period during which exercise players react to the events and injects with which they are presented. - exercise controller facilitates exercise play as required - EXCON delivers the main events list as required - exercise evaluators observe and assess as required - everyone records potential lessons When the exercise is complete (either by completion of the or by attainment of all objectives), the exercise controller (with director’s permission) announces that exercise play is finished.	The period following the exercise which continues until the exercise director agrees that all exercise outputs are completed, or the available time elapses. Immediate follow up (before participants leave): - ‘hot’ debrief(s) - issue exercise observation, lessons capture, and feedback forms - pack up and hand back the venue where appropriate Subsequent Follow up (in days/weeks following the event): - initial potential lessons identified assessment - post exercise report published - post exercise meeting or cold debrief conducted

Evaluate the exercise

66. At the end of the exercise, a short ‘hot debrief’ or after-action review should be conducted by the exercise director and/or the exercise controller. The purpose of the hot debrief is to:

• provide a structured and coherent finish to the exercise
• thanking all stakeholders for their effort and input, and reinforcing the purpose of the exercise
• to provide participants an opportunity to provide any immediate or urgent feedback (positive or negative) they have identified
• to expose and set out any observations or immediate actions that the exercise director deems urgent and/or important
• allow EXCON time to conduct a final check, and confirm they can conclude the exercise activity before participants disperse

More detail and guidance on debriefing is given in Part 3.

67. Nearly all exercises will require a considered and thought through post exercise report to be assembled by the exercise planner and signed off by the exercise director. An example can be found at Annex D. Its purposes are to (not an exhaustive list):

• provide an accurate record of the exercise for all stakeholders (particularly for those who did not attend)
• assess whether exercise objectives were achieved
• set out evaluation results to meet any specified assurance requirements (these may have a restricted reading audience)
• set out the findings, observations and potential lessons identified by the exercise evaluation process

Part Two: Types of Exercises

Tabletop Exercises (TTX)

Introduction

68. Part 2 draws from domestic and international examples of current good practice. This includes recent Cabinet Office work with local responders to develop TTX planning and delivery resources using an ‘exercise in a box’ approach, and resilience exercising doctrine developed by international resilience players. There are several examples of generic scenarios offered by organisations who have developed an ‘exercise in a box’ approach. This includes the National Cyber Security Centre’s online resource^{[footnote 9]} and the Cabinet Office Resilience Directorate’s exercising resource for vulnerable people which can be accessed through Resilience Direct.^{[footnote 10]}

69. A TTX is a structured, collaborative discussion. As the name suggests, the exercising players gather around a table or in a virtual setting to conduct the exercise. Unlike large exercises, TTXs are conducted as facilitated discussions in a controlled, non‑operational environment.

70. Tabletop exercising is commonly used at UK national and local levels by many agencies and represents, for many, the predominant choice of exercise format. This form of exercising offers a balance to some of the barriers to delivery such as cost, complexity, lead-in time, and time to deliver an exercise.

71. Crucially, TTXs can form a key component as ‘shaping’ or ‘foundation’ activity within a wider exercising effort by bringing a number of capability strands and/or agencies together in a planned, progressive programme of preparedness and readiness.

Discussion Exercises

72. Discussion exercises are similar to TTXs, where the participants, usually working in teams, are required to reflect on the organisation’s response to one or more hypothetical scenarios. It is a form of ‘what if..?’ analysis.11

73. It can be used to test the assumptions that are built into [planning the exercise] arrangements. Critical thinking and constructive challenge are key, to avoid the common error of teams becoming too ready to accept their own optimistic rationalisations.11

74. To achieve the desired aim participants may need independent, and expert, facilitation and reporting.^{[footnote 11]}

75. From a baseline assumption that most TTXs should be relatively simple to plan and deliver, the exercise planner might consider, consult on and confirm the following key appointments.

Key roles

76. Many TTXs do not require the extensive or complex resourcing that other exercise types require. A starting point for a TTX is suggested in Table 4. Noting that a virtual or ‘hybrid’ and a face to face mix is often highly cost effective and would require the inclusion of a ‘technical controller’.

77. The exercise planner can add additional roles or tasks if needed (as outlined in Part 1), as the exercise design requirements become clear in the early planning activity. Examples of exercise planning good practice are given at Annex B, Annex C, Annex D and Table 3.

Scenario selection

78. Consider whether the exercise can use an available generic setting and scenario, or adapt an existing scenario(s), or develop a focussed and audience-specific scenario. Developing a realistic and credible scenario is a fundamental planning consideration, but there will often be opportunities to make efficient use of work done by others for similar aims. The balance will be between the time and resources available to develop a more focussed and specific scenario and the perceived, tangible benefit that additional effort will produce.

Main events list

79. TTXs do not require a complex and detailed events list in the same way that a large exercise does. This reflects the fact that a TTX is usually ‘facilitated’ by an individual who typically sets the audience specified problems and tasks for discussion. The scenario is presented, explained, and developed as part of the exercise overall discussion.

80. The main events list helps build a narrative which is delivered by the exercise controller across the course of the exercise, breaking it into manageable phases or ‘turns’, with the script being delivered by the exercise controller at each relevant phase. Additional ‘injects’ to add realism can also be disseminated at each turn. There are usually ‘discussion points/questions’ after each inject. Devising these questions and facilitating the conversation with open end questions is key to making a TTX successful. Examples include Met Office forecasts, strategic or tactical coordinating group minutes, or other information that creates a sense of a moving time frame.

An example of such an approach is given at Annex E for a cyber crisis management TTX.

Audience and exercise working structures

81. The purpose and objectives of the TTX will determine the working structures. This may reflect organisational command structures or cross-boundary relationships.

82. Whilst a seminar approach (i.e. a group of individuals attending a facilitated event) is a common one, other options could be considered. For example, participants may be grouped into two or more ‘teams’ or ‘syndicates’ structured to represent/reflect their specific roles and capabilities and address specific problem sets or ‘focused questions’.

83. Alternatively, syndicates might be similarly structured to cover the broadest span of knowledge, role and functions. They will typically address and answer the same question set to provide diverse and contrasting analysis and options. A default syndicate structure of between five to eight members offers planners a baseline for planning. The table below provides an example of how a ‘syndicate’ approach for a TTX might be grouped for a multi-agency TTX, dealing with a response to an environmental (severe weather) risk:

Table 4: Example of a TTX syndicate approach

No	Role	Organisation/agency	Remarks
1	Syndicate Lead	Cat 1 (LRF lead)	Ideally a key LRF rep who can enable syndicate discussion.
2	Member	Cat 1 Emergency Services	Example
3	Member	Cat 1 Responder
4	Member	Cat 2 Essential Services
5	Member	Cat 2 Essential Services
6	Member	MHCLG RED[footnote 12]
7	Member	Voluntary and Community Sector (VCS)
8	Data Capturer	Any	Role can be filled by any other member other than lead

Planning pipeline

84. The planning ‘pipeline’ requires minimal or ‘light touch’ governance. This guidance does not define time periods for exercise delivery. A combination of coherent and realistic requirement setting and sound analysis of ‘conditions’ at outset should deliver the exercise planning pipeline and timings. A simple, logical process to deliver the required aims will be more effective and efficient than simply assuming any exercise must take ‘X’ months to deliver.

85. An example key activity process pipeline shown in Table 5 and an indicative planning matrix is at Annex F.

Table 5: Example of a key activity process pipeline

Activity /product	Structure/purpose/timings
Initial planning conference	The conference confirms the exercise director’s intent and direction for the event. It identifies key preparatory activity and any changes or adaptation to the exercise structure. It will also provide the essential detail to send a participant’s joining instructions.
Final confirmatory conference	This is a final ‘conditions check’ by the exercise planner that all aspects of exercise preparation are complete (or on track) and confirm this to the exercise director. Assuming all conditions are met, the planner can effectively handover the exercise to the exercise controller, to prepare for delivery.
Participants briefing and rehearsals	Rehearsals are the opportunity for the exercise controller and participants to come together and ensure all attendees are well prepared and informed. The controller will lead the briefing, including giving the exercise setting and starting scenario and providing clarity and re-assurance on the exercise aim and objectives. Participants will have the opportunity to question the controller and exercise planner on any areas of concern. If the exercise is to be run virtually it allows participants to test and rehearse using any functionality to be employed (e.g. chat, polls, breakout rooms).
Post exercise report	A formal document prepared by the planner/controller and signed off by the exercise director. It details the key findings and observations and any other evaluation from the exercise, to inform both individual professional development and organisational improvement.

Stress Test Exercising

Introduction

86. ‘Stress test exercising’ describes a structured, dynamic process within a collegiate ‘safe-to-fail’ environment, which helps explore what works and what does not, typically at relatively low cost. In the context of recent national resilience planning activity, stress testing is a planning tool which can be summarised as: ‘A scenario-based model in which the aims and sequence of events affect, and are affected by, the decisions made by the players.’.

87. Stress tests share many of the features of a TTX and the planning and preparation of both ‘look and feel’ broadly the same, particularly the cost-effective nature and ease of delivery. However, the conduct of a Stress Test differs significantly from the ‘discussion’ or ‘seminar’ based approach that a TTX usually adopts, in that they are dynamic and adversarial.

88. The exercise players are grouped into teams (or ‘cells’), depending on their role. The exercise proceeds through a series of ‘turns’ which are played out as the scenario and player reactions demand.

89. It offers a structured and disciplined methodology for a diverse audience of stakeholders to understand, discuss, challenge, and thereby improve the overall outputs and aims they seek for the resilience problems they face.

90. Some organisations commonly use ‘Wargaming’ to refer to stress test exercising. However, in civil protection exercises where the approach is less militaristic, the term stress testing is used.

Planning the exercise

Select the events/risks to be tested;

91. Ensure that there is very clear direction and guidance on what is to be ‘stress‑tested’. before any further detailed effort and resource is applied and develop specific ‘terms of reference’ (e.g. a description of risk or particular setting and scenario) that frames the exercise. Often setting out what is not to be included will be as helpful as defining what is to be considered.

Top Tip

Be willing to limit the scope and ambition of a game to bring clarity and reduce the time spent and the number of participants involved. If necessary, run further game(s) focussed on other specified objectives.

Identify the audience

Spotlight

92. Stakeholder/audience analysis will help in building the exercise (using a RACI1 approach may help). This is wider than simply selecting the participants. It requires analysis to confirm where responsibility and accountability lies (for the events/risks being exercised), where dependencies exist, and who and/or what represent the key challenge(s) to a successful outcome. Each exercise requires a sponsor (the exercise director) who sets the requirements for the exercise with the audience likely to comprise the following:

93. The scenario presents and makes choices and decisions as a result. The players need to be allocated roles within the exercise to enable adversarial play and keep to time. The number of players required will be determined by the exercise objectives. They are usually organised into cells, the size and shape of which can vary considerably. The norm is to ‘colour code’ cells to differentiate their roles and purpose.

The Red Cell should not be confused with a ‘red team’ which is a term used in relation to ‘red teaming’ activity and is different from exercising. For this reason, the word ‘team’ is best avoided in stress test exercises.

94. The colours suggested below simply represent an option for player organisation. A suggested baseline structure for organising players is as follows:

• Blue cell (‘supported’ players). The blue cell comprises those entities holding primary responsibility or authority to deliver operational effects or policy decisions and who are organised and resourced to perform those functions.
• Green cell (‘supporting’ players). The green cell comprises those organisations or entities who may hold dependencies with or from blue cell players. For example, a green cell player may have response options in support of blue cell for second/third order effects arising from the scenario being played.
• White cell (other factors). The white cell comprises players who will represent audiences (people, groups, organisations, entities, and natural phenomena) who may be affected or impacted by the events played in the scenario, or may in turn impact the scenario themselves and the decisions and choices made by blue/green cell players. However, they will not always play an active role in the exercise.
• Red cell (the risk/hazard/threat). The red cell represents the risks or disruptive events (including human adversaries) which affect and challenge the players and the blue cell particularly. The aim of the red cell is to bring the scenarios to life and ‘stress test’ players’ plans and options through the delivery of contingent, compounding and credible problems which arise from planning assumptions and identified risks.
• Senior judgement panel. This comprises a small number of suitably‑qualified and experienced individuals who act as critical friends to the blue and green cell participants. They should be independent of the process and plans which the blue/ green cells are playing but might have experience of responding to similar challenges. Their role is to consider the gameplay and provide constructive challenge and appropriate observation to inform future planning cycles and events. They can also advise the exercise director/controller on complex or technical aspects of discussion, or, where adjudication on points of contention are required.
• Exercise control (‘excon’). Those who run the game in a collegiate and controlled manner to ensure that it meets its aims and objectives and maximises the value for all participants and observers.
• Exercise director. The exercise director ‘owns’ the stress test and sets the requirements for the detailed planning. At the completion of the stress test the exercise director must endorse all its outputs and authorise the follow up actions.
• Exercise controller. The exercise controller is the critical role during stress test execution. They steer the stress test minute by minute to achieve the objectives, following direction and guidance from the exercise director as required. The role includes, but goes beyond, ‘umpire’; the controller should be the final arbiter of all routine decisions. These decisions might relate to adjudication, scenario evolution or any aspect of the stress test.
• Data capturer. The data capturer provides the freedom for the exercise controller and exercise director to concentrate on ensuring the stress test flows and delivers its objectives. They record, capture and articulate the discussions, arguments, agreements, frictions and decisions made by the players during the stress test. They must have been involved with planning the exercise and fully invested in its objectives, the scenario and format. A complex or large exercise may require more than one data capturer.
• Technical staff. Competent technical staff will be required to ensure that the infrastructure and equipment needed to deliver the exercise is prepared, tested and operated properly and does not distract or divert players and the exercise controller away from their primary functions.
• Observers. Those who are not required to participate directly in the exercise but who have dependencies with participants or other interests in understanding the aims. Observers should be briefed about their role to avoid them attempting to transition into player roles without authorisation.

Write the aims and objectives

95. Taking a little time to articulate the objectives at the outset will help focus the exercise participants on what must be done and what may not be entirely necessary. The aim should explain what is to be achieved and what should be understood at completion of the event(s). Aims as attainable goals can then break down what needs to be done or addressed by separate participants or by subject matter. The objectives for a stress test supporting the 2021 UK hosting of the G7 conference planning are shown below.

Table 6: The aim and objectives for the CCS major events team G7 conference stress test

AIms

The stress test has contributed to the wider cross government and local G7 planning process by ensuring evolving plans are collectively shared and understood before they are confirmed and subsequently rehearsed.

Objective 1

Identify, understand, test and agree the command, control & communications structures and relationships and information flows to enable the G7 CONOPS to be finalised and issued in week commencing XX.

Objective 2

Understand the events synchronisation in terms of time and space, detailing where tasks, authority and risks overlap, where gaps will arise and where decision points exist.

Objective 3

Identify additional key risks (including if they are compounding ones) against a reasonable worst-case scenario (RWCS) where contingency plans should be further rehearsed or refined.

Objective 4

Prepare stakeholders for subsequent events; the rehearsal of Concept (ROC) event on XX May and large exercises on XX May.

Top Tip

Once you have agreed and written the objectives and aims you can send an initial ‘priming’ instruction to the participants.

Choose the methodology

96. The risks/events and the scenario will drive how best the stress test can be structured and played. Often a stress test can be best enabled chronologically by playing a series of turns in a selected period along a timeline, so that the results of events and decisions can be followed through and where necessary by ‘jumping forward in time’ to a future point. This approach might typically be employed to play a ‘reasonable worst-case scenario’ to explore capability limitations and test planning assumptions.

97. Alternatively, a stress test could adopt a ‘thematic variation’. This repeats a scenario under varying conditions where causal factors and impacts will be varied. This may be useful to test policy development or event planning against the assessed ‘best case’, ‘worst case’, or ‘most likely case’.

Instructions

98. Send an initial written ‘priming’ instruction as soon as possible, to raise awareness and enable participants to prepare. The priming instruction will contain essential detail including dates of the exercise(s) and the supporting planning conference(s) and give readers outline details of organisation and any key questions that remain to be resolved or answered.

Top Tip

Do not delay sending a priming instruction in favour of including more detail by ‘holding out’ for a few more days. Get the essential details (date, time of exercise, key objectives and date/time of the first planning meeting) to participants.

99. When the detailed discussion and agreements on the exercise content, attendance and delivery (at one or more planning sessions) have been agreed, send a written confirmatory instruction to all participants and other stakeholders. It must be comprehensive and laid out in a logical sequence, ensuring above all else the vital coordinating detail required to get the right information to the players at the right time before the exercise start time.

Top Tip

Ensure the confirmatory instruction is issued in good time before any rehearsals and/or confirmatory conference takes place.

Planning conferences

100. As a guide, plan for two conferences: the Initial Planning Conference and the Final Confirmatory Conference, but be prepared to be flexible as the Initial Planning Conference may require further engagement. The first should be held immediately following the issue of the initial priming instruction. Its aim is to orientate the key participants on to the exercise objectives, answering any concerns or fundamental questions and gaining support and agreement to participate. The conference should then spark the further effort and activity necessary to ensure the exercise is planned and resourced. It may be common for people to drop out at short notice, and occasionally have unexpected guests turn up to participate on the day.

101. There will be resultant forums/workshops which make up the main planning effort and may require a specific main planning conference with required players to build setting, scenarios and exercise injects following the initial planning conference.

102. The confirmatory instruction should be issued at the earliest opportunity. Shortly before the exercise date all participants should attend the final confirmatory conference. Its aim is to ensure all participants receive all the advice, guidance and information to fully participate in and gain value from the event.

Preparing the exercise

Rehearsals

103. Exercise controllers will consider the requirement for internal and external rehearsals. Internal team rehearsals will allow the enablers to test and refine the delivery and management of the game in slow time, using role play. This is particularly important for exercises being conducted virtually, where conference chat and break out rooms may be used.

104. If the stress test is complex, or involves large numbers of participants, an external rehearsal should be considered. This will be player focused, allowing them to practice and be comfortable with joining the game and their role in it, using all of the conferencing tools and understanding the choreography of each turn.

Top Tip

If possible, run the external rehearsal immediately after the final confirmatory conference to ensure that players are ready and to allow sufficient time for participants to be comfortable with what is required of them.

Supporting Products

105. The exercise subject matter and objectives will dictate what supporting detail and products are needed, and when they should be circulated or exposed to the participants. As a rule, a setting and scenario must be given to the players before the start of the exercise and often will be briefed in detail at the FCC. Other data, information and intelligence, which would be available to players in real life, must be prepared and distributed by appropriate channels. Examples of this might include Met Office forecasts and warnings for the period the exercise is covering or an assessment of a threat or hazard being played in by the red cell.

106. The exercise controller should direct those products that are required to ensure players’ situational awareness and understanding is maintained. Graphical representation is a key enabler. A detailed ‘synchronisation matrix’ can be built as scenario planning proceeds assembled and displayed throughout the exercise to ‘step’ players through the turns. Often, mapping or other representation of geography or other domains where areas of interest lie will help.

Executing the exercise

Conduct

107. The exercise should be structured as a progression through one or more turns. Each turn will be initiated by a scene-setting brief and turns can be played to suit the scenario and /or risks faced. For example, turns may be played according to the adopted methodology:

• Chronologically. Turns will be played sequentially through a period of time. Consequently, ongoing issues/actions/decisions posed in an earlier turn may roll into the next turn. Similarly, satisfactory resolution of an issue may remove it from the agenda; or
• Thematically. Each turn may examine the same period and set of events but with different scenarios based on the identified drivers (from optimistic to pessimistic), stressors and pinch points.

Top Tip

The detailed ‘start-state’ of scenarios and underpinning events or ‘injects’ may not be apparent at the point a priming instruction is issued. Accordingly, the decision on how best to conduct the exercise can be made fairly late on and confirmed in rehearsals.

Example of Turns

108. Each turn will be structured as a series of actions (Fig 2). The exercise controller will guide each turn and ensure that timings are adhered to and that players deal with the scenario as presented and effectively ‘address the question they have been asked rather than the question they would have preferred to have been asked’.

• Action. At the beginning of each turn the red cell players will detail a series of events, occurrences, factors or conditions (the ‘injects’) relevant to the setting and scenario of the exercise.
• Reaction. Following the initial action the blue cell supported as required by the green cell will react, setting out how capability, capacity and contingency responses will mitigate, prevent or otherwise impact or affect the red cell injects.
• Counter action. Once the blue and green cells have reacted, red cell members can challenge the reaction statements. This phase should see valuable discussion and insights being surfaced. This phase should see valuable discussion and insights being surfaced, and commentary on how and when planned capability will be used to respond and mitigate.
• Consolidation. At the end of the turn the controller will invite comments with observations, challenge and any recommendations pertinent to the turn activity or wider event issues. Finally, the exercise director will direct (where required) where and how player decisions from the turn will be carried forward and affect or influence the following or subsequent turns.

Fig 2: Example of conduct of a game turn

Game Director

Game controller Facilitation

1. Action: Red Cell lead SJP context 20% of time
2. Reaction: Blue cell leads Green cell (60% of tome allocated)
3. Counter action: Blue cell, Red cell (60% of tome allocated)
4. Consolidation: RED team/SJP Game Director 20% of time

Exercise ‘game’ play: turn procedure and time allocation

Consolidation

109. Following the final turn, the exercise director will invite each cell to make any final comment and observation. The exercise director will then summarise the key outputs of the turn, including decisions and follow up actions on specific organisations, before calling the end to the stress test.

Evaluation and learning

110. The exercise controller should direct all participants to review and submit their observations within a short period following the game, in accordance with the exercise evaluation plan. The purpose of those observations is to identify themes and potential lessons to follow up. The scope of the observations should be defined by the exercise director and should include observations to validate the exercising itself in the spirit of continuous improvement. The method for submitting observations and capturing potential lessons arising is covered in Part 1.

Top Tip

How do I protect the integrity of the exercise? There may be some real-world constraints around using specific communication systems or methods during an exercise. The exercise planner will risk assess the potential for non-exercise players to be exposed to injects and regard exercise play as real life. For this reason, most exercise injects will be pre‑faced with a prominent ‘warning’ (typically ‘exercise – exercise – exercise’) either as a written header or watermark to accompany any required security classification and/or handling instructions.

Live play exercises

111. This part draws upon the Joint Emergency Services Interoperability Principles (JESIP)^{[footnote 13]} exercise assurance framework which sets out the planning process for a live play multi-agency exercise. However, it does not seek to replicate the detailed guidance within the framework or to repeat information contained earlier in this guide. Rather its intent is to focus on those key aspects of exercise planning, preparation and delivery which may be the difference between success or failure.

Introduction

112. A live play exercise, often shortened to ‘LIVEX’, is a practical activity designed to test individuals and teams in real time, by using techniques, drills and equipment in as close a replication of real events as possible. The intent is to engage participants in a controlled environment that is as close as safely practicable to the expected response to a real incident. Given their realism and challenge for participants, live play exercises are considered to be the highest-level rehearsal of the readiness of responder organisations to manage incidents. However, a LIVEX is often the most costly and time-consuming form of exercise to plan and deliver.

113. LIVEX has a wide application. At the ‘low end’ of complexity and effort, organisations will run regular, small-scale exercises which might be more accurately described as ‘drills’. The key feature of such ‘drills’ is that they will usually have been developed and templated, rehearsed and risk assessed and then delivered (regularly) by a trained and experienced team. Each exercise may have one or two scenario variations or options, but the fundamental aims, objectives and methodology will be the same. Often these exercises have highly specific assessments possibly with pass/fail gradings and they represent pathways to more complex or detailed training.

114. At the ‘high end’ of complexity, a LIVEX may be the culminating point of significant capability development which requires a large, multi-agency audience to be rehearsed and assessed in a highly demanding specific scenario. Large LIVEXs which seek to exercise large numbers over an extended time period are extremely demanding and resource intensive.

Planning the LIVEX

115. Such exercises are the most challenging and demanding to deliver, and considerable time, effort and resources will need to be expended for them to be a success.

116. The planning process needs to be comprehensive and rigorous, and it is recommended that a project management approach be adopted. A business case, possibly presented as a project initiation document, should be considered to outline the risks associated with designing and delivering the exercise, the exercise aim, and expected aims, the identified training audience, any resources/constraints and the governance arrangements including the necessary top management commitment.

117. Key deliverables and milestones will need to be determined and planning progress documented in a project management tool such as a Gantt chart. This project management approach needs to be founded upon effective stakeholder engagement and good communications.

Planning team

118. The careful selection of a competent exercise planning team is crucial. This team needs to have the necessary experience, knowledge and skills set to avoid any adverse outcomes and will need to be representative of those key agencies participating in the exercise. The team will need access to appropriate expertise to meet, for example, any compliance requirements with all relevant health and safety, legislative, regulatory and contractual obligations that might arise from the exercise. The team will need to meet on a frequent basis in a series of well-structured meetings with set agendas to drive forward the planning process.

119. Getting the scenario right is another key factor in a successful live exercise. The outline scenario should be agreed by all key stakeholders at an early stage, so that detailed development of it can begin. ‘Key’ stakeholders will likely include the exercise director, the primary audience (those being exercised in real time on the day) and those responsible for safety and evaluation respectively. It is essential that it is realistic, credible and has enough depth to achieve the set aims and to support the amount of exercise activity that will be needed.

Main events list

120. This captures pre-scripted and phased injects to develop the unfolding scenario and presents players with a series of events and issues which require some form of response. It serves as a guide for the exercise control organisation (EXCON) to manage the progress of the exercise. Flexibility is essential and injects may be added or deleted to allow EXCON to vary the pace and tempo of the exercise in response to how players are managing their tasks.

Preparing the live play exercises

Timing

121. The selection of the best date to run the exercise is vital. This will be dependent upon factors such as the aim and objectives of the exercise, other commitments of participating organisations and availability of staff and physical locations, assets, equipment, or facilities that may be required for the exercise. The time of year will be another consideration; any activity that requires deployments of personnel and equipment outdoors is probably best conducted in the non-winter months to minimise disruption due to adverse weather, unless of course the exercise has a specific objective of testing the response in extreme conditions.

122. Once the exercise date is fixed, it is then important that all organisations are given sufficient notice to keep the date(s) free and to allow them to begin their own preparations in good time. Early booking of live play sites is another key consideration and time will need to be factored in by the planning team to visit these locations, to prepare them for the exercise itself and to restore the sites after the exercise has ended.

Documentation

123. In addition to the Main Events List, the planning team is also responsible for the production of other key documentation to support the delivery of the exercise. This needs to be produced in a timely manner and approved by the exercise director, to ensure that it is comprehensive and supports the objectives of the exercise. The initial warning order will provide notice of the exercise and allow organisations to start their preparations, but it is the main instruction which will provide all the necessary details, including key administrative, logistical and training requirements, for participants to join the exercise.

124. The main instruction will need to include detailed risk assessments for the live play sites and related activity, clearly setting out appropriate risk mitigation measures. Key aspects of the exercise instructions will need to be reinforced by targeted briefings for participants and control staff prior to the start of the exercise and rehearsals may be required.

EXCON

125. EXCON’s function is to manage and control the exercise in a safe manner, using the Main Events List as a guide to drive exercise play. EXCON should be resourced with experienced staff for the role with the right level of authority to monitor play and intervene when necessary to ensure exercise progress and achievement of its objectives. Robust communications to players, safety and evaluation staff will be required.

Top Tip

Carefully select the control staff and consider a rehearsal of EXCON procedures including a test of communications.

Health, safety, and wellbeing

126. The most significant additional consideration in live play exercises, when compared to other exercise types, is safety. For this reason, it is recommended that there is a dedicated safety and wellbeing team, which focuses entirely on monitoring play to ensure safe working practices are being followed and to identify any emerging safety or wellbeing risk. The head of this team will need the authority to intervene and stop exercise play, should activity reach an unacceptable level of safety. There remains a duty on all participants in the exercise to protect the health, safety and wellbeing for themselves and their colleagues and to stop or halt any activity or practice that could cause injury or harm to themselves or another.

Staff welfare

127. Success of a LIVEX heavily depends on how well coordinated the logistics and administration plan is. It is essential to get right important matters such as managing working hours and rest breaks, the provision of refreshments, access to toilets and generally caring for the wellbeing of all participants including role players.

Observations and lessons

128. One of the principles of exercising is that exercises must be evaluated and used to identify potential lessons. To maximise learning from the live play exercise, an evaluation team will first need to capture observations and assessment during the exercise, against set criteria, and then report back to EXCON. This process will require credibility, rigour and accuracy, to ensure that observations are defensible and evidence based.

129. Once the exercise has ended, a carefully coordinated and managed programme of lessons identification needs to take place, including the use of hot and cold debriefs in order to identify lessons and to inform the post exercise report (PXR). The PXR should identify any potential ‘lessons identified’ by the exercise and ensure that they are ‘captured’ and forwarded to the authority responsible for lessons management.^{[footnote 14]} These will then be defined into the lessons management process which can be found in the Lessons Management Guidance Best Practice Guidance 2024.

Top Tip

Identifying potential lessons and associated post exercise activity needs to be conducted with a high level of rigour and objectivity.

Top Tip

Further detail can be found in the Lessons Management Best Practice Guidance 2024.

Part Three: Evaluation

Evaluation

Introduction

130. Evaluation is a fundamental exercise planning consideration. There must be clarity in the planners’ minds as to what is being evaluated and why along with who is doing it and how.

131. The purpose of this section is to build on the guidance in Part 2 and provide detail on applicable methodologies, techniques, and processes for delivering effective and appropriate evaluation for exercising.

132. The Lessons Management Best Practice Guidance 2024 provides further information. Exercising should be an opportunity to identify potential lessons (the implementing of changes to improve outcomes) and examples of good practice. Evaluation of exercises is the discipline which generates those outputs.

133. The extent to which a specific exercise (or programme of exercises) contributes to a lessons management process and/or fully completes its own ‘lesson loop’ for one or more aspects of the event should be determined early in the exercise planning process. As a minimum it is expected that the exercise evaluation process should result in the gathering and collation of observations which may represent potential lessons for further analysis and action. They can be disseminated and shared as appropriate.

Terminology

134. For the purposes of this guidance, and to generally promote better understanding and coherence across a resilience audience, the following terms are defined here and included in the glossary.

Definitions

Evaluation

The systematic assessment and appraisal of something (an exercise) to determine its value, worth, effectiveness or significance.

Evaluation describes a process of examining something to determine its value, worth, quality, importance, or effectiveness. It involves gathering and analysing information or evidence to make judgments or conclusions about the object being evaluated. It often involves setting criteria or standards against which the subject is measured, and then comparing actual performance or outcomes against those criteria to make informed decisions or recommendations.

Assessment

An opinion or a judgement about someone or something that has been thought about very carefully.

Assessment is the process of gathering information, data, or evidence to make judgments or evaluations about the characteristics, qualities, performance, or outcomes of individuals, groups, organisations, or entities. It involves systematically collecting and analysing information using various methods such as tests, quizzes, surveys, interviews, observations, or performance evaluations. They serve various purposes including diagnosing strengths and weaknesses, monitoring progress, making decisions, providing feedback, and improving performance or outcomes.

Validation

The action of checking or proving the validity or accuracy of something.

Validation is the process of determining the accuracy, reliability, and effectiveness of something, such as a theory, method, process, model, or system. It involves gathering evidence or data to confirm that the object being validated achieves its intended purpose or meets predefined criteria or standards. It often involves comparing the results or performance of the object being validated against a reference or benchmark, assessing its consistency and reliability, and ensuring that it produces the desired outcomes under different conditions or circumstances. Validation is essential for ensuring the credibility, integrity, and usefulness of the object being validated.

Assurance

A statement or an assertion intended to inspire confidence or give encouragement. Assurance refers to the confidence or certainty that something will happen or is true. It involves providing a guarantee, promise, or confirmation that a certain outcome, result, or quality will be achieved or maintained.

Overall, assurance is about instilling confidence and trust by providing evidence or assurances that certain expectations will be met or standards will be upheld.

The evaluation process

135. Evaluation is a discipline that requires systematic collection, collation, analysis and dissemination of information to enhance understanding and improve capability. It is part of the overall exercise planning process. As part of the analysis process this guidance is ‘best practice’ that can be applied across organisations, whilst acknowledging that organisations may have their own defined practices. A coherent, well focussed evaluation process will produce sound, evidence-based conclusions, and actionable outcomes.

136. An exercise evaluation process should start at the initial planning conference and continue until the final debriefings have been completed. It should continue as the post- exercise report is issued and possibly beyond if the exercise is contributing to a wider or targeted lessons management process. A 4-stage exercise evaluation model is:

• plan and coordinate the evaluation
• observe the exercise, collecting data and information
• analyse and collate the data and information
• disseminate; contribute to the post-exercise report and share to wider lessons management stakeholders

137. During an exercise it may be useful to have observers (and challengers) from different disciplines (e.g. risk experts, subject matter experts, continuous improvement experts etc.) to give insight from a diverse perspective, which will provide a more comprehensive and holistic view.

138. Conducting a thorough and wide-ranging evaluation is not a minor undertaking. For larger, complex exercises it may require the formation of a dedicated and trained evaluation team to ensure that data and information is collected, and assessment questions answered.

Scope

139. Exercise evaluation has 2 main components.

Assessment of exercise outputs and outcomes

140. The primary component which is largely informed by the exercise objectives and is broadly characterised as ‘making assessments against standards of performance’. This can include, for example:

• measuring or testing individuals’ performance against specified learning (or ‘training’) objectives
• assessing teams or groups performance against specified ‘collective training objectives’ or standards
• challenging and making judgements against the effectiveness of contingency plans and the planning assumptions that underlie them
• gathering data to inform specific requirements for assurance or to answer specific questions (‘validation’) about capability or preparedness

Assessment of exercise effectiveness

141. Analysis and judgements about how well the exercise performed in terms of:

• meeting its purpose
• delivering its objectives
• providing an effective value for money solution

Planning the evaluation

142. Exercise evaluation is an intrinsic part of the exercise planning cycle. For smaller exercises it may require relatively little effort, making use of well tested and templated processes and delivered by ‘in house’ staff members.

143. For larger more complex exercises successful evaluation will be a significant element of the overall exercise planning effort. It may require attendance from external, independent evaluators seeking to deliver several separate lines of enquiry to meet different requirements. Example: a large Tier 1 national exercise may be assessed to validate a new or enhanced element of resilience capability. It will routinely deliver a number of observations to identify potential lessons arising. It might also provide a vehicle to conduct second or third line assurance where critical national infrastructure or vulnerable supply chains are involved.

The criteria for successful exercise evaluation include:

Appointment of an evaluation ‘lead or co-ordinator’

144. The evaluation coordinator should be a member of the exercise planning team and be familiar with or have awareness of:

• the purpose and aims for the exercise
• participating organisation(s) plans, policies and operating procedures
• multi-agency coordination requirements and processes
• information and data collection and analysis tools and techniques

145. For smaller exercises this may simply be a role to be filled by the exercise planner or other planning team member alongside their other tasks and duties. For larger exercises the role might require a dedicated team member, particularly where for example a team of external evaluators will attend the exercise.

146. For the largest, national level events it is likely that an exercise evaluation team or organisation will be created to ‘sit alongside’ (but not necessarily as ‘part of’) the exercise control function. In this situation the evaluation coordinator (as part of the exercise planning team) will fulfil a vital liaison role between those two functions.

Analysis of the exercise objectives

147. The exercise objectives will often be the primary determinant of the evaluation requirements. The analysis will inform which evaluation methodologies, tools and techniques will be appropriate for the exercise and who will be needed to deliver them. This informs the why, who and how of the evaluation process.

Select the appropriate evaluation method(s)

148. Determine how the evaluation will best be conducted. At its simplest, for a smaller exercise a set of assessment questions (possibly including self-assessment by the exercise participants) and a simple scoring/grading may be all that is required.

149. For large, complex exercises a more detailed methodology and ‘tool kit’ may be required. Typically, this may involve a comprehensive analysis tool using observation capture with assessment and grading against specific criteria such as specified training objectives or performance indicators.

150. A detailed discussion and exposition of evaluation methodologies, tools and techniques is beyond the scope of this guidance. However, some commonly used examples are summarised at Annex G.

Identify, train and brief evaluators

151. Evaluators need appropriate expertise. Whoever is conducting evaluation will require some basic awareness and skills and possibly some pre-training. This must be considered in the planning phase.

152. For smaller exercises (where trained external evaluators may not be an option) thought must be applied to a pragmatic balance of how much pre-training is required for in-house colleagues to fulfil the role.

153. There should always be time allocated to brief and prepare evaluators for the specific exercise they participate in. This might include the briefing of the evaluation plan and process to the exercise audience by the evaluators themselves. This is often valuable if only to re-assure participants that the evaluators are there for example to validate plans and test processes and are not there to assess or test individual performance and competence.

Observe and analyse the exercise

154. Evaluation can include input from all aspects of the exercise informed by:

• observations from all exercise participants
• assessments conducted by nominated evaluators
• outputs from exercise debriefs
• outputs from the exercise planning process and planning meetings

Observation capture

155. The evaluation plan will set out who is responsible for, and the methods required to ensure the ‘capture’ the observations, information, and data necessary for analysis and assessment in accordance with the exercise objectives.

156. The use of an agreed capture process is key to the effective gathering of observations. It should employ a simple methodology, using a uniform template (paper or electronic) which both evaluators and exercise players can use.

157. The approach is to encourage the ‘observer’ (whoever they are) to not only make a statement of what they have seen but to also provide any relevant context and supporting information along with their opinions and suggestions for any action or changes that they feel would be relevant and beneficial.

158. The template should use a layout and sequence appropriate to the format of the exercise. For example, it may follow the chronological order of the exercise with relevant subheadings to sign post readers to specific areas of focus such as key learning points or specified training objectives. Alternatively, it could group observations using the exercise objectives as the main headings.

Analysis and assessment

159. Post exercise analysis of some nature is required. At its simplest, it may be a section of the post-exercise report confirming whether the exercise objectives have been met and summarising or signposting detail on any potential lessons identified from the exercise.

160. More detailed post exercise analysis may be required for larger, longer exercises. This may involve effort to draw together a large number of observations captured across the exercise along with in-exercise assessments, data collection and possibly post exercise interviews and plenary sessions. Collectively these will inform a detailed exercise evaluation report which can cover some or all of the following:

• judgement on achieving the exercise objectives
• validate or assess capabilities and/or competence
• confirm attainment of specified (learning or training) objectives
• identify potential lessons arising or examples of good practice for further action

161. The exercise objectives may require detailed and very specific assessments carried out during the exercise to measure performance against specific criteria. Often these types of assessments will relate to specified requirements; for example, to meet assurance requirements.

Disseminate

162. How to disseminate and communicate the evaluation findings must be considered in the planning phase.

Hot debrief

163. In many cases opportunities and options to deliver evaluation findings will occur during as well as at the end of the exercise. A ‘hot debrief’ or ‘after action review’ methodology will be employed to focus on important or urgent findings and capture feedback from the exercise audience. This should be conducted immediately after the exercise. The session will be short and sharp. It will focus on specific objectives or aspects of exercise that have just occurred and it will be interactive, stimulating feedback from the exercise audience. It will be led by a facilitator, evaluator or possibly the exercise controller for smaller exercises.

Top Tip

164. A popular hot debrief methodology uses a grid approach to focus on 5 questions:

• Remember to thank all for participating
• What went well? (Observation) Why did it go well (Discussion)
• What did not go so well? (Observation)
• Why did it not go so well (Discussion)
• What key issues/actions arise? (Conclusion)

Cold debrief

165. The cold debrief is a planned conference or discussion format at a point (usually a few weeks but no more) after the exercise. Internal debriefs (usually held within two weeks) involve the same key players that were involved in the exercise. This debrief session should be used to discuss the strengths, weaknesses, lessons identified, and thank staff for their participation. Participants and other stakeholders will have provided considered feedback to inform the evaluation. The evaluator or evaluation team will have had time to analyse, collate and prepare the presentation of the evaluation findings to the exercise director.

166. The main purpose, and emphasis, of the cold debrief is to present and review the exercise evaluation findings with the stakeholder audience. Dependent on the purpose and objectives it might variously seek to cover areas from the exercise evaluation:

• develop or confirm understanding of any key issues arising from exercise play
• validate or assess capabilities and/or competence
• confirm attainment of specified objectives
• identify potential lessons or examples of good practice
• improve awareness and understanding across a defined wider audience

167. A cold debrief is regarded as good practice, although it may not be viable for every type of exercise, particularly where smaller exercises form part of a planned programme of exercising and a programme cold debrief is planned as a capstone event for the entire programme.

168. A cold debrief should be held within a few weeks after the exercise. The attendance of key participants is essential and the exercise director should, when required, engage to reinforce its importance.

Post-Exercise reporting

169. Post exercise dissemination will often be via the report, as a key section of the main body supported by annexes/appendices as required. This might include observations, findings, treatment options or recommendations based on the information gathered during the exercise.

170. There may also be a need or requirement for more discreet or targeted evaluation reporting separate from the post-exercise report. This may occur where elements of the exercise were at a higher security classification involving a restricted training audience and a similarly classified evaluation is needed.

1. HM Government, Resilience Framework, 2022 p.60 ↩

2. HM Government, Resilience Framework, 2022 p.7 ↩

3. ISO 22398: 2013 Societal Security – Guidelines for exercises ↩

4. A good example of optimism bias is where a lack of reporting about a potential impact to build situational awareness is taken as confirmation that this impact is not happening. ↩

5. Social and Behavioural Science analysis and insights should be included ↩

6. HM Treasury Magenta Book ↩

7. ISO 22398:2013 ↩

8. Red Teaming is different to exercising. Red Teaming describes the independent application of a range of structured, creative and critical thinking techniques to assist the end user make a better-informed decision or produce a more robust product. Red Teaming Handbook (PDF, 4MB) ↩

9. The NCSC leads on cyber exercising across HMG. Cyber incident exercising allows organisations to test the effectiveness of their incident response plans in a safe environment and strengthen their incident management processes. For more information see the [NCSC website) or email exercising@ncsc.gov.uk ↩

10. ResilienceDirect is a UK wide secure multi-agency collaborative working Service available to all emergency responders; enabling a secure environment for shared situational awareness including access as required for individuals and private organisations who come in to support on a particular incident/event. ↩

11. Discussion exercise ↩

12. Or equivalent from devolved governments. ↩

13. JESIP – A multi-agency process with the purpose of enhancing joint ways of working and models across all policies, procedures and processes. Principles for joint working ↩

14. Resilience Lessons Management is dealt with in the accompanying handbook Managing Lessons Best Practice Guidance September 2024 ↩