Guidance

2 - Analysis of Data

Updated 20 December 2022

1. Analysing data

1.1 Level 1 - Awareness - Apprentice/Placement

• Understands various statistical techniques and can explain choice of method of analysis.
• Considers how data being used has been constructed and what impact this might have on the analysis.

1.2 Level 2 – Practitioner: HEO

• Effectively applies appropriate statistical and spreadsheet software to deliver analysis.
• Can determine and apply the appropriate approach from a range of economic and other analytical tools, use judgement where data is incomplete and consider the context.
• Seeks input from colleagues/seniors on appropriate data sources and learns to confidently use these.

1.3 Level 2 – Practitioner: SEO

• Reviews, synthesises, and is able to critique and quality assure the analysis of others to ensure it is of high quality.
• Provides appropriate challenge to methods and results, and continually evaluates the quality of analysis to ensure that the Department is producing robust economic analysis which can stand up to scrutiny.
• Seeks opportunities to review the work of others and to consider its reuse for the benefit of self/team. This may include advising on appropriate data sources.

1.4 Level 3 – Expert: G7

• Analyses and handles data for the purposes of producing evidence to directly advise decision makers as well as the wider team.
• Is proactive and innovative in considering how the data / analysis will be used by non-economists, stakeholders, and Ministers.

1.5 Level 3 – Expert: G6

• Is able to steer and challenge where appropriate the analysis, results and evidence produced to advise decision makers as well as the wider team.

1.6 SCS

• Is able to identify and interrogate the critical assumptions to understand the strengths and weaknesses of the analysis and the robustness of results and the implications for policy. Is pro-active in communicating these to decision makers.

2. Quality Assurance and challenging analysis

2.1 Level 1 - Awareness - Apprentice/Placement

• Is aware of core QA methods of own work and open to peer review and QA from others.

2.2 Level 2 – Practitioner: HEO

• Provides appropriate challenge to methods and results.
• Continually develops and evaluates the quality of analysis to ensure that the department is producing robust economic analysis which can stand up to scrutiny.

2.3 Level 2 – Practitioner: SEO

• Designs Quality Assurance processes and applies them, following departmental best practice , and drawing on sources of external expertise; critically assesses economic analysis and improves it.
• Provides appropriate and proportionate critique and challenge to methods and results, in line with the Aqua book..
• Continually develops and evaluates the quality of analysis to ensure that the department is producing robust economic analysis which can stand up to scrutiny.

2.4 Level 3 – Expert: G7

• Ensures that appropriate quality-assurance procedures (see aqua book) for economic models are in place and contributes to their implementation
• Continually develops and evaluates the quality of analysis to ensure that the department is producing robust economic analysis which can stand up to scrutiny.
• Challenges input data and assumptions and interprets output data for the purposes of providing supporting evidence, in line with the Aqua Book.

2.5 Level 3 – Expert: G6

• Leads quality-assurance processes and determines the right level (e.g., team level, unit level, directorate level, external peer review, etc.).
• Challenges input data and assumptions and interprets output data for the purposes of providing supporting evidence on a wider array of topics, in line with the Aqua Book.

2.6 SCS

• Ensures profession approved government wide standards (recruitment, development, promotion) are applied in a structured way and maintained within a positive culture of transparent assurance in the department.
• Being accountable and signing off on key documents.
• Provides the highest level of QA in line with departmental processes.

3. Advising on data sources

3.1 Level 1 - Awareness - Apprentice/Placement

• Learns about data sources and undertakes training to understand the qualities of datasets

3.2 Level 2 – Practitioner: HEO

• Builds knowledge of data available in own work area and learns the limitations of these datasets

3.3 Level 2 – Practitioner: SEO

• Create and quality assure datasets with appropriate rigour.

3.4 Level 3 – Expert: G7

• Advises colleagues on the most appropriate source, handling and processing of data sets based on best evidence/economic theory and analysis to form estimates about outcomes of likely policies.

3.5 Level 3 – Expert: G6

• Advises team colleagues on the most appropriate source, handling, and processing of data sets based on best evidence/economic theory and identifies when to bring in data experts as necessary.

3.6 SCS

• Influences development of new data sources to maintain or improve analytical capabilities.

4. Using Economic models to inform analysis

4.1 Level 1 - Awareness - Apprentice/Placement

• Acquires knowledge of how models can be used to inform analysis.

4.2 Level 2 – Practitioner: HEO

• Develops ability to run economic models, under supervision.

4.3 Level 2 – Practitioner: SEO

• Develops a good understanding of how models can inform analysis, and of their limitations.
• Able to communicate effectively what the model results show.

4.4 Level 3 – Expert: G7

• Supervises analysis informed by economic models, assuring all parts of the process (Inputs, Technical Operation, Outputs, Analysis, and Communication of results).

4.5 Level 3 – Expert: G6

• Ensures that economic models are quality assured, consistent with the principles outlined in the Aqua Book.
• Can make the case to non-technical stakeholders for how new or existing Economic Modelling / Estimation / Forecasting could help to inform work, policy and departmental objectives.

4.6 SCS

• Is accountable for the governance of models, with structured quality assurance and approval processes.
• Ensures modelling stands up to external scrutiny, engaging across government and beyond.

5. Understanding results, including identifying assumptions, risks and sensitivities

5.1 Level 1 - Awareness - Apprentice/Placement

• Understands what assumptions have been made and why.

5.2 Level 2 – Practitioner: HEO

• Applies appropriate assumptions to analysis and clearly communicates the rationale and the associated risks.
• Is able to interpret analytical outputs.

5.3 Level 2 – Practitioner: SEO

• Can evaluate assumptions made by self and other.
• Understands of impact of assumptions on results.

5.4 Level 3 – Expert: G7

• Identifies disparities, considers options to resolve and recommends. analytical solutions.
• Ensures that assumptions underpinning results are transparently presented with results.
• Identifies key risks and assesses impact on results with sensitivity analysis.

5.5 Level 3 – Expert: G6

• Identifies disparities with wider context and where disparities exist, including with findings from other teams, challenges with staff and provides support to resolve.
• Works with teams to ensure assumptions are clearly and transparently presented with results.
• Works with teams to identify key risks and offer advice and challenge on sensitivities to test them.

5.6 SCS

• Ensures key information is distilled for senior and ministerial audiences without loss of rigour.
• Influences decisions to mitigate key risks, understands sensitivities in advice given.

6. Applying econometric knowledge

6.1 Level 1 - Awareness - Apprentice/Placement

• Skill not normally expected at grade, but initiative and learning are part of the apprenticeship programme.

6.2 Level 2 – Practitioner: HEO

• With support, will be able to identify opportunities for econometric analysis and carry it out depending on previous academic training.
• Will continue to invest in learning within this area.

6.3 Level 2 – Practitioner: SEO

6.4 Generally well placed to identify opportunities for econometric analysis and carry it out depending on previous academic and work training.

6.5 Level 3 – Expert: G7

• Where relevant, applies econometric knowledge to fully understand how model inputs are estimated as well as being aware of issues related to model specification and efficiency and how they impact on estimates.

6.6 Level 3 – Expert: G6

• Where relevant, identified opportunities for the team to apply econometric knowledge, including fully understanding how model inputs are estimated as well as being aware of issues related to model specification and efficiency and how they impact on estimates.

6.7 SCS

• Adopts a wider perspective with regards modelling best practice, enables high quality data and systems to perform best practice econometrics where beneficial.

7. Using technical IT skills

7.1 Level 1 - Awareness - Apprentice/Placement

• Develops ability to use Excel and other software packages, is willing to continuously improve skills.

7.2 Level 2 – Practitioner: HEO

• Is comfortable with applying and developing skills with Excel and other relevant software packages.

7.3 Level 2 – Practitioner: SEO

• Develops and applies advanced skills in using Excel and other relevant software packages.

7.4 Level 3 – Expert: G7

• Ensures self and team have relevant skills and resources to be comfortable using relevant software packages (Excel, and others as used in relevant department e.g., SPSS, STATA, R, SAS, etc.).

7.5 Level 3 – Expert: G6

• Ensures team has relevant skills and resources to be comfortable using relevant software packages (Excel, and others as used in relevant department e.g., SPSS, STATA, R, SAS, etc.).

7.6 SCS

• Ensures economists within departments are adequately resourced with learning software relevant for their roles but also future progression.
• Actively seeks opportunities for collaboration and for GES members to grow and develop their IT, Digital and Analytical Software experience and skill set.