Research and analysis

Evaluation report on Grip and bespoke-funded hot spot policing

Published 15 February 2024

Applies to England and Wales

Authors: Olivia Jeffery, Jessamy Bloom, Timothy Eales, Nick Morgan, Robert Patman, William Gould, Rajeevan Balachandran, Peppa Pancheva

Executive summary

Background

In April 2021, the Home Office (HO) announced that 18 police forces with the highest levels of serious violence (SV) would receive funding to deliver enhanced hot spot policing. The aim of this programme, called Grip, is to deter SV through visible patrol activity in hot spots whilst also adopting strategic problem-oriented policing to address the root causes of violence within those locations. In September 2021, 2 further police forces were awarded bespoke funding to conduct hot spot policing in the same way as the Grip-funded forces, as they had the next highest volumes of SV. As far as could be determined, this makes it one of the largest attempts to implement and measure hot spot policing ever.

The same 20 forces now have a multi-year Grant Agreement (years ending 31 March 2023 to 2024) to deliver the hot spot policing programme, making this the first in a possible series of evaluations capturing the effects. This report covers the year ending 31 March 2022 only.

Aims and methodology

This report details the HO evaluation of the Grip and bespoke hot spot policing funds for the financial year ending March 2022. The aims of the evaluation were to:

• evaluate the effect of treatment on the crime rate within a hot spot area, where treatment is defined as a visible additional scheduled officer patrol within the designated area
• provide an assessment of how much hot spot policing is being delivered in England and Wales and how it is being delivered
• assess the cost effectiveness of Grip and bespoke-funded hot spot policing
• provide forces with regular feedback to allow continuous improvements to be made to their hot spot policing approach
• expand the evidence base on hot spot policing, particularly in the UK

No two police forces’ approach to hot spot policing were identical. Therefore, this report effectively comprises 19 different evaluations ^{[footnote 1]}. These were all brought together in a meta-analysis to assess the overall national effect. The meta-analysis includes 19 model results from 16 forces:

• 15 analysed by the HO, using a repeat crossover design which compared crime rates in hot spots on days with patrols and days without patrols
• 4 force-conducted evaluation results, using various methods including a similar crossover design, as well as traditional randomised control trials (RCTs) using intervention and control areas

Key findings

At least 120,000 patrols were carried out by Grip forces and forces receiving bespoke hot spot policing funding in the year ending March 2022 and over 80,000 weapons were collected.

The overall estimate is that Grip resulted in an average -7.0% (confidence interval (CI): -2.0% to -10.7%) reduction in violence against the person and robbery offences in hot spot areas on days patrolled relative to days not patrolled. This statistically significant effect equates to an estimated 1,100 violence against the person and robbery crimes prevented (CI: 400 to 1,900), and £36 million in societal benefits, indicating a return on investment of about £2.20 (CI: £0.79 to £3.77) for every £1 spent.

This does not include some areas and activity for which effects could not yet be calculated, nor does it include impact on any other crimes. The true benefit may therefore be larger, although displacement of crime and diffusion of benefit have also not been assessed. It is hoped that the HO can measure these in subsequent years.

Though the intervention was cost effective, results were not as impressive as suggested by the literature on hot spot policing ^{[footnote 2]}. At force level, few forces demonstrated a statistically significant reduction in crime. From the 19 force-level results, only 2 or 3 (depending on the model used) were significant at the 5% level. There may be many reasons for this. Two that seem most likely are:

1. Most of the results in the literature come from small-scale pilots, whereas this was an almost national-level implementation ^{[footnote 3]}. There is a general tendency for pilots to show greater impact when compared to roll-out at scale.
2. Lack of statistical power linked to lower crime levels in England and Wales relative to countries like the US where much of the evidence originated.

Recommendations

Alongside the analysis, the HO conducted regular conversations with forces to support and improve the implementation of their hot spot policing approach. These have been instrumental in overcoming challenges, and improving the programme during its development, to make best use of funds and assess impact effectively.

Key recommendations for the police forces include:

• whilst hot spot policing is simple in concept, it is not simple to implement successfully; it requires a commitment to building up the right capability, particularly analytical and technological
• some forces may need to enlarge their hot spots to have enough crime to get statistically significant results
• selection criteria for hot spots (crime type, time period) must be chosen carefully, both for sufficient crime and to identify persistent hot spots
• operational delivery needs to account for travel time and officer availability to ensure high compliance with the patrol schedule

Learnings for the HO include:

• providing guidance and findings through feedback meetings to allow forces to adapt their approaches where beneficial and to share success with senior leaders and front-line officers
• building alternative analytical models to cope with forces’ different operational approaches to ensure there is an analytical monitoring/evaluation framework flexible enough to cope with real-world operational priorities

Acknowledgements

The authors would like to thank a number of colleagues for their support with the research. Particular thanks go to Alex Dunnett and Cloe Cole. Thanks to the funded police forces who facilitated the evaluation, as well as the force analysts and academics who provided their own internal evaluation results. Thanks also to Ricky Wang at the London School of Economics and Political Science and Matthew Bland at the University of Cambridge for their support with peer review of the report. Finally, the authors would like to thank the Cambridge Centre for Evidence Based Policing and the College of Policing for providing forces with their expertise through individual force assessments and for their support in delivering hot spot policing.

1. Introduction

1.1 Policy background

In 2019, the Home Office (HO) announced that 18 police force areas (PFAs) would receive funding for the year ending March 2020 to enhance the operational policing response to serious violence (SV). The HO selected the areas by their levels of hospital admissions for injury with a sharp object experienced between the financial year ending 2016 and that ending 2018. In early 2020, the HO confirmed funding for this approach, known as Surge, for a second year (April 2020 to March 2021). Within Surge, PFAs were provided with broad discretion to apply the funding in the manner they considered would be most effective, provided it was aimed at interventions to reduce SV, particularly homicide, knife crime and gun crime (Home Office, 2021).

From April 2021, Grip funding replaced Surge and required 18 forces to deliver enhanced hot spot policing activity. Grip aims to immediately suppress and reduce SV through visible patrol activity in hot spots whilst also adopting strategic problem-oriented policing to address the root causes of violence within hot spot locations.

The Serious Violence Fund also covers the development of Violence Reduction Units (VRUs). The same PFAs which received Grip funding also received funding for the development of VRUs, which, like Grip, aim to reduce SV but focus on multi-agency prevention programmes.

In September 2021, Cleveland and Humberside were awarded bespoke funding to address increases in SV because they were the next 2 highest forces for hospital admissions for injury with a sharp object. The purpose of the funding was to implement hot spot policing in the highest SV hot spots in Cleveland and Humberside. As their approaches were similar to the Grip-funded forces, the HO included them in the analysis for the evaluation of hot spot policing.

1.2 Hot spot policing

Hot spot policing is a place-based policing intervention that focuses police resources and activities on those places where crime is most concentrated. For Grip, hot spots constitute small geographical areas (generally not larger than ward size, often specific streets or neighbourhoods) identified using data and intelligence as experiencing the highest volumes of SV. Focusing resources and activities in hot spots aims to prevent crime in these specific areas and potentially reduce overall crime levels in the wider geographic area.

There is an extensive evidence base, both in the UK and internationally, demonstrating the effectiveness of hot spot policing. Key findings from this research are summarised below:

Studies have shown that proactive, visible police activity in hot spots can decrease crime and disorder (Weisburd et al., 2008).

Some tactics work better than others: problem-solving at hot spots seems particularly effective at reducing crime and decreases crime for longer than visible patrols (Taylor et al., 2014; Braga et al., 2019).

Recent UK-based evidence of effectiveness of visible patrols comes from Surge-funded Operation Ark (Basford et al., 2021). Officers delivered high visibility patrols for 15 to 20 minutes at 20 high-harm, square-shaped hot spots of 150m by 150m. Violent crime, visible street crime harm, community violence and its related harm were all reduced in patrolled areas on patrolled days compared to non-patrolled days.

A Surge-funded randomised control trial in Bedfordshire found that even minimal amounts of foot patrol can prevent serious violent crime across a large area. The study also found evidence of a cumulative effect on crime, with the largest reduction in crime harm found after 3 days of consecutive patrol in the same Lower Layer Super Output Areas (LSOA)^{[footnote 4]}. However, this was with a small sample size of 21 hot spots, over 90 days, with pandemic restrictions in place (Bland et al., 2021).

1.3 Evaluation aims and objectives

The aims of this work are to:

• evaluate the effect of treatment on the crime rate within a hot spot area, where treatment is defined as a visible additional scheduled officer patrol within the designated area
• provide data and information about implementing hot spot policing across the funded forces, including how much is being delivered and how it is being delivered
• assess the cost effectiveness of Grip and bespoke-funded hot spot policing
• provide forces with regular feedback to allow continuous improvements to be made to their hot spot policing approach (creation of a data-driven feedback loop)
• expand the evidence base on hot spot policing, particularly in the UK

Much of the existing research on hot spot policing has been conducted in the US, which has different levels and types of crime and potentially different crime drivers. There have been few UK studies that have tested the effect of hot spot policing on violence and SV, rather than neighbourhood crime, and those that have tended to be small-scale pilots. Grip presents an opportunity to test the effects of hot spot policing on these crime types at a much larger scale.

Originally, the HO hoped to test the effect of the hot spot patrols on the most serious categories of violence, like homicide and knife/firearm-enabled crime. However, numbers within these offence groupings were too small for meaningful evaluation analysis. Instead, police-recorded robbery and violence against the person were used. Offences intrinsically linked to the presence of officers and officer action in a location were removed. By including violence against the person and robbery, homicides and a large subset of the knife and firearm-enabled offences were also captured. See Annex A for the full list of included offence categories.

Many hot spot policing evaluations also attempt to measure other impacts, like whether the patrols cause crime displacement (or its opposite, diffusion of benefit); whether patrols on a given day cause deterrence on subsequent days and which kinds of tactics used by officers within hot spots produce the best results. These are not aims for this evaluation, but the HO hopes to explore these in future iterations of this report.

1.4 Evaluation methodology

The different elements of the evaluation methodology are:

Delivery metrics: Force patrol data and information were processed to allow for an assessment of how hot spot policing is being delivered and how much is being delivered.

Impact evaluation: A meta-analysis containing 19 hot spot policing evaluations, including 4 with randomised design and 15 with quasi-experimental design.

Cost-benefit analysis: Used to estimate the societal costs and benefits of the funded hot spot policing programmes in the year ending March 2022.

Assessment of implementation: Forces and the HO held regular meetings to discuss the data, but also views from participating officers/analysts to understand barriers to implementation or areas for improvements.

1.4.1 Data and information used

The HO used several sources of data, as described below. The first 3 were bespoke data collections from forces, set up as part of the hot spot funding conditions. The fourth data source – the crime data – was part of a separate data stream between forces and the HO that pre-dates the funding.

Force patrol activity data by date and hot spot: Forces provided data on their patrol activity, tracked through a variety of methods (such as, GPS trackers, officer-completed template returns). There is no universal definition of a crime hot spot and the geometry and size of the treatment areas were largely at the discretion of the police force. Consequently, there was some variation across forces in the implementation of the Grip programme. In some, units were defined with free-form shapefiles, others used hexagons and some used patrol beats or administrative boundaries. The geographic granularity of the evaluation was defined for each police force and determined by the patrolled area and granularity of the available activity data. In some cases, activity data was only available at ward level whilst the patrol areas were sub-ward level. Currently all evaluations have been at either:

• hot spot/bespoke shapefile level
• Output Area (OA) level^{[footnote 5]}
• LSOA level
• ward level

Data on weapons collected during the funded period: Forces provided a data return giving aggregated totals of weapons collected by quarter. Forces varied in their IT systems and recording mechanisms for weapons data. This included collection method (such as, seized, surrendered, collected through a weapon amnesty), and type of weapon (knife/sharp instrument, firearm, other). Forces also varied in whether they could identify those solely collected on funded hot spot patrols.

Financial information: Forces also provided data on the proportion of their allocated budgets that went on:

• analytical systems
• analytical capability
• operational policing
• problem-solving

Crime data by date at the same geography as patrol data: Forces are required to provide crime data quarterly to the Home Office Data Hub under the Annual Data Requirement (Home Office, 2022). For this evaluation, crime locations were mapped and aggregated to the area level matching the activity data.

Social cost of crime estimates: The cost-benefit section of this report uses published figures on the social costs of different offences (Heeks et al., 2018).

Qualitative information: Qualitative information on implementation issues was also gathered informally via qualitative returns from the force to the HO, and via meetings and quarterly learning events with forces.

1.4.2 Analytical approach

Delivery metrics: The data received by forces was cleaned using R code^{[footnote 6]} to standardise date formats, correct typographical errors in area name and filter for relevant activity. Obvious errors were also removed, and further refinement sometimes took place following discussions with forces. R code was then used to pull out specific metrics from the data, notably the number of patrols and the days on which they took place. The HO manually checked this process in a sample of forces.

Impact evaluation: No two police forces’ approach to hot spot policing were identical. Therefore, this report effectively comprises 19 different evaluations. The HO analysed 15 by using a repeat crossover design that compared crime rates in hot spots on days with patrols (‘days-on’) and days without patrols (‘days-off’). Four were force-conducted internal evaluation results (often in partnership with academics). Methods used include a similar crossover method, as well as traditional randomised controlled trials (RCTs) using intervention and control areas. Four forces were not represented in the results. In one case, it was not possible to process the patrol data from the force due to technological challenges. In 3 cases, the intended force pattern of visiting hot spots every day meant that it was not possible to calculate impact through the day on/day off method. The HO is exploring an alternative approach that can account for these differences using control areas for future iterations of this report (see section 4.2.2).

In the day-on-day-off model (DODO model), each area serves as its own control. Many forces randomised the visitation schedule to mitigate bias in the visitation pattern. However, due to operational practicalities and compliance, actual visitations were often non-random. For non-randomised approaches, police may actively try to patrol during high-crime days or more frequently during high-crime periods. To account for this, the regression model introduces controls to account for the area’s crime rate, within-week variations and seasonality in crimes. Annex A discusses the regression model in more detail.

The HO applied the regression model to each PFA operation (some police forces had multiple operations), which yielded an estimate and CI for the treatment effect. A pooled effect for the programme as a whole (meta-analysis) was calculated by taking the average of the individual force-experiment estimates. This comprised 19 PFA model results from 16 forces (some forces ran more than one trial within the year). Two-tailed 95% CIs were applied to both the force-level results and the meta-analysis. As above, independent academic evaluations not assessed through the DODO model but funded as part of Grip were also included in the meta-analysis.

This report presents DODO results from both a negative binomial regression model and a Poisson regression model. Both models are common in the literature and reported with clustered robust standard errors. Two types of filters were applied to address 2 data issues:

Activity filter: This was applied because of operational challenges around extremes in visitation (either very low levels or visitation every day). The filter limits the analysis timeframe to periods with at least 4 activity/control days in every 28-day total.

Zero-crime day filter: This was applied because some selected hot spots had a high percentage of days without an offence, which is potentially problematic for the crossover design due to their appropriateness as hot spots, and the ability to identify a treatment effect (Hinkle et al., 2013). The filter removes these areas.

The HO tested the sensitivity of the results to these filters by generating results without the activity and zero-crime day filters. The only restriction applied was to limit the period analysed for each unit between their first and last visited date. This mitigated the risk of contamination on control days. The non-filtered results are presented in Annex B for reference.

The central estimates for percentage change in violence/robbery rates in the hot spots on patrol days relative to non-patrol days were converted into volumes of crimes prevented using a counterfactual method. It was assumed with no intervention that the average crime rate on days not patrolled would have applied to all days. The difference between this and the actual number of violence/robbery offences therefore provides an estimate for total violence/robbery prevented. This was done force by force and then combined into a pooled estimate with a CI using Monte Carlo simulation (see Annex A for more details).

Cost-benefit analysis: The analytical approach to cost benefit is set out along with the results in section 2.3.

Implementation assessment: The HO conducted regular conversations with forces to support and improve the implementation of their hot spot policing approach. Topics of conversation include but are not limited to:

• sharing initial findings
• method and accuracy of the patrol activity data
• pattern of visitation
• hot spot coverage
• methods of tasking and briefing officers
• officer compliance
• sharing best practice

The HO has also arranged learning events for forces to share how they have addressed challenges more widely and examples of best practice, as well as sharing initial findings at an England and Wales level. These have been instrumental in overcoming challenges, improving the programme during its development, and to make best use of funds and assess impact effectively. Key challenges raised during these conversations are discussed in chapter 3.

1.5 Limitations to the evaluation findings

The HO designed the evaluation to be as robust and comprehensive an approach as possible. However, the approach had some limitations which are important to bear in mind when considering the evaluation findings in this report. Key limitations to the data collection and analysis are listed below.

Treatment and control day definitions: Treatment and control days were defined for each area and day, and recorded crime counts were aggregated to this level. The HO designated a day as treated irrespective of the time of day that the patrol occurred. Similarly, a day was defined as a control provided it did not receive a patrol, irrespective of the time that had elapsed since its last patrol. For example, a visit at 23:00 would mean that day was considered treated and subject to the same treatment effect for the entire day, including the 23 hours before the patrol began.

Residual deterrence: A fundamental limitation of the repeat crossover experimental design and DODO model was residual deterrence. That is, if treatment on a given day affects crime rates on subsequent unvisited control days, then the treatment effect will be diluted (Sherman, 2022). The wider evidence on residual deterrence is unclear, especially in UK studies (Barnes et al., 2020; Basford et al., 2021; Bland et al., 2021). Residual deterrence has not been assessed in this evaluation, but it is hoped that it will be in future iterations.

Displacement and diffusion: The presence of a patrol in a designated area may have displaced crime by time or place, or had a beneficial diffusive effect, modifying the crime in the surrounding area. These effects were out of scope for this evaluation but may have modified the overall crime reduction narrative. Weisburd et al. (2006) found that crime displacement in hot spot policing areas was small and that the diffusion of crime control benefits was more likely.

Accuracy of crime location: Crime was mapped to a patrolled area using x-y coordinates. Uncertainty in the crime location may have resulted in crimes being mis-allocated into or outside a hot spot area. This risk was more acute in smaller areas where a greater proportion of crimes were near the area perimeter, or low crime rate areas that were more likely to be sensitive to specific crime events. In addition, as part of the crime mapping process, adjacent areas which share a boundary may double count crimes. Some crime recording systems used gazetteers that accentuated these issues.

Accuracy of crime date and time: Some recording systems automatically populated date and time of offence if no other information was recorded. The default time is set to 00:00 and the date will default to the first of the month, and the first of January of that year if no other information aside from year is recorded. This means there were recorded increases of offences on default dates, many of which will not relate to actual numbers of offences. These challenges limited the scope for widening the analysis to consider time-of-day analysis. However, the HO could account for some challenges by adding controls to the model for first of the month, to limit the impact of the recording bias (see Annex A).

Accuracy of patrol location: The HO assumed a patrol would uniformly treat and be limited to the entire designated area. Patrols that were visible outside of the patrol location, including travel to/from the area, could have affected crime rates in the surrounding areas and potentially other nominally untreated hot spots.

Area granularity: Some forces returned data at a higher level of geography, like wards, but conducted patrols in specific hot spots within those wards. This limited the evaluation’s capacity to detect statistically significant results (Andresen & Hodgkinson, 2018).

Quality of patrol data: Forces used a variety of methods for collecting patrol data. Some forces provided raw GPS data, some forces collated data manually, and some used a mixture of automated and manual. The quality and completeness of patrol data therefore varied by force. Additionally, the absolute number and proportion of control and treated days varied between hot spots and PFAs, especially when conditioned on co-variates. This may have led to a subset of areas dominating the result and sparse data bias.

Experimental conditions/contamination: The Grip programme ran alongside other normal police activities and other interventions which may have coincided with Grip treatment and control area/times. The crossover design should have mitigated most of these concerns because other interventions were unlikely to only affect certain days within the same area. The HO assumed that scheduled visitation was additional to normal levels of police patrols and response. One force – the Metropolitan Police Service – was able to accurately record the additional amount of patrol delivered on top of business-as-usual presence. See Annex C for further details.

Pooling results across multiple experiments and PFAs: The pooled result took an average of each force-experiment evaluation. However, there was a large degree of operational freedom in how the police force enacted Grip. Consequently, each estimate was derived from experiments that may have varied significantly in properties, such as, the patrol length, number of patrols, patrol area and crime rates.

2. Results

This chapter presents the results, including the delivery metrics, impact evaluation and cost-benefit analysis. Section 1.4 explains the methodological approach used (with Annex A providing additional detail).

2.1 Delivery metrics

Grip forces and forces receiving bespoke hot spot policing funding in the year ending March 2022 carried out at least 120,000 patrols. Force data returns encompassed a wide variety of approaches to conducting and recording hot spot policing, which made it very difficult to draw meaningful comparisons from total patrol figures. For example:

Forces differed in their patrol lengths, ranging from 15 minutes to 8 hours.

Forces also recorded patrols at different geographies, ranging from areas as small as 150m x 150m boxes to entire wards. Annex C also contains estimates for some forces of the proportion of (1) the force’s geographical area, and (2) the proportion of crime (or basket of crimes) covered by the hot spots. Again, there was considerable variation with one force focusing on just 0.22% of the force’s area, which accounted for about 3.4% of violent crime, and another force focusing on 7% of the geographic area and 44% of the crime. Generally, the forces that used larger areas still defined smaller areas within them to focus patrol activity on. This requires noting when considering the degree of overall crime reduction that can be expected from hot spot policing.

Different forces also had different numbers of officers within each patrol.

Annex C contains details of each force’s approach and other aspects of delivery, like whether forces tasked their patrols via overtime or specialist teams, whether they employed randomisation, and whether they used GPS systems to track patrols or more manual methods.

One force – the Metropolitan Police Service (MPS) – used GPS pings to accurately record the additional amount of patrolling delivered on top of business-as-usual presence in the hot spots. Whilst there was already considerable business-as-usual presence in the hot spots, Grip increased officer patrol minutes by 54% in the first operation and 132% in the second (see Annex C for further details).

Due to the differences in patrols highlighted above, comparisons should not be made between forces in relation to level of activity. These differences also meant that it was not possible to produce a programme-level estimate for compliance (the proportion of hot spot patrols that were completed as planned). But the HO calculated estimates for a subset of forces, and these are shown in Annex C.

Table 2.1: Total patrols carried out by funded forces, year ending March 2022

Police force area	Additional patrols from Home Office funding	Funding allocated
Avon and Somerset	5,596	£773,955
Bedfordshire	559	£630,630
Cleveland	5,327	£390,000
Essex	15,536	£786,000
Greater Manchester	5,742	£2,178,540
Hampshire	2,157	£574,660
Humberside	5,597	£389,826
Kent	2,088	£745,290
Lancashire	1,174	£831,285
Leicestershire	4,031	£630,630
Merseyside	3,127	£1,889,000
Metropolitan Police	30,826	£9,430,785
Northumbria	5,345	£1,060,500
Nottinghamshire	1,942	£687,960
South Wales	662	£465,500
South Yorkshire	4,428	£1,174,650
Sussex	2,213	£601,965
Thames Valley	4,223	£859,950
West Midlands	20,612	£3,439,800
West Yorkshire Police (WYP)	831	£1,834,560
Total	121,185	£29,375,486

Notes:

1. Bedfordshire and WYP could not return separate data for all activities conducted using their funding allocation. Their total patrol estimates above should therefore be treated as a lower bound.

2.1.1 Weapons

The aims of the funding were to reduce SV and, therefore, whilst the primary activity for the funded forces was to conduct visible hot spot patrols, forces also conducted additional activity in the hot spots. They often aimed these tactics at increasing visibility, as well as improving safety and reducing SV. This included weapons sweeps, stop and search, and holding weapons amnesties. To capture the additional activity, forces were required to submit returns on the number of weapons collected within the force during the funding period. Across the 18 Grip-funded forces for the whole financial year ending March 2022, and the 2 bespoke hot spot fund forces for the latter 6 months (September 2021 to March 2022), over 80,000 weapons were collected. These were split fairly evenly across the quarters.

Figure 2.1 Total weapons collected during hot spot funding 2021/22 by quarter

Forces varied in their IT systems and recording mechanisms for weapons data. This included collection method (such as, seized, surrendered, collected through a weapon amnesty), and type of weapon recorded (knife/sharp instrument, firearm, other). Forces also varied in whether they could identify those solely collected on funded hot spot patrols. As a result, it is not possible to report on collection method, weapon type, and whether the collection can be solely attributed to the funding.

2.2 Impact analysis

The results for estimated impacts on violence/robbery are in the following forest plots and tables. These include both estimated force-level impacts and our estimated pooled impact for the programme as a whole:

Figure 2.2 Forest plot for the effect size using the Poisson model

Table 2.2: Results for estimated impacts on violence/robbery using the Poisson model

Police force area	CI lower	CI higher	Effect	P-value	Sig
Police force 1, Design A (RCT)	-48.2%	4.2%	-22.0%	0.10
Police force 2 (RCT)	-40.1%	5.4%	-20.5%	0.11
Police force 3, Design A (RCT)	-28.2%	-0.6%	-14.4%	0.04	**
Police force 5 (RCT)	-28.1%	2.9%	-12.6%	0.11
Police force 6, Design A	-28.6%	11.9%	-10.6%	0.33
Police force 1, Design B	-21.8%	2.3%	-10.6%	0.10
Police force 4	-46.0%	48.6%	-10.5%	0.67
Police force 7	-21.4%	5.2%	-9.0%	0.20
Police force 8	-15.0%	0.0%	-7.8%	0.05	**
Police force 9 (RCT*)	-17.7%	2.9%	-7.4%	0.16
Pooled average	-10.7%	-2.0%	-7.0%	<0.01	***
Police force 10 (RCT*)	-12.5%	-1.0%	-6.9%	0.02	**
Police force 6, Design B (RCT)	-17.6%	6.4%	-6.4%	0.31
Police force 11 (RCT*)	-13.2%	6.2%	-4.0%	0.43
Police force 12	-11.5%	9.7%	-1.5%	0.79
Police force 3, Design B	-7.2%	6.4%	-0.7%	0.85
Police force 15	-5.4%	6.2%	0.3%	0.93
Police force 14 (RCT)	-17.0%	25.2%	1.9%	0.85
Police force 13	-21.8%	39.9%	4.6%	0.76
Police force 16	-7.2%	20.5%	5.7%	0.40

Figure 2.3: Forest plot for the effect size using the negative binomial model

Table 2.3: Results for estimated impacts on violence/robbery using the negative binomial model

Police force area	CI lower	CI higher	Effect	P-value	Sig
Police force 1, Design A (RCT)	-48.2%	4.2%	-22.0%	0.10
Police force 2 (RCT)	-40.2%	7.1%	-20.0%	0.13
Police force 3, Design A (RCT)	-28.2%	-0.6%	-14.4%	0.04	**
Police force 4	-49.4%	49.1%	-13.1%	0.61
Police force 5 (RCT)	-28.1%	2.9%	-12.6%	0.11
Police force 6, Design A	-28.5%	13.0%	-10.1%	0.36
Police force 1, Design B	-20.8%	3.2%	-9.6%	0.14
Police force 7	-22.5%	7.9%	-8.5%	0.29
Police force 8	-16.1%	1.9%	-7.5%	0.12
Police force 9 (RCT*)	-17.7%	2.9%	-7.4%	0.16
Pooled average	-11.0%	-2.1%	-7.2%	<0.01	***
Police force 10 (RCT*)	-11.4%	-0.2%	-5.9%	0.04	**
Police force 6, Design B (RCT)	-16.6%	7.9%	-5.1%	0.42
Police force 11 (RCT*)	-13.2%	9.5%	-2.5%	0.67
Police force 3, Design B	-8.9%	5.9%	-1.8%	0.64
Police force 12	-10.9%	9.0%	-1.5%	0.77
Police force 13	-24.8%	30.8%	-0.8%	0.95
Police force 14 (RCT)	-18.0%	22.9%	0.4%	0.97
Police force 15	-5.4%	7.0%	0.6%	0.85
Police force 16	-9.6%	21.8%	4.9%	0.53

Notes:

1. RCT* denotes cases where randomisation was involved but only for part of the period, or some areas or had a known issue, for example, where an element of officer choice compromised the randomisation process to some degree. Note that the other RCT experiments also had varying levels of compliance, which can lead to randomisation issues, but this was mitigated by still using day and month controls within our models.
2. Independent academic evaluations are Police Force 1 Design A, Police Force 3 Design A, Police Force 5 and Police Force 9. These academic evaluations were not passed through the DODO model and remain unchanged between the 2 forest plots. For CIs not provided by the independent studies, the HO estimated these from the p-value.
3. Statistical significance at the * 10% level; ** 5% level; *** 1% level.

At the programme level, a statistically significant reduction in violence/robbery is found for both the Poisson model and the negative binomial model. For the Poisson model, a reduction of 7.0% was seen (CI: -2.0% to -10.7%) in the funded forces during the year ending March 2022. For the negative binomial model, a reduction of 7.2% was seen (CI: -2.1% to -11.0%)^{[footnote 7]}. This means that, on average and controlling for other factors, crime was 7% lower on patrol days than non-patrol days in the hot spots. Note that this is different from saying that crime fell 7% in the hot spots; this analysis compares patrol days within hot spots to non-patrol days, so a 7% difference between the 2 would only equate to a 7% reduction in offences within the hot spots if the hot spots were all visited every day, which was not the case^{[footnote 8]}.

The meta-analysis was unweighted, meaning every force’s result was counted equally. Some studies use weighted meta-analyses. For sensitivity analysis, a weighted result was calculated using the inverse variance-weighted average model with random effects (see Annex D). The central result, when weighted, still showed a statistically significant reduction for the programme.

At force level, few forces demonstrated a statistically significant reduction in crime. For the negative binomial model, this was 2 forces at the 5% significance level; for the Poisson model, this was 3 forces at the 5% significance level. No forces showed a significant increase in crime.

Some other points about the results are worth noting. First, the RCTs tended to have better impact results. Their average reduction was about 10% compared with around 4% for the studies that did not use a randomised design. This is somewhat at odds with the previous literature in which less robust designs tended to produce larger impacts (Braga et al., 2019). It is not immediately clear what is driving the difference, but some possibilities include:

1. The commitment to run an RCT may indicate a greater commitment to hot spot implementation generally.
2. The RCTs may have captured the effect more cleanly. For forces without randomised patrol patterns, many will have focused resources on periods when crime is highest. It is possible the controls may have not completely removed this bias.

Second, it is important to recognise that the pooled central estimate is, in effect, an average across the hot spots in this programme of funding. As already discussed, these hot spots were not sized consistently. It was generally true that areas with larger hot spots had smaller percentage reductions. This makes sense and is likely to reflect a real-world trade-off. Bigger hot spots bring more total crimes into play, but percentage reductions in them are likely to be smaller. For example, Annex D shows that in aggregate the smaller, bespoke-area results tend to have larger percentage reductions and smaller p-values when pooled, whereas the (larger area) ward level results have smaller percentage reductions and larger p-values when pooled. But the larger areas would generally have greater crime volume reduction for any given percentage reduction.

Using the counterfactual approach described in section 1.3, the central estimate for violence/robbery crimes prevented was 1,100 (CI: 400 to 1,900), equating to around 100 per month. The next section explains how value for money was assessed.

2.3 Cost-benefit analysis

The estimate for crimes prevented enables a cost-benefit analysis to be completed. This is set out below using the results from the Poisson model. The results from the negative binomial model are very similar. The methodology for the cost-benefit analysis uses the published Economic and Social Cost of Crime estimates (Heeks et al., 2018). Full details are in the publication; in summary, the report considers 3 main cost areas:

1. Costs in anticipation of crime, for example, the cost of burglar alarms.
2. Costs as a consequence of crime, for example, the cost of property stolen or damaged, or the physical/emotional effects on victims of violent crime.
3. Costs in response to crime, for example, costs to the police and criminal justice system.

The HO used these costs to estimate the societal benefit from policies that reduce crime. In this case, the central estimate for crimes reduced in the year ending March 2022 was 1,100 violence against the person and robbery crimes (CI: 400 to 1,900). The Costs of Crime publication lists social costs for the relevant offences for the year ending March 2016 as follows: robbery (£11,320), violence with injury (£14,050) and violence without injury (£5,930). These were up-rated to year ending March 2023 prices using the latest HMT deflator (HM Treasury, 2023) to produce current estimates of robbery (£13,600), violence with injury (£16,900) and violence without injury (£7,100).

The results treat violence and robbery as a single basket of offences – there are no estimates for the individual components. So, the central estimate for crimes prevented (1,100) was weighted by the proportion of the different component parts in the forces that received Grip funding. In other words, the HO assumed that the make-up of the Grip crimes prevented (in terms of the proportion that was violence and the proportion that was robbery) was the same as for the total amount of crime across those forces.

An additional assumption here is that the minor categories of violence that were removed, for example, assault on a constable, do not affect the average cost of violence across the whole category.

Finally, multipliers were also used to account for the fact that this evaluation uses crime that has been reported to and recorded by the police. Not all crime is reported and recorded. So, the Social and Economic Cost of Crime publication includes multipliers to estimate the additional crime that goes unrecorded. For the relevant offences, these are:

• violence with injury: 2.6
• violence without injury: 1.5
• robbery: 4.3

Combining the weighting and the multipliers, the cost of a single Grip crime prevented was estimated to be £32,500 in year ending March 2023 prices. Given the central estimate of 1,100 crimes prevented, this equates to £36 million in societal benefits (CI: £13 million to £62 million).

The HO broke down the estimated costs of the programme into 3 main components:

1. Funding allocations to the police forces.
2. HO and other non-police staff involved with delivering or evaluating the programme.
3. College of Policing funding to deliver support to forces.

Section 2.1 shows the force allocations for the year ending March 2022 total £29.4 million. For the cost-benefit analysis, not all the allocated spend was included on the cost side of the equation. This is because some of the funded activity could not be evaluated and hence could not generate a benefit. Therefore, the HO excluded these cases, which total £13.7 million, from both costs and benefits. Examples include:

• one force ran both a crossover (DODO) design and a parallel-track design (using intervention and control areas), but due to technical difficulties it has not yet been possible to evaluate the latter
• several forces began problem-oriented policing (POP) activity in the year ending March 2022, separate to the main visible patrols, which has not been evaluated at this stage
• some forces ran both an RCT and a non-randomised version in other areas; due to resources and data issues, the HO could only evaluate the RCT
• a couple of forces could not provide data that split out Grip activity from business-as-usual activity within hot spots – this made evaluation impossible

These issues have largely been fixed for the year ending March 2023, so it is hoped that estimates for all forces will be available in the next update of this report.

Excluding these ‘non-evaluated’ spends, the remaining allocation cost of the programme (the amount entered into the cost-benefit model) was £15.6 million.

HO staff costs were estimated using internal wage and time-spent figures. The London School of Economics and Political Science was also involved, providing guidance on the evaluation method, so the cost of their time was also included. Taken together, this added a further £0.7 million in costs.

Adding these staff costs to the allocation total gave £16.4 million (when rounded). Combining this with the central estimate for benefits (£36 million) implied an estimated benefit-cost ratio (BCR) for the Grip and bespoke hot spot funding of £2.18 (CI: £0.79 to £3.77), indicating a return on investment of about £2.20 for every £1 spent.

It should be noted that some possible effects have not been captured in this analysis. For example:

• the effects on other crime types
• any residual deterrence effects (patrols on day 1 could still deter crimes on day 2)
• displacement and/or diffusion of benefit effects
• effects on charge/arrest rates, which could generate/reduce additional criminal justice system costs
• other community impacts

Plans are in place to develop this evaluation to include all the above, although some will take longer to achieve than others. It is hoped that several of the above will be included in the next iteration of this evaluation covering the year ending 2023.

3. Discussion

This evaluation is based on the first year of centrally funded hot spot operations for SV across England and Wales, and, as such, is considered a development year. As well as developing the ability to identify, resource and conduct visible hot spot policing, forces also needed to develop technology, skills and methods relating to the operations. The HO expected forces would make changes based on feedback and on their own observations, as well as through the improvement of technology and analytical skills, throughout the year.

This chapter summarises the challenges faced when implementing and analysing hot spot policing. The HO discussed these challenges with forces during the feedback loop process, along with methods for improvement. The chapter considers challenges against the various stages of the process, including:

• identifying hot spots
• resourcing Grip
• deciding what to do in the hot spots
• briefing and engaging staff
• monitoring delivery
• evaluating impact

3.1 Identifying hot spots

3.1.1 Selection of crime types for hot spot identification

The HO provided forces with recommendations on which crime types to include for the selection of hot spots but allowed flexibility to account for local drivers in different areas. This was generally useful in ensuring that forces targeted the areas that required visible hot spot policing most. But there were also issues. Some forces, for example, included possession offences, which were likely to be linked to police activity and were perhaps better removed both from hot spot identification and impact analysis (this has been advised for subsequent years). In addition, the HO’s approach also meant there was inconsistency between the crime types forces used to identify hot spots and the crime types used in this evaluation to measure impact for all forces (see Annex A). The results presented here may therefore underestimate the impact for forces where the offence categories used to select hot spots differed from those in the evaluation model. It is also worth noting that many forces restricted violence offences to those that were non-domestic ones or public space. Although this makes sense for hot spot policing, the Home Office did not have the capability to do that for all forces. This is also a potential improvement for future years.

3.1.2 Selection of time periods for hot spot identification

The HO encouraged forces to identify areas with persistent high crime using several years of crime data. However, COVID-19 lockdowns led to reductions in most crime types, especially SV in areas with high footfall, night-time economy and transport hubs. Therefore, using data affected by COVID-19 would present a skewed picture of hot spots if not countered by including pre-lockdown data and, when available, post-lockdown data, to understand longer-term hot spots and changes made by the impact of the pandemic on society.

3.1.3 Crime harm versus crime volume for hot spot identification

Traditionally, hot spot policing uses crime volume to determine the ‘hottest’ areas in need of visible patrols. More recent work has also looked at harm, recognising that the impact of offences is not equal across all offence types, and that, especially when considering SV, accounting for the level of harm is a key determinant of the impact on both victims and society in general. For example, a homicide offence has a much larger harm cost than a robbery offence (Heeks et al., 2018).

For the year ending March 2022, forces had autonomy to select their hot spots on either volume, harm, or both, but they needed to be analytically driven. However, because of the higher harm associated with SV offences, a single high-harm offence like a homicide can skew the selection. The HO found clusters of SV, as indicated by volume, to be more reliable over time than a few high-harm offences over a long period. As a result, the HO advised forces to use crime volume as the primary determinant of hot spots when they re-profile for future years.

The DODO model only used volume, to provide a consistent approach across all the forces. There is scope for considering the impact on crime harm as well in future analysis.

3.1.4 Hot spot shape

Forces had autonomy to decide what shape and size worked best for their force area and operations. The HO required trained force analysts to use mapping software to identify the areas with the highest concentration of crime or harm. One method of doing so is to:

1. Divide the entire force area into a grid system (typically squares or hexagons).
2. Map offences over the grid system.
3. Identify the highest crime zones for patrol.

Forces found that grids did not always produce suitable shapes for patrols. For example, a square would cut off street corners that would naturally be part of the patrol. This also created issues with capturing patrol data where GPS signals were lost as soon as officers stepped out of the hot spot. One approach to address this is to use the grid system to find the areas, then manually create a shape for a patrol which is more operationally feasible. This could mean amending the square to cover the streets effectively or by drawing a bespoke shape based on the geography of the area. This challenge is being addressed by forces as they have re-profiled for future years, and the issue should reduce in future operations and analysis.

3.1.5 Hot spot size

Much of the evidence on hot spot policing uses micro hot spots, typically 150m x 150m (Braga et al., 2019). However, most evidence is based on programmes conducted in places (US and Central and South America) with different crime rates, density and profiles than areas in England and Wales. Forces have used a variety of hot spot sizes, from approximately 150m x 150m up to ward level (although at ward level, specific areas were targeted rather than patrolling the entire ward). Many of the smaller areas in this evaluation arguably did not have a sufficient density of crime to test for statistically significant effects within a single year. Clearly though, there may be an operational trade-off. Bigger hot spots will have more crime, but the tactic’s effectiveness may be diluted. This evaluation cannot say what the ideal hot spot size is, and it is likely this will vary by force. The HO aims to look further at this question in future funding years. Note that this also affects patrol length. Some studies suggest 15-minute patrols (Braga et al., 2019; Bland et al., 2021), whilst the length of patrols for this evaluation ranged from 15 minutes to several hours. If some areas of England and Wales are better served by slightly larger hot spots, then these may need slightly longer patrols in order to provide visible deterrence across the whole area.

3.2 Resourcing Grip

3.2.1 Ring-fencing operational resource

Forces resourced Grip through several methods, including officers on overtime, a dedicated Grip officer team, utilising neighbourhood policing teams, or a combination of methods (see Annex C). Feedback from forces suggested that, whilst overtime was mostly effective, it was difficult to staff the patrols during high-intensity periods (such as event days, public holidays, during high-resource response activity, or due to a reduction in staff during high levels of COVID-19 sickness absence). Often this would occur at times the force would want to target, such as weekends, when crime is typically higher. Several forces were better able to ensure high compliance (patrols being completed when scheduled) with a dedicated team and relying on overtime only to add resource if the dedicated team were short of staff. That said, one or two forces still achieved very high compliance rates using pure overtime models suggesting that they can still be effective if they have strong senior buy-in.

3.2.2 Time distribution of crime

A challenge that particularly affects resourcing for forces is the time-profile of crime. Whilst it depends on crime type and the local profile of the hot spot (for example, night-time economy, transport hub, school route), most crime does not occur during a standard working day. Whilst policing is not fixed to typical hours, it can be more difficult to resource shifts outside of the core working day. This may affect sign-up for overtime shifts where officers already have late shifts assigned as part of their working schedule. Therefore, targeting hot spots at the times when offending is more likely to occur can be a resourcing and compliance challenge for forces. To address this, some forces have used a combined overtime/dedicated team approach; others have worked with existing night-time economy teams or non-police patrols in night-time economy hours; and some have changed their hot spots to reduce the number but increase attendance. This will continue to be developed in future years of funding.

3.2.3 Analytical support for forces

Forces have varying levels of existing analytical resource and skills, as well as variations in technology available to them in-house. It has taken time for forces to increase their analytical capability to support the Grip work. This has improved for future funding years and should have less of an impact operationally in the future. Separately, the HO has aimed to support forces analytically through reporting internal analysis, providing feedback through shared learning events, and assigning HO analyst contacts for forces to liaise with. This has been beneficial in the development of analysis in the forces as well, but requires force analysts to provide support at all stages.

3.2.4 Staff turnover

A challenge from an operational perspective is the frequent change in personnel. This occurs across all roles, including patrolling officers, analysts and senior leads. A key learning point is that hot spot policing is not a quick fix. It requires commitment over several years to achieve a good model, considerably helped by a consistent team of people. Turnover presents several challenges, including loss of knowledge about the programme, resource to train new members, and a loss of buy-in at senior level in some instances. It is difficult to prevent the frequent change and forces have fed back to note the issues that short-term funding can cause in relation to the recruitment and retention of analytical staff. The HO is funding the programme based on a multi-year Grant Agreement, covering the financial years ending March 2023 to 2024, which may help to mitigate this issue.

3.3 Deciding what to do in the hot spots

3.3.1 Patrol activity

The HO advised forces on options and they implemented a variety of methods, including:

• visible patrol (baseline requirement)
• community policing
• suppression/offender targeting
• stop and search
• covert activity
• problem-solving including using Risk Terrain Modelling (RTM)

Due to the early stage of the programme, and the developmental nature of the first year, it has not yet been possible to isolate the different approaches taken to identify best practice and advise forces accordingly. However, as the funding continues, the evaluation aims to assess these approaches, to determine the more successful methods.

3.3.2 Pattern of visitation

A key challenge for forces was to develop a pattern of visitation which was proactive, rather than reactive. For the DODO model to be as robust as possible, visitation patterns should be randomised or quasi-randomised with a fixed pattern of patrols to be adhered to regardless of crime patterns. However, a key issue is the need to balance operational aims with analytical purity. For example, consistently visiting hot spots on the highest crime days could have the biggest reduction effect on crime. Feedback has been provided to forces on the rationale for randomisation or quasi-randomisation, to encourage this for future years where possible.

The HO is also working to develop a method that is more robust to instances where visitation is targeted at high-crime days (see section 4.2.2). The aim is to be able to compare results from both methods and gain a fuller picture of the impact of visible hot spot policing and to be flexible to operational realities.

3.3.3 Travel time between hot spots

One factor that forces fed back as an operational challenge was the travel time between hot spots. In some cases, the high-crime hot spots were clustered in close proximity and officers could easily travel between them without detracting from the time spent on patrols during their assigned shift. In other cases, hot spots were spread across a wider geography. This meant officers were having to spend a long time travelling between the areas, and this reduced the number of patrols or the time they could spend patrolling. This challenge has been addressed in several ways, depending on the exact profile of each force. This has included clustering hot spots that are in proximity for a shift, so that officers are assigned hot spots all within a reasonable distance; increasing the patrol length but not requiring a return to the same hot spots on the same day; and working with local teams to resource the more isolated hot spots.

3.4 Engaging and briefing staff

3.4.1 Cultural barrier

The levels of support for hot spot policing within police forces varied, presenting a cultural barrier to implementing hot spot policing. This may include senior buy-in being limited, with a lack of support for analytical approaches for patrol selection over intel, and reluctance over implementing patrol schedules requiring fixed resource that could take away from other work. Some forces reported that some officers enjoyed the community engagement and the perceived return to ‘bobbies on the beat’, but others found the patrols repetitive and without action. Key messaging around success and the impact the patrols had can support buy-in at all levels, encouraging officers by demonstrating the purpose of the patrols, and persuading senior buy-in through results.

3.4.2 Best type and method of briefing

To achieve the best possible compliance, both in attendance and full engagement in the patrol activity, and to increase officer buy in to hot spot policing, the briefing for the activity is key. Forces have trialled varied methods to maximise the impact of the briefings, as well as ensure consistent messaging. So far, there is not a consensus on best practice, but forces have been able to refine their approach throughout the funding and improve their communication to patrolling officers.

3.5 Monitoring delivery

3.5.1 GPS versus manual tracking

Police forces used a variety of methods for tracking patrols, including GPS trackers, body-worn video, and police radio (airwaves). Some were easier to implement than others, but all had strengths and weaknesses. Where forces utilised pre-existing technology, the methods for extracting or accessing the relevant data could be complicated or required significant data sharing work to arrange. Where new technology was easier to access, the time required to set this up impacted on the operations or their recording. Some forces used manual tracking (such as officer-completed forms) to track patrols whilst other technology was being explored, which ensured data was available for the analysis, but was not as precise or robust as automated tracking. Therefore, a wide range of methods with varying quality were used to capture the data for the patrols.

This had both an impact operationally, as some methods were resource intensive, and analytically, as the quality of the data in the evaluation was mixed and therefore could have affected the accuracy and reliability of the evaluation. The quality of data could have been affected in the accuracy of location, and poor data recording in the completing of the returns (varying from missing data to incorrect spelling and date/time formats). All these issues could have affected the reliability of the analysis. For future funding years, all forces are being encouraged to move towards automated tracking solutions.

3.6 Evaluating impact

3.6.1 Overlapping hot spots

During the mapping process, there were instances of overlap between hot spots. This posed a challenge for the analysis, as the DODO model relied on being able to capture the crimes recorded in a particular hot spot on a particular day. If hot spots overlap, this could lead to double counting and contamination of controls/treatment days or locations. For the DODO analysis, overlapping areas were dissolved into a single hot spot, with the entire area assumed to be treated when any constituent hot spot was Grip patrolled.

3.6.2 Availability and accuracy of crime data

Some forces struggled to provide timely crime data. This particularly applied to forces changing recording systems and, in some cases, limited the ability to include these forces in the analysis.

An additional issue was the accuracy of the coordinates recorded for each offence. This can manifest in 2 ways:

1. Some recording systems automatically populate the coordinate field with the GPS location of where the offence is being recorded, often the police station. Although this can be changed to the actual offence location, sometimes this does not happen.
2. The method of recording location on some systems requires manual input. This may mean that a postcode is entered, for example. The system may then record the crime coordinates at the centre of the postcode rather than where the offence actually occurred.

Some forces have commissioned work to improve the coordinate recording in the police-recorded crime data.

3.6.3 Changing hot spots and statistical power

As the year ending March 2022 was a development year for Grip, many forces regularly changed their hot spots. Whilst it was important for forces to make changes and improve their approach, this presented a challenge for analysis. Due to low power in the model (especially when areas were visited for only short periods), it was not possible to assess the impact of the shorter visitations effectively. Therefore, the HO could not effectively assess some of these initial approaches to see if they were having an impact. Forces have been advised to keep hot spots as consistent as possible to aid future evaluations. Operationally, keeping chosen hot spots consistent should be the most effective approach, as the evidence base suggests that crime hot spots remain ‘hot’ for long periods (Braga et al., 2010; Groff et al., 2010; Weisburd, 2008).

3.6.4 Separating visible patrols in hot spots from baseline police activity

Many hot spots already had police resource directed towards them in a variety of forms including existing police patrols, for example, from neighbourhood teams; night-time economy work; and general police response to incidents in high-crime areas. This evaluation aims to measure the effect of the additional funded patrols. It is not an assessment of police presence more generally. MPS was able to measure this additionality – see Annex C – and, for at least one of their experiments, the non-Grip presence outweighed the Grip presence, which is likely to dilute the effect of the additional patrols.

3.6.5 Issues with randomisation

The HO encouraged forces to run RCTs, and several took up that opportunity. These experiments revealed 2 important learnings.

First, although patrol schedules can be randomly set, non-compliance to those schedules is likely to be non-random (for example, Sundays may be a consistently harder day to get good compliance). This can bias results and although it may be mitigated by using an intention-to-treat approach – in which the analysis is conducted based on planned treatment days rather than actual treatment days – that is a slightly different type of test because it tests compliance as much as the effectiveness of the patrols themselves.

Second, crime levels vary hugely even within hot spots. That means that randomisation between control and intervention areas can result in unequal groups of areas if, for example, the two or three hottest spots get allocated to the same group by chance. One way to mitigate this is to sub-divide the randomisation; for example, randomise first among the top set of areas only, then the next set, and so on.

4. Conclusions

4.1 Summary of findings

The aim of Grip is to immediately deter serious violence (SV) through visible patrol activity in hot spots whilst also adopting strategic problem-oriented policing to address the root causes of violence within hot spot locations. From April 2021, the Home Office (HO) awarded Grip funding to 18 forces. In September 2021, the HO awarded 2 further police forces with bespoke funding to conduct hot spot policing in the same way as the Grip-funded forces.

This evaluation has shown that the main impacts of the funding in the year ending 31 March 2022 were that:

• forces receiving hot spot policing funding carried out at least 120,000 patrols and collected at least 80,000 weapons
• an average -7.0% (CI: -2.0% to -10.7%) reduction in violence against the person and robbery offences in hot spot areas on patrol days relative to days not patrolled
• an estimated 1,100 violence against the person and robbery crimes were prevented (CI: 400 to 1,900), and £36 million in societal benefits, indicating a return on investment of about £2.20 (CI: £0.79 to £3.77) for every £1 spent

The latter 2 results stemmed from a meta-analysis comprising 19 results from 16 forces. Methods used include a repeat crossover design which compared crime rates in hot spots on days with patrols and days without patrols, as well as RCTs using intervention and control areas. The meta-analysis does not include some areas/activity for which effects could not yet be calculated, nor does it include impact on any other crimes. The true benefit may therefore be larger although displacement has not yet been examined.

At force level, few forces demonstrated a statistically significant reduction in crime. And even at the programme level, the effect was smaller than many in the wider literature. There are several potential explanations but 2 that seem particularly important are:

1. Whereas the existing evidence is based largely on small-scale pilots, this programme was almost national in scope (the 20 police forces accounted for over 80% of hospital admissions for assault with a sharp object in England and Wales).
2. Much of the existing evidence for hot spot policing comes from US cities where crime rates are generally higher.

Alongside the analysis, the HO conducted regular conversations with forces to support and improve implementation of their hot spot policing approach. These have significantly helped to overcome challenges and improve the programme during its development, to make best use of funds and assess impact effectively.

Key recommendations for improvement in the year ending March 2023 include:

• whilst hot spot policing is simple in concept, it is not simple to implement successfully; it requires a commitment to build up the right capability, particularly analytical and technological
• some forces may need to enlarge their hot spots to have enough crime to get statistically significant results
• selection criteria for hot spots (crime type, time period) must be chosen carefully, both for sufficient crime and to identify persistent hot spot
• operational delivery needs to account for travel time and officer availability to ensure high compliance with the patrol schedule
• the HO should continue to provide additional guidance from findings through feedback meetings to allow for forces to adapt their approaches where beneficial
• the HO should build alternative analytical models to evaluate forces with different operational approaches

4.2 Future development of Grip

4.2.1 Future Grip

The Home Office is committed to delivering at least 2 further years of funding for the programme (up to year ending March 2024) for the same 20 police force areas to deliver targeted operational policing activity in violence hot spots. This will allow many of the challenges discussed in chapter 3 to be addressed. The HO will continue to work with forces to improve their hot spot programmes and evaluate impact.

4.2.2 Alternative models

The day-on-day-off (DODO) model compares crimes on patrol days versus crimes on non-patrol days. In some cases, the force planned a visitation pattern of visiting hot spots every day. This may be effective in reducing SV in high-crime areas but meant that it was not possible to calculate impact through the DODO method. It would therefore be preferable to have other models that could calculate a robust effect where hot spots were visited every day, by using robustly derived control areas.

4.2.3 Displacement and diffusion

The presence of a patrol in a designated area may have the effect of displacing crime by time or place, or have a beneficial diffusive effect, modifying the crime in the surrounding area. The HO have started work to examine displacement or diffusion of benefit, that is, whether Grip patrols displace crime to another area or the opposite, whether benefits extend beyond hot spot boundaries (diffusion of benefit).

4.2.4 Problem-oriented policing (POP)

Most forces have plans to run POP alongside hot spot policing in the year ending March 2023. This is to be welcomed as the evidence for POP is strong. The HO is working with forces to ensure their POP analytical setup is right, to enable analysis.

4.2.5 Community insight

There is little evidence to show hot spot policing negatively impacts on police and community relations (Braga et al., 2019). However, the evidence is limited, and the potential impacts of hot spot policing on communities may depend on the approaches used and the context of the hot spots affected. To help understand more about the impact of Grip on individuals and communities, the HO has created a separate fund for successful research proposals from forces.

Annex A: Technical annex

This technical annex details the evaluation methodology for the crossover design using the day-on-day-off (DODO) regression model. The aim of this work is to evaluate the effect of treatment on the crime rate within a hot spot area.

The analytical pipeline comprised the following stages:

1) Activity and crime data merge

The Home Office (HO) cleaned and formatted the activity data received from police forces so that each area and day was assigned a binary visitation flag, denoting ‘0’-control or ‘1’-treatment day. Treatment is defined as at least one additional scheduled visible officer patrol within the designated area.

Home Office crime data from the Home Office Data Hub (HODH) was mapped to each area and aggregated to the area and day, matching the geospatial granularity of the activity data. All evaluations have been defined by either: (1) shapefiles – a bespoke area defined by the police force, or (2) by an administrative boundary such as Output Area (OA), Lower Layer Super Output Area (LSOA) or ward. Where HODH data was unavailable, data was provided independently by the police force.

For evaluations using the DODO model, crime data was limited to most forms of violence and robbery. These crimes are listed at the end of this Appendix. Note that there may be some difference in the exact crime categories used in the independent evaluations included in this report and data cuts provided by the police force areas (PFAs).

The HO merged the processed activity and crime data to generate a record of treatment and crime count for each area and day.

There are instances where the offence location occurred directly on the boundary of a hot spot. This offence could be counted both in the hot spot and outside of it and could theoretically have led to double counting. This could have caused problems if the boundaries of hot spots were touching and therefore an offence could be counted in 2 hot spots. This was uncommon, as most forces left buffers around their hot spots but did occur in a few locations.

2) Data processing and filters

This report presents results with and without filters.

For the set of results with filters, 2 types of filters were applied to address operational challenges such as ramp-up periods, changes in experimental design and implementation (with periods of low or constant visitation) as well as low-crime areas.

The activity filter was designed to limit analysis to an appropriate period for each unit. The activity filter tested if a treated day was within a window between the first and last visited date extending 28 days (inclusive), forwards and backwards, which contained a minimum of 4 control days and 4 treatment days. Note: the 4 control and treatment days did not have to come from the same 28-day window. The HO then filtered the activity time series for the earliest and latest visited dates which met the above conditions, defining the analytical period. In addition, to be included in the filtered model, the analytical period associated with a unit had to be at least 60 days and contain at least 10 control days and 10 Grip patrol days.

The zero-crime day filter was designed to remove areas with a high proportion of days with zero crimes. This was due to their appropriateness as hot spots, and the ability to identify a treatment effect (Hinkle et al., 2013). Zero-crime days may be present because the area was poorly selected or because of intelligence that indicated the presence of organised crime, which may justify its visitation even if its recorded crime counts are suppressed/unreliable. The zero-crime day filter removed units with over 90% zero-crime days in their analytical period.

For the unfiltered model, the only restriction applied was to limit the period analysed for each unit between their first and last visited date. This was to mitigate contamination from other activities/operations.

3) DODO regression model

The HO initially chose the negative binomial model as it has been used extensively in the evaluation of hot spot policing and the wider criminological literature to account for observed overdispersion in crime count data (Braga et al., 2019; Weisburd et al., 2022). Following analytical quality assurance, it was recommended that a Poisson model was used instead of negative binomial, as it has been shown that the fixed effects Poisson model is robust to over dispersion (Wooldridge, 1999). Testing the effect of the model with the current configuration, both the Poisson and Negative Binomial results with cluster robust standard errors were very similar and did not significantly impact the pooled treatment effect and associated confidence intervals (CIs). The negative binomial and Poisson regressions were both implemented in R using the stats and MASS libraries, respectively. Both fixed effect regression models used the following regression design with a log-link function:

crime count = treated + area + month + weekend + area: weekend + 1st month [Eq.1]

Treated is a binary dummy variable indicating whether the area received a Grip patrol. Controls were also included for the area, month, weekend, and area-weekend interaction. The month and weekend controls were introduced to account for seasonality and within week variation in crime rates. The weekend variable is defined here as Friday to Sunday inclusive. The ‘1st of the month’ fixed effect was introduced to account for police-recorded crime recording effects, where an uncertain event date may be assigned to the start of the month. Treatment was assumed to be randomly assigned, conditional on controls such that there was no bias from unaccounted confounders.

4) Crime reduction model

The effect size of visitation for each police force was collated in a spreadsheet model, together with associated p-values and estimate standard errors and CIs. The change in the number of crimes due to visitation was estimated as:

where N_v is the aggregate number of crimes on visited days and ε is the effect size.

Hot spot areas and crimes which were filtered out of the analysis were assumed to have an effect size equal to the mean effect size across all shapefile or ward-level evaluations, as appropriate.

The estimated uncertainty associated with the effect size was propagated through Eq.2 using a Monte Carlo simulation, which randomly sampled the effect size of each police force area from a normal distribution characterised by the standard error estimate associated with the treatment regression coefficient. The central estimate and 95% confidence intervals on ∆_c were then reported.

Serious violence offences included in the regression model:

• assault with injury
• assault with intent to cause serious harm
• assault without injury
• assaults on emergency workers (other than constables)
• attempted murder
• endangering life
• manslaughter
• murder
• racially or religiously aggravated assault with injury
• racially or religiously aggravated assault without injury
• robbery of business property
• robbery of personal property
• threats to kill

Assaults on constables were not included in the crime count as these can only occur if there is a police presence.

Annex B: Non-filtered results

Poisson:
Effect size (significant): -5.5% (CI: -1.6% to -8.7%)
Crimes prevented (significant): 800 (CI: 100 to 1,500)
Benefit cost ratio: £1.59 (CI: £0.20 to £2.98)
Benefits: £26.0 million (range: £3.2 million to £48.7 million)

Negative binomial:
Effect size (significant): -5.1% (CI: -1.2% to -8.5%)
Crimes prevented (significant): 700 (CI: 0 to 1,500)
Benefit cost ratio: £1.39 (CI: £0 to £2.98)
Benefits: £22.7 million (range: £0 to £48.7 million)

Annex C: Force-level delivery descriptions

Avon and Somerset: Avon and Somerset ran 15-minute patrols in 21 main hot spots. The hot spots were circles, approximately 200 metres in diameter, but adjusted slightly for local geographical features. They also had a further 79 ‘Tier 2’ hot spots that also received some additional patrols through Grip. Some of these – those that had received enough to pass through our filters – were included in the crime-impact analysis. The patrols were largely tasked via over time and followed a non-randomised pattern and targeted times of the day when crime was highest. The top 15 hot spots accounted for 5% of all non-domestic violence against the person in the force. No GPS tracking was used and patrols were instead verified by an inspector watching (dip-sampled) footage from body-worn video. Data returns were completed by officers.

Bedfordshire: Bedfordshire ran a randomised controlled trial (RCT) in 22 hot spots from 8 November 2021 to 28 February 2022 and also used Grip funding to separately send patrols into other hot areas. Hot spots were initially Lower Layer Super Output Areas (LSOA) areas but were subsequently re-drawn into smaller defined manual shapes. For the RCT, Bedfordshire randomised by day with 10 areas targeted for patrol each day, half via neighbourhood policing teams and half with armed policing teams. Compliance measured by the force was generally between 60% and 90% for the former but lower for the latter. Bedfordshire used a third-party GPS app to track patrols. Due to data issues, the non-RCT patrols were not included in the crime-impact analysis.

Cleveland: Cleveland received Home Office (HO) funding for hot spot activity from September 2021. They began patrols in December 2021 at 56 hot spots that were 200m-sided hexagons. The patrols were not randomised.

Essex: Essex ran 15-minute patrols into 77 hot spots that were 150m x 150m boxes based on kernel density for violence offences, excluding weapon/drug possession and domestic offences. Essex also used Risk Terrain Modelling in 3 force districts. The hot spots covered between 25% to 40% of crime harm across the districts. Delivery also varied across districts, with some using randomised patrolling and a single unit, and others using non-randomised patrolling and overtime. Compliance could not always be measured but certainly varied across different parts of Essex, according to the force. For tracking, Essex used self-reported patrols and QR codes where the officers scan in when they enter the patrol zone and then enter the activity they undertook.

Greater Manchester Police (GMP): GMP ran an RCT in 3 areas within the force and also funded Grip patrols in hot spots in other areas on a non-randomised basis. The RCT, which ran for 3 months, had 60 hot spots in total, each of which was a group of three 150m x 150m boxes merged together. These were randomised 50/50 into treatment and control groups. The treatment areas received about 20 minutes of additional patrolling on 6 days per week. They also received a ‘contextual safeguarding’ intervention. This involved a leaflet drop and social media campaign with officers surveying young people, business owners and residents about the causes of crime and the victims/offenders in the hot spots, and collecting this into a series of documents to send to partners to suggest solutions. The control areas received just business-as-usual activity. GMP measured compliance using both officer forms and GPS systems, reporting that the latter showed 70% to 77% compliance across the 3 areas whilst diary compliance was over 90%. Note that GMP’s RCT result is the only one in the meta-analysis that compares crime in one set of areas versus crime in control areas. All other results are within-area comparisons of treatment versus control days.

GMP also returned data on other, non-randomised, hot spot patrols at ward level, covering 151 different wards.

Hampshire: Hampshire ran patrols of various lengths into hot spots throughout the year ending March 2022. They started by using LSOA-sized hot spots based on homicide, knife crime and violence with injury offences, excluding domestic/private-space crimes and possession offences. However, for the first 3 quarters (April to December), the data returned to the HO on these patrols was at ward level. This meant that, at the ward level, most areas had at least one additional patrol every day, making analysis very difficult due to the lack of control days. Hampshire switched to providing hot spot-level data (15 x 250m-sided boxes) in quarter 4 (January to March) but this time period was deemed too short for robust results, so Hampshire does not feature in the meta-analysis. Hampshire also generated 3 control areas but did not select them randomly.

Humberside: Humberside received HO funding for hot spot activity from September 2021. They ran patrols from December 2021 into 98 hot spots (150m x 150m) on a randomised basis. Of these 98, Humberside randomly selected 52 to be patrolled for 15 minutes each day, with the remainder receiving no additional patrol that day. The force tracked compliance with a third-party GPS app.

Kent: Kent identified 29 hot spots at ward level for patrols in the year ending March 2022. They did not use GPS tracking and manually completed data returns. Unfortunately, due to data complications, the HO could not include Kent in the crime-impact results. These problems are being rectified and so results for Kent may be available in the year ending March 2023.

Lancashire: Lancashire ran an RCT in 22 hot spots at the LSOA level. Officers were tasked via overtime to patrol in a 200m radius around the centre of the LSOA. However, the force returned data to the HO for most quarters at the ward level so that was used as the unit of analysis in the model (the hot spots were spread across 21 wards). Lancashire randomised by day within areas, with each area aiming for 2 separate 30-minute patrols on treatment days. Lancashire started using manual data returns but moved to a third-party tracking GPS app by the end of the year.

Leicestershire: Leicestershire ran patrols in 31 hot spots that were LSOA level but the hottest output areas/streets within these were also identified to focus patrol activity. Fifteen were subject to delivery by a dedicated uniform visibility trial team, 9 were subject to visible patrolling by neighbourhood officers, and the remainder as ‘control’ areas. They aimed for 15 minutes of patrolling per visit with some visits lasting up to an hour. They could not get a GPS tracking solution running so they manually collected patrols and verified them using dip-samples of body-worn video.

Merseyside: Merseyside ran an RCT and did non-randomised patrols in other areas. The RCT involved 12 hot spots that were bespoke polygons following the patterns of streets and crossroads. Their average size was 0.11km². The target patrol length was 15 minutes and on any one day, 6 of the hot spots received the intervention. The force reported that the average time per visit was 23 minutes (with the median being 17 minutes) and that 94% of the patrols were completed successfully, which was defined as 13+ minutes of patrol. The experiment ran from 15 November 2021 to 27 February 2022, excluding certain days like Christmas and New Year’s Day.

Merseyside aimed the non-random patrols at selected foot-beat hot spots, which often involved longer patrols (several hours). Due to data issues, the HO did not examine these patrols for crime impact.

Metropolitan Police Service (MPS): MPS ran 2 RCTs in the year ending March 2022. Both used 228 hexagon-shaped hot spots, measuring 66m per side. In the first experiment, which ran for 8 months from 26 July 2021, MPS randomly divided the hot spots into 2 groups: a ‘pure control’ group of 58 locations that received no additional resources; and 170 that were randomised by day to either receive up to three 30-minute patrols on treatment days (42% of days) or no patrols on control days.

In the second experiment, which began on 17 January 2022 and ran to 31 March 2022, MPS increased patrolling on treatment days to 150 minutes per day and reduced the number of areas to 72. Of the original 170 areas, MPS selected a further 36 for 150 minutes of patrol every day, whilst the other areas received no additional patrolling.

For both operations, the evaluation meta-analysis included the 2 crossover experiments (so the comparison is within area, comparing days of treatment versus days without). Compliance generally increased throughout the experiment, with MPS reporting 80% to 95% of the desired patrol time each day in the final 3 months, independently verified using a reliable system of GPS tracking. This system also allowed estimates for how much additional presence Grip provided. Whilst this was significant – around 171 officer-minutes per treated day in the first operation and 292 in the second – there was already considerable business-as-usual presence in the hot spots. This amounted to 311 minutes in the first operation and 222 in the second, meaning Grip added about 54% and 132% of presence, respectively.

MPS also reported that the 228 participating hot spots made up 0.22% of the geographical surface area of London but produced 4.2% of all offences and 3.4% of all violent crimes between 26 July 2021 and 31 March 2022.

Northumbria: Northumbria ran an RCT in 18 hot spots in South Tyneside and regular non-randomised hot spot patrols in a further 15 hot spots across the rest of the force area. The RCT ran from 12 July 2021 to 31 March 2022. The force randomly selected 8 of the 150m x 150m hot spots to be patrolled each day for 15 to 20 minutes.

The 15 other hot spot areas, which were foot-beat size, were initially patrolled 3 days per week for a minimum of one hour and 15 minutes each treatment day. This ran from 12 July to 30 September 2021, at which point Northumbria increased the patrol schedule to every day (varying between daytime and evening patrols). The force reported compliance as 93% across all the hot spots, tracked by a third-party GPS app.

Nottinghamshire: Nottinghamshire used crime harm and crime count data going back 2 years to select the top 30 beats out of 213. Of these 30, 15 were quasi-randomly selected to be included in the experiment by police commanders together deciding which to target based on available resources. The hot spots represented about 7% of the force area and accounted for about 45% of crime. The Nottinghamshire hot spots were the standard police beats of varying size but up to several square kilometres. Nottinghamshire had a longer duration of tasked patrol, as their hot spots were much larger than the traditional micro hot spots. The task was 2-hour patrols with 2 officers. Times were 11:00 hours to 14:00 hours and 15:00 hours to 18:00 hours, but the officers did not undertake the patrols at set times and varied them as much as possible. Nottinghamshire opted for a mixture of overtime hours and incorporation of patrols into regular patterns. They did not use GPS tracking so officer-completed returns provided the data.

South Wales: South Wales identified 20 hot spots in the year ending March 2022 at the LSOA level, but they returned data to the HO at ward level. These reflected both crime data (particularly knife crime) and intelligence. South Wales used some overtime for the patrols but also used Grip funding to pay for a specialist team.

South Yorkshire: South Yorkshire ran an RCT in 15 (later increased to 18) hot spots that were circular (131m). They randomised on a day basis, with one hot spot per day randomly selected not to be patrolled per day, meaning that the hot spots were patrolled on average about 14 in every 15 days for 30 minutes. South Yorkshire used off-the-shelf GPS trackers and achieved an 84% compliance rate, which increased through the year (hitting 99% in quarter 3 (October to December) and quarter 4 (January to March) according to the force).

Sussex: Sussex ran patrols into 28 hot spots on a non-random basis for most of the year ending March 2022 and also began an RCT in quarter 4 (January to March). Because this ran into the following year, the HO will evaluate the data for the next iteration of this report. The patrols during the year ending March 2022 were 15 to 20 minutes and focused around 150m-sided boxes, although the actual areas patrolled were sometimes slightly larger than this. Sussex sometimes returned data to the HO at ward level (hot spots were present in 15 different wards). They tracked the patrols with a third-party GPS app. The 28 hot spots covered less than 1% of the force area but 9.7% of serious violent crime, according to the force. They did not measure compliance centrally but it increased from 39% to 66% in quarter 1 (April to June) to over 80% in quarter 4 (January to March), according to the force.

Thames Valley Police (TVP): TVP ran an RCT across 45 hot spots from 8 November 2021 to 31 March 2022. These were 150m-sided hexagons, split between 19 patrolled on the day shift (because analysis showed high concentration of violence during daytime hours) and 26 patrolled on the night shift (because more crime at these hot spots occurred in the evenings). Patrols were randomised by day, with half the hot spots randomly selected for treatment (15 minutes of patrol) each day. TVP delivered patrols via the Joint Operations Unit. TVP tracked patrols using GPS airwave pings, and they also developed their own tasking app, which showed the officers the randomly selected hot spots to patrol each day. Whilst this was extremely innovative, it gave officers an element of choice of which treatment hot spots to patrol on any given day, resulting in variable compliance across hot spots.

West Midlands: West Midlands identified 39 ‘impact areas’, 19 of which they adopted, representing 4.4% of the total area of the force. The force divided these into hexagons with 150m sides, measured crime by hexagon and combined this data with professional judgement to further focus patrol activity. They did not implement GPS tracking. They collected data manually by area and then collated centrally and recorded patrols at the ward level. The manual element meant data quality was poor and hence it was mutually decided that the HO would not assess a crime-impact result for West Midlands. The force has implemented a new data collection system for the year ending March 2023 so a result should be included in the next iteration of this evaluation report.

West Yorkshire Police (WYP): WYP ran patrols in 24 hot spots that were mostly 250m-sided hexagons, although 6 were much larger (5 hexagons joined together). They also funded patrols into other hot spot areas. Although patrols in the 24 areas were of random design, only a low number of patrol days were scheduled per month. In addition, it was problematic to separate additional Grip patrols from business-as-usual activity meaning, so the HO did not calculate results for WYP. This issue has now been resolved and results will be forthcoming in the year ending March 2023.

Annex D: Alternative meta-analysis

The main overall result reported in the publication is using the mean-average of the effect size across each evaluation. This implicitly weights each evaluation equally. An alternative approach in the meta-analysis literature weights the evaluations by the inverse variance with random effects. The table below summarises the results of this approach, showing a statistically significant reduction in the weighted average result across all model variants.

Overall pooling	NegBin filter	Poisson filter	NegBin no filters	Poisson no filters
Effect size	-4.3%	-4.4%	-3.6%	-3.8%
CI-Lower	-6.8%	-6.8%	-5.9%	-6.2%
CI-Higher	-1.7%	-1.9%	-1.2%	-1.4%
Significance	<0.01 ***	<0.01 ***	<0.01 ***	<0.01 ***

One important limitation of the inverse variance methodology in this evaluation is that, generally the larger ward-level evaluations are assigned substantially more weight than the bespoke shapefile evaluations. As discussed previously, ward-level evaluations had sub-ward patrols, but data were only available at the ward-level. Due to the reduced granularity of the analysis, the ward-level evaluations generally constitute the less robust set of evaluations, and therefore the effect sizes are expected to be diluted. Consequently, the application of the inverse variance random effects method presented above generally assigns higher weight to less robust studies. In response to this limitation, the results of the inverse variance-weighted meta-analysis separately for shapefile and ward-level forces are also presented below.

For each model variant, the shapefile force evaluations have statistical significance at the 0.05 level. For the pooled ward-level force evaluations, the filtered model variants have statistical significance at the 0.1 level (but not at the 0.05 level) and statistical significance at the 0.05 level for the no-filter model variants.

Shapefile pooling	NegBin filter	Poisson filter	NegBin no filters	Poisson no filters
Effect size	-6.6%	-6.5%	-4.4%	-4.7%
CI-Lower	-10.7%	-10.5%	-8.3%	-8.8%
CI-Higher	-2.2%	-2.4%	-0.2%	-0.4%
Significance	<0.01 ***	<0.01 ***	<0.05 **	<0.05 **

Ward-level pooling	NegBin filter	Poisson filter	NegBin no filters	Poisson no filters
Effect size	-3.0%	-3.3%	-3.2%	-3.4%
CI-Lower	-6.2%	-6.7%	-6.2%	-6.2%
CI-Higher	0.2%	0.2%	-0.2%	-0.5%
Significance	<0.1 *	<0.1 *	<0.05 **	<0.05 **

Annex E: Sample size and RCT compliance

Police force	Number of hot spots used in analysis (either filtered or conducted by force)	Number of days (unfiltered)	Number of location days used in analysis (filters applied)	Total location-days treatment	Total location-days control
Police Force 1, Design A (RCT)	18	263	4734	1924	2810
Police Force 2 (RCT)	12	105	1224	532	692
Police Force 3, Design A (RCT)	60	182	14214	3294	10920
Police Force 5 (RCT)	22	113	2486	559	1927
Police Force 6, Design A	21	74	1533	777	756
Police Force 1, Design B	15	261	3045	2097	948
Police Force 4	5	273	1184	947	237
Police Force 7	12	284	2713	2332	381
Police Force 8	11	353	2309	289	2020
Police Force 9 (RCT)	45	128	5760	1965	3795
Police Force 10 (RCT*)	20	267	2997	1130	1867
Police Force 6, Design B (RCT)	61	114	6621	2904	3717
Police Force 11 (RCT*)	26	305	7146	4138	3008
Police Force 12	10	304	2526	1340	1186
Police Force 3, Design B	19	365	3242	826	2416
Police Force 15	20	274	3614	1438	2176
Police Force 14 (RCT)	19	121	2258	1069	1189
Police Force 13	15	121	1590	1235	355
Police Force 16	14	259	2592	1446	1146

Police force	Location-days treatment (intended – full RCTs only)	Location-days intended control – full RCTs only)	Proportion of treatment location days that received any patrol activity (full RCTs only)
Police Force 1, Design A (RCT)	2042	2692	94%
Police Force 2 (RCT)	630	630	84%
Police Force 3, Design A (RCT)	4530	5460	73%
Police Force 5 (RCT)	1130	1356	49%
Police Force 9 (RCT)	2874	2886	67%
Police Force 6, Design B (RCT)	2921	4033	99%
Police Force 14 (RCT)	1220	1079	88%

Notes:

1. Where forces have reported patrols via GPS and self-report, the HO have generally used GPS unless aware that the GPS was picking up non-Grip activity too, in which case self-report was used.
2. The number of hot spots used in analysis may differ from the number of hot spots the force selected to receive treatment.
3. Note that Force 3’s figures were reported by the force; the HO did not receive the underlying data.

References

Andresen, M.A. & Hodgkinson, T.K. (2018) ‘Evaluating the impact of police foot patrol at the micro-geographic level’. Policing: An International Journal, vol. 41(3), pp. 314-324; [accessed 23/05/2023].

Barnes, G.C., Williams, S., Sherman, L.W., Parmar, J., House, P. & Brown, S.A. (2020) Sweet spots of residual deterrence: A randomized crossover experiment in Minimalist police patrol; [accessed 23/05/2023].

Basford, L., Sims, C., Agar, I., Harinam, V. & Strang, H. (2021) ‘Effects of one-a-day foot patrols on hot spots of serious violence and crime harm: A randomised crossover trial’. Cambridge Journal of Evidence-Based Policing, vol. 5(3-4), pp. 119-133; [accessed 23/05/2023].

Bland, M., Leggetter, M., Cestaro, D. & Sebire, J. (2021) ‘Fifteen minutes per day keeps the violence away: A crossover randomised controlled trial on the impact of foot patrols on serious violence in large hot spot areas’. Cambridge Journal of Evidence-Based Policing, vol. 5(3-4), pp. 93-118; [accessed 23/05/2023].

Braga, A.A., Papachristos, A.V. & Hureau, D.M. (2010) ‘The concentration and stability of gun violence at micro places in Boston, 1980–2008’. Journal of Quantitative Criminology, vol. 26, pp. 33-53; [accessed 23/05/2023].

Braga, A., Papachristos, A.V. & Hureau, D.M. (2014) ‘The effects of hot spots policing on crime: An updated systematic review and meta analysis’. Justice Quarterly, vol. 31, pp. 633-663; [accessed 23/05/2023].

Braga, A.A., Turchan, B.S., Papachristos, A.V. & Hureau, D.M. (2019) ‘Hot spots policing and crime reduction: An update of an ongoing systematic review and meta-analysis’. Journal of Experimental Criminology, vol. 15(3), pp. 289-311; [accessed 23/05/2023].

Groff, E.R., Weisburd, D. & Yang, S.-M. (2010) ‘Is it important to examine crime trends at a local “micro” level?: A longitudinal analysis of street to street variability in crime trajectories’. Journal of Quantitative Criminology, vol. 26, pp. 7-32; [accessed 23/05/2023].

Heeks, M., Reed, S., Tafsiri, M. & Prince, S. (2018) The economic and social costs of crime (second edition). Research report 99. London: Home Office; [accessed 22/05/2023].

Hinkle, J.C., Weisburd, D., Famega, C. & Ready, J. (2013) ‘The problem is not just sample size: The consequences of low base rates in policing experiments in smaller cities’. Evaluation Review, vol. 37(3-4), pp. 213-238; [accessed 23/05/2023].

HM Treasury (2023) GDP deflators at market prices, and money GDP; [accessed 22/05/2023].

Home Office (2021) £35.5 million to support young people at risk of involvement in serious violence; [accessed 22/05/2023].

Home Office (2022) Home Office Annual Data Requirement (ADR) data – Privacy Information Notice. Home Office Guidance; [accessed 22/05/2023].

Hutt, O., Bowers, K., Johnson, S. & Davies, T. (2018) ‘Data and evidence challenges facing place-based policing’. Policing: An International Journal, vol. 41(3), pp. 339-351; [accessed 23/05/2023].

Koper, C.S. (1995) ‘Just enough police presence: Reducing crime and disorderly behavior by optimizing patrol time in Crime hot spots’. Justice Quarterly, vol. 12(4), pp. 649-672; [accessed 23/05/2023].

Koper, C.S. (2014) ‘Assessing the practice of hot spots policing’. Journal of Contemporary Criminal Justice, vol. 30(2), pp. 123-146; [accessed 23/05/2023].

Lum, C., Hibdon, J., Cave, B., Koper, C.S. & Merola, L. (2011) ‘License plate reader (LPR) police patrols in crime hot spots: An experimental evaluation in two adjacent jurisdictions’. Journal of Experimental Criminology, vol. 7(4), pp. 321-345; [accessed 23/05/2023].

Sherman, L.W. (2022) ‘“Test-as-you-go” for hot spots policing: Continuous Impact Assessment with repeat crossover designs’. Cambridge Journal of Evidence-Based Policing, vol. 6(1-2), pp. 25-41; [accessed 23/05/2023].

Taylor, B., Koper, C.S. & Woods, D.J. (2010) ‘A randomized controlled trial of different policing strategies at hot spots of violent crime’. Journal of Experimental Criminology, vol. 7(2), pp. 149-181; [accessed 23/05/2023].

Weisburd, D., Wyckoff, L.A., Ready, J., Eck J.E., Hinkle, J.C. & Gajewski, F. (2006) ‘Does crime just move around the corner? A controlled study of spatial displacement and diffusion of crime control benefits’. Criminology, vol. 44(3), pp. 549-592; [accessed 23/05/2023].

Weisburd, D., Telep, C.W., Hinkle, J.C. & Eck, J.E. (2008) ‘The effects of problem‐oriented policing on crime and disorder’. Campbell Systematic Reviews, vol. 4(1), pp. 1-87; [accessed 23/05/2023].

Weisburd, D., Telep, C.W., Vovak, H., Zastrow, T., Braga, A.A. & Turchan, B. (2022) ‘Reforming the police through Procedural Justice training: A multicity randomized trial at crime hot spots’. Proceedings of the National Academy of Sciences, vol. 119(14); [accessed 23/05/2023].

Wooldridge, J.M. (1999) ‘Distribution-free estimation of some nonlinear panel data models’. Journal of Econometrics, vol. 90(1), pp. 77-97; [accessed 23/05/2023].

1. Four forces were not represented in the results due to technological or data issues. Some forces ran multiple evaluations. ↩

2. Braga et. al (2019) finds that hot spot policing programs produced statistically significant (p < 0.05) positive mean effect sizes for violent crime outcomes (0.102), property crime outcomes (0.124), disorder outcomes (0.161), and drug crime outcomes (0.244). ↩

3. The 20 forces involved account for over 80% of hospital admissions for assault with a sharp object in England and Wales. ↩

4. LSOAs are a statistical geography which comprise between 400 and 1,200 households and have a usually resident population between 1,000 and 3,000 persons. ↩

5. OAs are the lowest level of geographical area for census statistics which comprise between 40 and 250 households and had a usually resident population between 100 and 625 persons. ↩

6. R is a programming language for statistical computing and graphics. ↩

7. All these numbers are rounded to one decimal place. ↩

8. Note also that one of the 19 estimates represents the percentage reduction compared with randomly selected control areas rather than with control days within the same area. ↩