Make your next car electric: a behavioural science messaging trial
Published 1 February 2021
© Crown copyright 2021
This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: psi@nationalarchives.gov.uk.
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at https://www.gov.uk/government/publications/behavioural-science-messaging-trial-make-your-next-car-electric/make-your-next-car-electric-a-behavioural-science-messaging-trial
Introduction
The Department for Transport (DfT) worked with the Driver and Vehicle Licensing Agency and the Cabinet Office to run a behavioural trial to investigate messaging to best encourage electric vehicle uptake, supporting the government’s net-zero ambitions. The trial commenced on 6 March 2020 and ran for 130 days, taking place during the government’s consultation on bringing forward the phase-out date for the sale of new petrol, diesel and hybrid cars and vans (February to July 2020).
The trial targeted people who had just renewed their car tax online by implementing a message that encouraged users to click through to the Go Ultra Low (GUL) website, which encourages electric vehicle purchase.
A total of 8 messages (7 test messages and a control) were compared. Messages were developed by DfT’s in-house behavioural scientists, and policy and communications colleagues from the Office for Low Emission Vehicles (OLEV). Content ranged from cost savings to environmental and air pollution benefits. Messages were compared to see which produced the highest click-through rate to GUL.
The most successful message related to the government’s consultation on ending the sale of internal combustion engine vehicles:
The Government are consulting on ending the sale of new petrol, diesel and hybrid cars and vans by 2035 or earlier. Are you ready? Make your next car electric.
The next most effective message was one that appealed to social norms:
Join the 6,000 new drivers every month who make the switch to an electric vehicle. Make your next car electric.
The message around the number of deaths related to air pollution also performed well, but to a lesser extent than the previous 2:
Between 28,000 and 36,000 people die every year as a result of air pollution. What you drive makes a difference. Make your next car electric.
All messages, including the control, contributed to increased traffic to GUL. Scaling the findings from this study up, it is estimated that:
- the control message (‘Information is available on electric vehicles. Make your next car electric.’) would produce about 350,300 annual click-throughs to GUL
- that ‘The Government are consulting on ending the sale of new petrol, diesel and hybrid cars and vans by 2035 or earlier. Are you ready? Make your next car electric.’ would result in around 419,900 annual click-throughs to GUL
- that ‘Join the 6,000 new drivers every month who make the switch to an electric vehicle. Make your next car electric.’ would generate around 407,900 more annual click-throughs to GUL
Three messages performed worse than the control and were therefore unsuccessful. The lowest click-through rate was produced by the message around rapid charge point availability:
Rapid charge points for electric vehicles are available at almost all motorway service stations in the UK. Make your next car electric.
The second-lowest click-through rate was produced by the message around the ease of charging:
Charging your electric vehicle at home can be as easy as charging your phone overnight. Make your next car electric
The third unsuccessful message (although to a lesser extent) was around the cost per mile of using an electric vehicle:
Fully electric vehicles could cost from as little as 1p per mile to run – less than a quarter of the cost of the most fuel-efficient petrol or diesel vehicles. Make your next car electric.
The message around traffic and carbon emissions performed similarly to the control: ‘Road traffic is the biggest single contributor to carbon emissions in the UK. What you drive makes a difference. Make your next car electric.’
COVID-19 impact
Click-through rates of each message was compared from pre-lockdown to post-lockdown. Overall, patterns in the trial are similar across both periods. However, during lockdown, click-through rates for all messages decreased significantly by an average of 14%.
This suggests that people were less receptive to all messages during this period. ‘Are you ready?’ continued to have the best click-through rate overall. ‘Are you ready?’ and ‘6,000 new drivers’ performed better than the control pre- and post-lockdown. ‘28,000 to 36,000 deaths’ only performed better than the control post-lockdown. The success of messages varied through lockdown, highlighting the importance of message context. It is therefore important to continue developing and testing messages to ensure we apply messages correctly.
Messages
Message background
The messages that were tested were developed by DfT’s in-house behavioural scientists, and policy and communications colleagues from the OLEV. The content of the messages ranged from cost savings to environmental and air pollution concerns.
The messages were hosted on the GOV.UK Tax your vehicle service. After successfully renewing car tax online, users were presented with a ‘Thank you’ page. A single message was displayed randomly to each user on this page and all linked to GUL.
Message content, summary click-through rates and shorthand
Table 1: message content, click-through rates and shorthand descriptor
|
Message |
Content |
Click through rate |
Shorthand |
|---|---|---|---|
| Test E |
The Government are consulting on ending the sale of new petrol, diesel and hybrid cars and vans by 2035 or earlier. Are you ready? Make your next car electric |
3.50% |
‘Are you ready?’ |
| Test B |
Join the 6,000 new drivers every month who make the switch to an electric vehicle. Make your next car electric. |
3.40% |
‘6,000 new drivers’ |
| Test D |
Between 28,000 and 36,000 people die every year as a result of air pollution. What you drive makes a difference. Make your next car electric. |
3.10% |
‘28,000 to 36,000 deaths’ |
| Test C |
Road traffic is the biggest single contributor to carbon emissions in the UK. What you drive makes a difference. Make your next car electric. |
2.93% |
‘Traffic and carbon emissions’ |
|
Control |
Information is available on electric vehicles. Make your next car electric. |
2.92% |
Control |
| Test A |
Fully electric vehicles could cost from as little as 1p per mile to run – less than a quarter of the cost of the most fuel-efficient petrol or diesel vehicles. Make your next car electric. |
2.80% |
‘1p per mile’ |
| Test F |
Charging your electric vehicle at home can be as easy as charging your phone overnight. Make your next car electric. |
2.74% |
‘Ease of charge’ |
| Test G |
Rapid charge points for electric vehicles are available at almost all motorway service stations in the UK. Make your next car electric. |
2.36% |
‘Charge point availability’ |
Results and findings
Which message was most effective?
Click-through rates were compared to identify which message produced the highest click-through rate overall and which had performed significantly better than the control and other messages.
Comparisons were made using data over 130 days (6 March to 14 July 2020). In total, there were 4,272,526 tracked users of the government website for paying vehicle tax. On average, there were 534,066 users per condition/message.
Table A in the annex presents a breakdown of how many users saw each test message, how many clicked through, and what the click-through rate was for each message. Chart 1 highlights the click-through rates of each message compared with the control.
Chart 1: percentage of visitors to the government vehicle tax payment website that clicked through to GUL for each message for the whole trial (130 days from 6 March to 14 July 2020) (%). ‘Are you ready?’ performed best.
| Message | Click-through rate (%) |
|---|---|
| Control | 2.92 |
| Charge point availability | 2.36*** |
| Ease of charge | 2.74*** |
| 1p per mile | 2.8** |
| Traffic and carbon emissions | 2.93 |
| 28,000 to 36,000 deaths | 3.1*** |
| 6,000 new drivers | 3.4*** |
| Are you ready? | 3.5*** |
In the chart and data table above, N=4,272,526. ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Note
For each chart and table in this guidance:
- where N is reported, N represents the number of participants
- Z-test is a test of the probability that a difference between two groups means (averages) is a chance (false positive) result
- a p-value indicates the probability out of 1 that a result is due to chance, a significance level indicates the maximum probability that a result is due to chance
Effectiveness of messages over the entire trial (130 days)
Table B in the annex shows a breakdown of comparisons made. ‘Are you ready?’ produced the highest click-through rate overall and was significantly higher than all other messages. ‘6,000 new drivers’ was the next most effective message, and ‘28,000 to 36,000 deaths’ produced the third-highest click-through rate. Both were significantly higher than the control message.
Three messages resulted in significantly fewer click-throughs than the control: ‘Charge point availability’ produced the lowest click-through rate, then ‘Ease of charge’ followed by ‘1p per mile’.
Assessing impact
Over the 130 days that this trial ran for, there were 4,272,526 users of the Tax your vehicle service. This means, an average of around 32,800 users a day. Using this value, we can anticipate around 12 million users a year.
By using the anticipated number of users of the Tax your vehicle service, the click-through rate for the different messages can be used to estimate the potential increase in click-throughs to GUL over a year.
The estimates are:
- control (2.92%), about 350,200 click-throughs
- ‘Are you ready?’ (3.50%), about 419,900 click-throughs
- ‘6,000 new drivers’ (3.40%), about 407,900 click-throughs
- ‘28,000 to 36,000 deaths’ (3.10%), about 371,900 click-throughs
Did COVID-19 lockdown measures impact results?
Lockdown measures put in place to protect the public from COVID-19 may have impacted public attitudes and click-through behaviour.
Further analysis was conducted to explore whether this affected the message testing results by comparing which message generated the highest click-through rate pre-lockdown (6 to 23 March 2020; 17 days) and which generated the highest click-through rate post-lockdown (24 March to 14 July 2020; 113 days).
Chart 2: comparison of click-through rates pre- and post-lockdown for each message (130 days from 6 March to 14 July 2020). Generally, post-lockdown click-through rates were statistically lower than pre-lockdown.
| Message | Pre-lockdown | Post-lockdown |
|---|---|---|
| Are you ready? | 4.06 | 3.4*** |
| 6,000 new drivers | 3.64 | 3.35*** |
| 28,000 to 36,000 deaths | 3.41 | 3.02*** |
| Traffic and carbon emissions | 3.42 | 2.92*** |
| Control | 3.32 | 2.86*** |
| 1p per mile | 3.22 | 2.76*** |
| Ease of charge | 3.25 | 2.68*** |
| Charge point availability | 2.73 | 2.3*** |
In the chart and data table above, pre-lockdown N= 440,269; post-lockdown N=3,828,349. ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Overall, lockdown reduced the click-through rate of all messages by an average raw percentage point of 0.47%. This is an average proportional decrease of 14% and was significant for all messages (see chart 2). ‘Are you ready?’ decreased the most from pre- to post-lockdown (4.06% to 3.40%), and ‘6,000 new drivers’ the least (3.64% to 3.35%) (see table C in the Annex).
See tables D to G in the annex for comparisons and breakdown of how many users saw each test message, how many clicked through, and what the click-through rate for each message was pre- and post-lockdown. See chart 3 and chart 4 for how each message performed against the control.
Pre-lockdown, ‘Are you ready?’ performed significantly better than the control and the rest of the messages. ‘6,000 new drivers’ was also effective and performed significantly better than the control but to a lesser extent. No other messages were significantly better than the control.
Chart 3: percentage of visitors to the government vehicle tax payment website that clicked through to GUL for each message pre-lockdown (%) (17 days between 6 to 23 March 2020). ‘Are you ready?’ performed best.
| Click-through rate (%) | |
|---|---|
| Control | 3.32 |
| Charge point availability | 2.73*** |
| Ease of charge | 3.22 |
| 1p per mile | 3.25 |
| Traffic and carbon emissions | 3.41 |
| 28,000 to 36,000 deaths | 3.42 |
| 6,000 new drivers | 3.64** |
| Are you ready? | 4.06*** |
In the chart and data table above, N=440,269. ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Post-lockdown, the top-performing messages were ‘Are you ready?’, ‘6,000 new drivers’ and ‘28,000 to 36,000 deaths’. ‘Are you ready?’ and ‘6,000 new drivers’ both performed similarly to one another and were significantly better than the control. ‘28,000 to 36,000 deaths’ was significantly worse than the 2 top-performing messages, but still performed better than the control.
Chart 4: percentage of visitors to the government vehicle tax payment website that clicked through to GUL for each message post-lockdown (%) (113 days from 24 March to 14 July 2020). ‘Are you ready?’ and ‘6,000 new drivers’ performed best.
| Click-through rate (%) | |
|---|---|
| Control | 2.86 |
| Charge point availability | 2.3*** |
| Ease of charge | 2.68*** |
| 1p per mile | 2.76** |
| Traffic and carbon emissions | 2.92 |
| 28,000 to 36,000 deaths | 3.02*** |
| 6,000 new drivers | 3.35*** |
| Are you ready? | 3.4*** |
In the chart and data table above, N=3,828,349. ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Annex
Cookie compliance policy
In accordance with General Data Protection Regulations (GDPR), GOV.UK has a cookie compliance policy. For this reason, we are unable to track all users to GOV.UK. This means numbers from this trial are likely to be lower than the total number of visits to the Tax your vehicle service.
Unique page views, click-throughs to GUL and click-through rates were tracked using Google Analytics. Due to the way data is processed and stored in Google Analytics, numbers are often rounded up or down. This causes minor inaccuracies in unique views and click-throughs when evaluating different data periods. This results in varying total page views in our analysis. Click-through rate is unaffected by this.
Tables
Table A: breakdown of unique views to the government tax payment website, click-throughs to GUL and click-through rate for each message for the entire trial (130 days from 6 March to 14 July 2020)
| Message | Unique views of the Tax your vehicle page | Click-throughs to GUL | Click-through rate (%) |
|---|---|---|---|
| A | 535,918 | 15,019 | 2.8 |
| B | 531,965 | 18,081 | 3.4 |
| C | 533,830 | 15,624 | 2.93 |
| D | 535,189 | 16,597 | 3.1 |
| E | 535,575 | 18,729 | 3.5 |
| F | 533,574 | 14,602 | 2.74 |
| G | 536,729 | 12,669 | 2.36 |
| Control | 529,746 | 15,493 | 2.92 |
Table B: percentage point difference in click-through rate between Control, Message E and other trial messages (130 days from 6 March to 14 July 2020)
| Message | Control | Message B | Message D |
|---|---|---|---|
| A | -0.12 *** | -0.60 *** | -0.30 *** |
| B | + 0.48 *** | n/a | +0.30 *** |
| C | +0.01 (p=0.950) | -0.47 *** | -0.17 *** |
| D | +0.18 *** | -0.30 *** | n/a |
| E | +0.58 *** | +0.10 ** | +0.40 *** |
| F | -0.18 *** | -0.66 *** | -0.36 ** |
| G | -0.56% *** | -1.04 *** | -0.74 *** |
| Control | n/a | -0.48 *** | -0.18 *** |
In the table above, ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Table C: comparisons of click-through rates pre-lockdown versus post-lockdown for each message (130 days from 6 March to 14 July 2020)
| Message | Pre-lockdown click-through rate (%) | Post-lockdown click-through rate (%) | Proportional decrease (%) |
|---|---|---|---|
| A | 3.22 | 2.76 | 14.29 |
| B | 3.64 | 3.35 | 7.97 |
| C | 3.42 | 2.92 | 14.62 |
| D | 3.41 | 3.02 | 11.44 |
| E | 4.06 | 3.4 | 16.26 |
| F | 3.25 | 2.68 | 17.53 |
| G | 2.73 | 2.3 | 15.75 |
| Control | 3.32 | 2.86 | 13.86 |
Table D: breakdown of unique views of the government tax payment website, click-throughs to GUL and click-through rate for each message pre-lockdown (17 days from 6 to 23 March 2020)
| Message | Unique views of the Tax your vehicle page | Click-throughs to GUL | Click-through rate (%) |
|---|---|---|---|
| A | 55,110 | 1,772 | 3.22 |
| B | 54,959 | 1,999 | 3.64 |
| C | 55,037 | 1,880 | 3.42 |
| D | 55,261 | 1,882 | 3.41 |
| E | 55,449 | 2,252 | 4.06 |
| F | 54,563 | 1,775 | 3.25 |
| G | 55,238 | 1,509 | 2.73 |
| Control | 54,652 | 1,817 | 3.32 |
Table E: percentage point difference in click-through rate between Control, Message E and other trial messages (17 days from 6 to 23 March 2020)
| Message | Control | Message E |
|---|---|---|
| A | - 0.10 (p=0.310) | -0.84 *** |
| B | + 0.32 ** | -0.42 *** |
| C | +0.10 (p=0.400) | -0.64 *** |
| D | +0.09 (p=0.460) | -0.65 *** |
| E | +0.74 *** | n/a |
| F | -0.07 (p=0.510) | -0.81 *** |
| G | -0.59 *** | -1.33 *** |
| Control | n/a | -0.74 *** |
In the table above, ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.
Table F: breakdown of unique views to the government tax payment website, click-throughs to GUL and click-through rate for each message post-lockdown (113 days from 24 March to 14 July 2020)
| Message | Unique views of the tax payment website | Click-throughs to GUL | Click-through rate (%) |
|---|---|---|---|
| A | 479,276 | 13,225 | 2.76 |
| B | 476,394 | 15,954 | 3.35 |
| C | 479,113 | 13,988 | 2.92 |
| D | 479,055 | 14,461 | 3.02 |
| E | 479,991 | 16,312 | 3.40 |
| F | 477,909 | 12,794 | 2.68 |
| G | 480,580 | 11,043 | 2.30 |
| Control | 476,031 | 13,630 | 2.86 |
Table G: percentage point difference in click-through rate between Control, Message B, Message E and other trial messages (113 days from 24 March to 14 July 2020)
| Message | Control | Message B | Message D | Message E |
|---|---|---|---|---|
| A | -0.10** | -0.59*** | -0.26*** | -0.64*** |
| B | + 0.49*** | n/a | +0.33*** | -0.05 (p=0.180) |
| C | +0.06 (p=0.100) | -0.43*** | -0.10** | -0.48*** |
| D | +0.16*** | -0.33*** | n/a | -0.38*** |
| E | +0.54*** | +0.05 (p=0.180) | +0.38*** | n/a |
| F | -0.20*** | -0.67*** | -0.34*** | -0.72*** |
| G | -0.56*** | -1.05*** | -0.72*** | -1.1*** |
| Control | n/a | -0.49*** | -0.16*** | -0.50*** |
In the table above, ** indicates significant difference at the 1% level, *** indicates significant difference at the 0.1% level.