Measuring completion rate

From:: Performance analysis community

Your service’s completion rate is the number of digital transactions that your users complete as a percentage of all digital transactions that your users start.

This includes transactions where the user receives support from someone to use the digital service (called ‘assisted digital support’).

Calculating completion rate

To calculate your completion rate:

Count the number of completed transactions - the numerator.
Divide it by the total number of transactions (including partially completed or failed ones) - the denominator.
Show the result as a percentage.

Make sure you count only genuine users, ie set up your analytics tools to exclude internal users, test users and web robots from your data.

What to include as a transaction

A transaction is an exchange of:

money
information
permission
goods or services

Transactions on your service will have clearly defined start and end points. They will also result in a change to a government system, eg someone’s personal information or the details of an interaction are stored in a register or database.

The following activities do not have clearly defined start and end points and are not recorded in a government system, so they are not transactions:

general advice or enquiries
informal complaints
visits to websites

When a transaction is completed

A transaction is completed when someone finishes the task that your digital service provides. For example, when a user:

submits an application
makes a payment
makes a booking

The service should tell the user that the transaction is complete with a confirmation or ‘thank you’ message.

A user’s application does not have to be successful for you to consider the transaction complete.

For example, the Apply for a patent transaction is considered complete when the user has submitted an application, regardless of whether the application is successful.

When a transaction has failed

A failed transaction is when a user gives up before reaching a defined end page (often called a ‘confirmation page’). This includes when a user abandons the service from an error page.

Configure your service to measure completion rate

To make it easier to analyse user behaviour give each step in your service a unique URL so you can identify them in your analytics tool.

Measure from the start of a transaction

Most services will have a start page on GOV.UK, which should be the only way to access the service. You should usually only consider a transaction to have started when the user continues from your service’s start page.

Make sure users are not able to bypass the start page via links or search engine results.

Some services do not have a start page. In these cases it helps to understand the whole user journey so you can identify the exact points at which the user starts the service. This is usually another page on GOV.UK.

For example, there is no start page if someone wants to book an appointment at a UK consular office. There are links to this service from other pages on GOV.UK, such as the information page on getting married in France. In this case, the page on getting married in France is effectively the start page for booking an appointment at a UK consular office.

Know when a service ends

Services often have multiple end points. You must have appropriate end pages at each of them.

For example, the ‘Check if you can get legal aid’ service has two main end points:

one for those who are eligible (that tells them what to do next)
one for those who are not eligible (that suggests other forms of support)

When a user reaches one of these end pages, you must consider the transaction to have finished.

You’ll have to list all of your end pages in your service assessments.

Counting users who ‘save and return’

If your service allows users to save a transaction and finish it later, make sure you can match transaction starts to their subsequent completions.

For example, in Google Analytics you can use cohort analysis and custom dimensions to track the number of times a user returns to complete a transaction on your service.

Services with both digital and non-digital steps

For legal reasons, some services require users to carry out steps digitally and non-digitally, for example starting and finishing a transaction online, but sending a paper copy of a form in the middle.

To measure the 2 parts of the service as a whole is complicated. Users complete the second part of the service much later, which can lead to misleading completion rates. You can resolve this by using cohort analysis and custom dimensions.

However, if your service does not need to match the 2 digital parts of the user journey, you can treat them as separate transactions and measure completion rates for each.

Counting users who are ineligible for a service

Some services have complex eligibility criteria. It can be helpful to include a series of questions - a smart answer - to test eligibility.

If a user continues to use a service after failing an eligibility test, count them as having started the transaction. For example, people can still apply for Carer’s Allowance even if the eligibility test tells them they’re not entitled to it.

You should not consider users who abandon a service after failing an eligibility test as users who have started a transaction.