Crossover randomised controlled trial: comparative studies
How to use a crossover randomised controlled trial to evaluate your digital health product.
This page is part of a collection of guidance on evaluating digital health products.
A crossover randomised controlled trial (RCT) is a specific type of RCT where you assess 2 or more interventions. In this design, all participants receive all the interventions, but the order in which they get the interventions is randomised.
What to use it for
Using crossover RCTs can help you to find out whether your digital products or services achieve their aims, so they can be useful when you have developed your product (summative evaluation).
A crossover RCT assesses the whole intervention, so it is generally less helpful if you are still developing your product or service. However, you could use it to assess different interventions within your digital product.
Pros
Benefits include:
- you can evaluate 2 or more interventions with all participants
- fewer participants are needed for a crossover trial in comparison to other designs where each participant only receives one intervention
- as with any RCT, it can produce definitive answers because randomisation can show cause and effect
Cons
Drawbacks include:
- the order in which the interventions are introduced may influence the results (called order effects)
- there may be a higher burden on participants because they receive all the interventions evaluated
- some interventions are less appropriate for a crossover RCT – for example, where the effects of the interventions are long-lasting
How to carry out a crossover RCT
In a crossover trial, all participants receive all the interventions but the order in which they receive the interventions (the sequence) is randomised.
For example, if you have 3 interventions, participants will be randomised to one of the 3 sequences (ABC or ACB or BCA). Each stage of the assessment in a crossover trial is called a period. Period 1 is when the first intervention is introduced, period 2 is when participants move to the next intervention in the sequence, and so on.
It’s important to consider that some interventions have a carryover effect. Interventions that can produce longer-lasting effects are not appropriate for crossover trials because the sequence of interventions may impact the results. For example, crossover trials are good for interventions that treat symptoms, but do not work for interventions that cure a condition.
Between the periods, you can introduce a washout period, where participants receive no intervention to let the effects of the previous intervention diminish.
In a crossover trial, participants act as their own control. Their data is analysed by comparing participants to themselves – before the intervention was introduced and after each intervention period – to find out if there was a change in the outcome you are measuring. This makes a crossover RCT potentially more efficient than other RCTs of a similar size. The crossover design removes the variation between participants which exists in a parallel trial where each participant only receives one intervention.
Example: a feasibility trial for a crossover RCT of 2 popular apps to increase physical activity
Researchers wanted to evaluate if 2 very popular apps could help inactive people increase their physical activity. They ran a crossover trial to assess both apps in one study. It was important to check that the study was possible to conduct, so they designed a feasibility trial first.
The participants were assessed on how much physical activity they were already doing before the study started. Then each participant was randomised to a sequence of app use:
- app A (7 Minute workout Challenge) then app B (One You Couch to 5K)
- app B then app A
The researchers needed to think carefully about the length of the app assessment. If each period was too long, participants might drop out in period 2. This could introduce imbalances that would affect the results. However, participants needed enough time to experience the interventions. Researchers chose a 2-week period for each app assessment.
It’s important to consider whether anything would have changed over time regardless of the intervention – called a period effect. The researchers had to find out if the relationship between physical activity and each app was influenced by the period in which the app was assessed. They could then take this into account in the analysis. The feasibility trial had 66 participants. Researchers found a significant difference between baseline and period 1 (after introduction of the first app) on self-reported physical activity, intentions to exercise and confidence to exercise. There was no difference between objective measures of physical activity (measured using a device called an accelerometer).
The team was cautious about interpreting any results with confidence as this was a feasibility study with a small sample size. They concluded that the study was feasible to conduct as the recruitment and follow-up rates were high. The learning from this feasibility study helped to refine the methods for the full-scale trial.
More information and resources
CONSORT 2010 statement: extension to randomised crossover trials. This explains what you should report when describing a crossover trial. It can also be used as a checklist when designing a crossover RCT.
Lally and others (2009): How are habits formed: Modelling habit formation in the real world. The team looked into how long it takes to develop habit for different behaviours. In crossover trials, it is important that the interventions can produce an effect quickly, and that effects are not long-lasting so that there is no carryover effect.