Case-control study: comparative studies
How to use a case-control study to evaluate your digital health product.
This page is part of a collection of guidance on evaluating digital health products.
A case-control study is a type of observational study. It looks at 2 sets of participants. One group has the condition you are interested in (the cases) and one group does not have it (the controls).
In other respects, the participants in both groups are similar. You can then look at a particular factor that might have caused the condition, such as your digital product, and compare participants from the 2 groups in relation to that.
A case-control study is an observational study because you observe the effects on existing groups rather than designing an experiment where participants are allocated into different groups.
What to use it for
A case-control study can help you to find out if your digital product or service achieves its aims, so it can be useful when you have developed your product (summative evaluation).
It can be a useful method when it would be difficult or impossible to randomise participants, for example, if your product aims to help people with rare health conditions.
Pros
Case-control studies have many benefits.
They can:
- help to estimate the effects of your digital product when randomisation is not possible
- use existing data, which could be cheaper and easier
- operate with fewer participants compared to other designs
Cons
There can also be drawbacks of a case-control study.
For example:
- you need to pay careful attention to factors that may influence your results, confounding factors and biases – see explanation in ‘How to carry out a case-control study’ below
- there may be challenges when accessing pre-existing data
- you cannot draw definitive answers about the effects of your product as you haven’t randomly selected participants for your evaluation
How to carry out a case-control study
In a traditional case-control design, cases and controls are looked at retrospectively – that is, the health condition and the factor that might have caused it have already occurred when you start the study.
Sources of cases and controls typically include:
- routinely collected data at medical facilities
- disease registries
- cross-sectional surveys
Some researchers use the term prospective case-control study when, for example, a prospective group exposed to an intervention is compared to a retrospective control.
Choosing your control
Selecting an appropriate control is an important part of a case-control study. The comparison group should be as similar as possible to the source population that produced the cases. This means the participants will be similar to each other in terms of factors that may influence the outcomes you’re looking at. Ideally, they will only differ in whether they received your digital product (cases) or not (controls).
There are 2 main types of case-control design: matched and unmatched.
Essentially, in an unmatched case-control design, a shared control group is selected for all cases at random given certain attributes. In a matched case-control design, controls are selected case-by-case based on specified characteristics. You should pick characteristics that have an effect on the usage of digital devices and services.
Commonly used matching factors include:
- age
- sex
- socio-economic status
However, think about other characteristics and attributes that might influence the use of your product, and the subsequent outcomes.
Confounding variables and biases
Confounding variables (variables other than the one you are interested in that may influence the results) and biases (errors that influence the sample selected and results observed) are important to consider when conducting any research. This is especially important in designs that are non-randomised.
For example:
- selection bias can happen when participants are assigned without randomisation
- attribution bias may occur when patients with unfavourable outcomes are less likely to attend follow-ups
Analysing your data
The analysis most commonly used in case-control studies is an odds ratio, which is the chance (odds) of the outcomes occurring in the case group versus the control group.
Example: Can telemedicine help with post-bariatric surgery care? A case-control design
In 2019, Wang and colleagues published a paper entitled Exploring the Effects of Telemedicine on Bariatric Surgery Follow-up: a Matched Case Control Study.
The study showed that people who go through bariatric surgery have better outcomes if they attend their follow-up appointments after surgery in comparison to those who do not. However, attending appointments can be challenging for people who live in remote areas. In Ontario, Canada, telemedicine suites were set up to enable healthcare provider-patient videoconferencing.
The researchers used a matched case-control study to investigate if telemedicine videoconferencing can support post-surgery appointment attendance rates in people who live further away from the hospital sites. They used the existing data from the bariatric surgery hospital programme to identify eligible patients.
All patients attending the bariatric surgery were offered telemedicine services. The cases were the participants who used telemedicine services; they were compared to those who did not (the controls).
Cases and controls were matched on various characteristics, specifically:
- gender
- age
- time since bariatric surgery
- body mass index (BMI)
- travel distance from the hospital site
Researchers measured:
- the percentage of appointments attended
- rates of dropout
- pre-and post-surgery weight and BMI
- various physical and psychological outcomes
They also calculated rurality index to classify patients into urban, non-urban and rural areas. These variables were used to compare cases (those who used telemedicine) and controls (those who did not).
During the study period, they identified that 487 patients of 1,262 who received bariatric surgery used telemedicine services. Of those, 192 agreed to participate in the study.
They found that patients who used telemedicine did as well as patients who attended in person, both in terms of appointment attendance rates and in terms of physical and psychological outcomes.
Moreover, the researchers found that the cases (telemedicine users) came from more rural areas than the controls. The authors argued that this demonstrated that telemedicine can help overcome the known challenges for patients in more rural areas to attend appointments.
Randomising patients to telemedicine or withdrawing the telemedicine would be difficult, undesirable and possibly unethical. Case-control was a good alternative to assess the potential impact on patient outcomes in a service that is already up and running.
More information and resources
A 2003 study by Mann provides an accessible overview of observational research methods, including an explanation of biases and confounding variables.
On the website for Strengthening the Reporting of Observational Studies in Epidemiology (STROBE), there is a checklist of items that should be included in reports of case-control studies.
A 2016 study by Pearce offers considerations for the analysis of a matched case-control study.
Examples of case-control studies in digital health
In a 2020 study by Heuvel and others, researchers assessed a new digital health tool to monitor women at increased risk of preeclampsia at home. They investigated if the digital tool allows for fewer antenatal visits without compromising women’s safety, and whether it positively affects pregnancy outcomes. This study used a prospective case group compared to a retrospective control group.
In a 2019 study by Depp and others, the research team examined whether schizophrenia symptoms were associated with mobility (measured using GPS sensors). They compared participants with schizophrenia to healthy controls and they found that less mobility was associated with greater symptoms of schizophrenia.