Economic evaluation: evaluation in health and wellbeing
Helping public health practitioners conducting evaluations – using economic evaluation to identify the value gained from an intervention.
Introduction to economic evaluation
Economics is concerned with how limited resources are used to produce goods and services. Resources are scarce because people always want more from them than what is available. This is particularly pertinent in health, so the field of health economics applies economic principles to understand and make decisions about the production of health and health services, specifically through the use of economic evaluation.
Economic evaluation aims to identify the value gained from resources used to implement a policy, programme or intervention. It helps to determine how effectively resources are being used and whether there are better ways of using them. Economic evaluation does this by considering the alternative use of those resources, and the benefits that would have been gained from taking an alternative course of action. This is called the opportunity cost.
For example, if public health resources are used on a scheme to improve walking among the over-60s, then those resources are no longer available to be used to run a football group for primary school children. If we carried out an economic evaluation of the over-60s walking group, not only would we consider the resources used to encourage the participants to walk more (the costs) and the effects on the participants’ physical and mental health (the benefits) but also consider the fact that the resources used are no longer available for other potentially beneficial projects, such as the football group. The value of the benefits from alternative projects are the opportunity costs.
A key feature of economic evaluation is that it makes explicit the perspectives from which costs and benefits are assessed. The perspective informs which costs and outcomes to include in the evaluation. The main perspectives are that of society, health and social services, a specific healthcare provider, and patients and their families.
For example, moving community family planning services to one centralised location is likely to be more resource efficient from the health provider perspective in terms of number of staff needed, staff time, costs of running the building and so on. However, users would have to spend more on travel and transport and possibly take time off work. So it would not be more resource efficient from a patient perspective. In addition, if moving the service to a central location means that people are less likely to use it, and this results in more sexually transmitted infections or unwanted pregnancies, then although cost-savings might be made in the family planning service itself, more resources would be used in other areas of health and social services.
Assessing opportunity costs
Assessing opportunity costs requires assessment of benefits gained from the new programme and the existing service or alternative course of action. How the benefits are measured and whether they are assigned a value to allow for comparison with other types of programmes depends on the type of economic evaluation used. Cost-benefit analysis places a monetary value on benefits, for example, by asking people what they would be willing to pay for the outcomes achieved. If the benefits of 2 alternatives are known to be exactly the same value (and this should be evidenced), then cost-minimisation analysis can be used to compare the programme and the alternative solely in terms of resource use – meaning to find out which costs the least.
In cost-effectiveness analysis, the benefits or outcomes are expressed in natural units but the extent to which they are achieved might differ between the programmes. These benefits are related to the cost on a unit basis, and the programme yielding the highest unit benefit for unit of resource use is the preferred option (for example, cost for each kilogram of weight loss, for each year of life gained).
Cost-utility analysis
Cost-utility analysis is a particular type of cost-effectiveness analysis that is commonly used in the health sector. Here, benefits are measured in terms of quantity and quality of life. Often, these are measured using quality-adjusted life years (QALYs), which multiply years of life by weights (utility values) ranging from 0 (death) to 1 (perfect health) that are derived from people’s preferences for different health states. This approach incorporates both increases in survival time (extra life years) and changes in quality of life (so improvements in health without necessarily increased survival) into one measure. The incremental cost-effective analysis (ICEA) is usually expressed as the incremental or changes in cost to gain an extra QALY.
For example, an increased duration of life of one year (without change in quality of life), or an increase in quality of life from 0.5 to 0.7 utility units for 5 years, would both result in a gain of one QALY. This allows for easy comparison across different types of health outcome, but still requires value judgements to be made about increases in the quality of life (utility) associated with different health outcomes. The use of incremental cost-utility ratios enables the cost of achieving a health benefit by treatment with a drug to be assessed against similar ratios calculated for other health interventions (for example, surgery or screening by mammography). Therefore, it provides a broader context in which to make judgements about the value for money of using a particular intervention.
When to use economic evaluation
Public spending needs to be accountable. Economic evaluation aims to provide a framework to support transparent decision-making about how resources are allocated. As far as possible, economic evaluations allow the full implications of choices to be identified, made explicit and considered before decisions are made. This allows resources to be allocated in a way that maximises the benefits to society as a whole.
However, resources might also be allocated on the basis of health policy or government and local priorities. Using the previous example, if a choice needed to be made between funding the walking group for the over-60s or implementing the football programme for primary school children, health funders would consider the costs and benefits of each programme. But they may also consider whether intervening with either of these groups was aligned with a government priority. Given a health policy aimed at reducing health inequalities, they might decide to fund the older people’s walking group if this was in a deprived area.
Alternatively, if health policies were aimed at decreasing childhood obesity, they might choose to fund the football group if it was demonstrated that this could lead to weight loss in obese children. In this way, decision-making may not follow the economic evaluation, but is informed by it, and an implicit valuation of benefitting a priority group is revealed.
Challenges of economic evaluation in public health
In public health, it can be very challenging to do a full economic evaluation of programmes, services or projects. Often, information on resources and their associated costs is difficult to obtain or collect, staff conducting evaluations do not have the necessary knowledge or skills, or there are insufficient time and resources to support this type of work. The long time-frame over which the effects of public health programmes are likely to have an impact also makes the measurement of benefits difficult, and potentially necessitates the use of discounting (to reflect the fact that people place greater value on benefits in the short term than longer term).
For these reasons, there is limited existing high-quality economic evidence in many areas of public health research to inform decision making about future initiatives. Nonetheless, there is an increasing expectation that some economic analysis or evaluation will be included in grant applications for research on public health programmes and be part of comprehensive evaluations. See the ‘Evaluability’ section. Any economic analyses that can be undertaken will add value to decision making. However, if you are going to carry out an economic evaluation, you will need to enlist the support of someone with expertise in this area.
Benefits of economic evaluation
Despite the challenges of conducting an economic evaluation (described above), the benefits include:
- assessing resource requirements
- justifying the use of resources in terms of the benefits gained
- using explicit methods for decision making and choosing priorities
- potentially attributing a QALY or monetary value to the impacts of any proposed policy, project or programme to allow comparison to other initiatives
- assessing whether the spending is in alignment with government policies
- identifying other possible approaches which may achieve similar benefits
Review of economic evaluations
What makes economic evaluations different
While outcome evaluations are concerned with whether an intervention works and process evaluations are concerned with how they work, economic evaluations are interested in the value and cost of an intervention. The purpose of an economic evaluation is to assess the value gained from resources used to implement a policy, programme or intervention. By applying economic principles, they seek to understand and make decisions about health and health services. To do this, the benefits of interventions (for example, weight loss, shorter stays in acute settings, quality of life, or survival) are measured and often assigned a value to allow for comparison with other types of programmes. It is important to note that the perspective taken (individual, community or societal) determines which potential costs and benefits are included in an economic evaluation. Economic evaluations can therefore differ according to their scope and intent.
Cost-effective services that are not implemented
Although a service might be an effective use of resources, this does not mean that the resources are available (this can include money, skills, available facilities) or the will is there to implement it (barriers could be political priorities, advocacy groups, patients willingness to use a service and so on).
Benefits and challenges in public health
Some benefits are choosing priorities, justifying use of resources, attributing monetary values to all impacts of any proposed policy, project or programme, assessing resource requirements, understanding costs for benefits, supporting the development of health policy, assessing whether spend is in alignment with government policies and identifying other possible approaches which may achieve similar results.
Some challenges include lack of economic evidence available, lack of economic evaluation skills and knowledge, difficulty of measuring costs (especially when interfacing with other sectors such as social services and education) and the various possible perspectives of costs spent and savings made. Resources spent in one area may cause savings to be made in another area (for example, providing handrails in elderly people’s houses may reduce falls and result in fewer hospital admissions but these savings will be apparent in acute care rather than in public health) and there may be a long time frame for accrual of benefits.
References
Drummond MF, Sculpher MJ, Claxton K, Stoddart GL and Torrance GW (2015): Methods for the economic evaluation of health care programmes. Oxford University Press.
Miller P (2009): An introduction to health economic evaluation. The NIHR Research Design Service for the East Midlands, The NIHR Research Design Service for Yorkshire and the Humber.
Public Health Economics and Tools (2015) Centers for Disease Control and Prevention.
Phillips C and Thompson G (1998): ‘What is a QALY?’ London: Hayward.
Acknowledgements
Written by Jane Smith, Sarah Morgan Trimmer, Margaret Callaghan, and Krystal Warmoth.
This work was partially funded by the UK National Institute for Health Research (NIHR) School for Public Health Research, the NIHR Collaboration for Leadership in Applied Health Research and Care of the South West Peninsula (PenCLAHRC) and by Public Health England. However, the views expressed are those of the authors.