Research and analysis

Background quality report: international defence statistics 2021

Published 11 November 2021

1. Contact Details

Analysis-Expenditure Head of Branch

Tel: 030 679 84442

Email: Analysis-Expenditure-PQ-FOI@mod.gov.uk

Analysis Directorate (Defence Expenditure Analysis)
Ministry of Defence
Oak 0 West, #6028
MOD Abbey Wood North
Bristol
BS34 8QW

2. Introduction

These statistics present comparative information on UK defence spending and that of other countries. The underlying data are used for a wide range of purposes both within the Department and externally to provide accountability and transparency to parliament and the public.

The International Defence Expenditure Bulletin (henceforth in this document ‘International Defence’) is produced annually and contains spending data from the most recent full calendar year for which data are available.

This Bulletin presents defence expenditure of The North Atlantic Treaty Organisation (NATO) member states and expenditure as a percentage of their GDP. A comparison of two international defence spending data sources focuses on the top worldwide spenders. Defence expenditure trends for the UK, Germany, France and the USA are also displayed.

The Bulletin comprises of an HTML report, which focuses on commentary and data visualisations, and is supported by separate data tables that contain the data behind the text and visualisations.

The Ministry of Defence (MOD) has published international comparison statistics since 2001.

3. Statistical Processing

Prior to 2013, these statistics were published as part of an annual MOD statistics UK Defence Statistics Compendium for the year 2010 onwards). Since 2013 these figures have been published as the standalone International Defence Bulletin. Although this Bulletin began relatively recently, the external data sources we use allow some of the time series we report to extend back to 1980.

The 2020 Bulletin includes: NATO countries’ expenditure (in real terms and as a percentage of GDP), NATO countries’ equipment expenditure as a percentage of defence spending, top world-wide military spenders, the differences between two widely used sources of international defence spending estimates (these being The International Institute for Strategic Studies (IISS) and The Stockholm International Peace Research Institute (SIPRI)), the effect on estimates of the currency conversion methodology employed, and a section providing in-depth focus on the UK and some key NATO allies (USA, France and Germany) on a set of key comparators (percentage of GDP spent on defence, defence spending per person, spend per service person, and the number of military personnel per thousand population).

All of the data sources used are administrative data from either NATO, IISS or SIPRI and would previously be published in an external bulletin by at least one of these institutes.

4. Quality Management

The MOD’s quality management process for Official Statistics consists of three elements:

1. Regularly monitoring and assessing quality risk via an annual assessment.
2. Providing a mechanism for reporting and reviewing revisions/corrections to Official Statistics.
3. Ensuring BQRs are publishing alongside reports and are updated regularly.

5. Relevance

These statistics are often used within the MOD to provide context in papers, press lines and briefings. In 2010 and 2015 they were used to underpin benchmarking work for the UK Strategic Defence and Security Review (as well as for the Australian SDSR) and to inform the corresponding Spending Review settlements. The production of the tables has placed the Defence Expenditure Analysis branch (the producers of this Bulletin within MOD) in a position to offer guidance on the use of these statistics to others within the Department and they have formed the basis of several international collaboration projects; this is a strength of the International Defence Bulletin.

There have been some internal requests for performance metrics measuring international defence capabilities. There is consistent interest in making international comparisons based on defence ‘outputs’ rather than ‘input’ measures, and these statistics provide information solely about the latter of these. The lack of ‘output’ measures could be considered a weakness of this Bulletin.

Externally to the MOD, policy think tanks such as the Royal United Services Institute (RUSI) and IISS have used the data for policy papers and think pieces. Occasionally contextual questions from academics and interested parties are received.

6. Accuracy and Reliability

6.1 Overall Accuracy

There are a number of issues surrounding the accuracy of NATO, SIPRI, and IISS data which include compliance with definitions, different treatment of pension contributions and tax regimes, and conversion methods. This is explained further in this document under the Coherence and Comparability section. In addition, as some countries do not provide returns to SIPRI or IISS, some of their figures are estimates. MOD has no control over the quality, reliability and coverage of the data contained in tables sourced from these organisations.

6.2 Revisions

Revisions made by the statistics producers are reflected with ‘r’ markers. We follow the Defence Statistics revisions policy.

7. Timeliness and Punctuality

7.1 Timeliness

The aim is to publish this Bulletin in late summer and the statistics usually refer to the previous Calendar Year. However, the timeliness of the Bulletin ultimately depends on when the source data is released by NATO, SIPRI and IISS. In 2013, no expenditure data was published by NATO. This delay meant that there was no International Defence Bulletin in 2013, and the 2014 Bulletin contained both the 2012 and 2013 data.

7.2 Punctuality

The International Defence Bulletin is not an official statistic publication so does not appear on the MOD’s Calendar of Upcoming Releases section of GOV.UK.

8. Coherence and Comparability

The statistics in all tables are consistent with those published by NATO, SIPRI and IISS. Where MOD has converted data into US dollars or to constant prices, this methodology has been agreed with NATO statistical experts.

NATO, SIPRI and IISS have different definitions of military expenditure which, additionally, differ from the definition of Defence Expenditure used to compile the MOD’s accounts, presented in the Departmental Resources Statistical Bulletin. It is not therefore possible to easily compare spending figures from these different sources.

When undertaking international comparisons of defence it is important to consider some well documented issues relating to the comparability and granularity of the international source data. Making direct comparisons will never be straightforward because:

• Whilst there are standardised definitions of defence spending and accounting conventions used by international organisations, principally the UN and NATO, not all countries record and publish their defence spending in accordance with such definitions and conventions. The standards themselves are not always transparent, adding to the difficulties of comparing estimates produced by different organisations. More information relating to the revised NATO definition can be found on the NATO website. Expenditure is included for countries that provide Military Assistance; expenditure is not included for countries receiving assistance. More information relating to the SIPRI definition of military expenditure can found on the SIPRI website.

• Some countries’ actual defence expenditure may be very different from their budgeted expenditure.

• Differences in national tax regimes and the treatment of pension contributions can lead to significant distortions in expenditure.

• Departments other than defence departments may be deemed to contribute to defence whilst some spending by defence departments can be categorised as supporting other activities.

• The choice of conversion method (e.g. Market Exchange Rates (MER) or Purchasing Power Parity (PPP) rates) used to convert to a common currency can result in significantly different rankings of global defence spending. Using MER for instance tends to undervalue the currency, and hence the scale of expenditure, for countries with lower national incomes. Attempts are often made to circumvent this problem using PPP rates. These use currency conversion rates which equalise the overall price of a bundle of goods and services in each country. However PPP rates can be highly inaccurate because of the difficulty of allowing for differences in quality and devising appropriate and relevant “weightings” of individual goods and services. Civilian based PPPs may also not be representative of defence goods and services. Further information on the measurement and uses of Purchasing Power Parity (PPP) rates can be found in the Organisation for Economic Co-operation and Development’s Statistics Brief (published March 2002).

9. Accessibility and Clarity

An HTML version of this Bulletin and accompanying data tables are published and freely available on the GOV.UK website. Screen readers can be used and alternative text has been provided for every chart.

The level of detail in the Bulletin and accompanying tables is limited by the available data but the formats chosen should be clear and understandable.

The strengths of these statistics include the clarity of presentation and the clear signposting of data limitations. The selection of key facts to focus on in the commentary and charts is also a strength, helping the reader make sense of the large amount of underlying data.

10. Trade-offs between Output Quality Components

There is occasionally a trade-off between timeliness and comparability; the timing of the publication of these statistics ultimately depends on when the source material is released.

11. Performance, Cost and Respondent Burden

The tables provided alongside the Bulletin are produced purely using data collected by external organisations. The Bulletin published by MOD placed no additional burden on the respondents.

12. Confidentiality and Security

In producing these statistics, MOD adheres to the Defence Statistics confidentiality policy.

MOD maintains good links with policy colleagues to ensure that these statistics are understood and prevent misuse. The International Defence Bulletin contains commentary explaining these statistics.

The NATO table for exchange rates from US$ to another national currency is restricted and all of the data used in this bulletin is stored securely online on an SQL server.