Housing market and house price information: notes and definitions
Information for local authorities and data analysts, including methodology, the Communities and Local Government House Price Index, the Regulated Mortgage Survey
Applies to England
Overview
This guidance is to be read in conjunction with the series on housing market and house price information which includes the latest statistical releases and sets of live tables.
The Department for Communities and Local Government (DCLG) is interested in monitoring house prices and housing market information to identify areas of high and low demand and to look at levels of housing affordability. The monthly house price index is a weighted average of prices for a standard mix of dwellings.
History and context
The data used for this is a survey known as the Regulated Mortgage Survey (RMS). From 1993 to 2002, the sample size ranged from 26,000 to 36,000 cases per year. Since 2001 a number of lenders have started supplying 100 per cent of completions instead of a five per cent sample. This allowed us to launch the more reliable monthly house price index in September 2003, with a monthly series starting from February 2002. The number of cases included in the calculations, currently averages over 25,000 cases per month. We estimate that in 2007 we obtained data for 60 per cent of all UK mortgages for house purchase.
The questionnaire on which the survey is based has been revised from time to time, notably in 1982 when the question on the previous tenure of borrowers was extended to identify sitting tenants. The method of analysis was introduced in the second quarter of 1968 and most of the detailed series now published have their origins in this period. A description of the current methodology can be found below.
From 1969 to 1992 the index reflected dwellings mortgaged by building societies. From 1981 sales to local authority sitting tenants were excluded and from 1982 sales to all sitting tenants were excluded. From 1983 the standard mix of dwellings was updated annually to reflect the mix of dwellings purchased during the previous three years. In the early 1990s many of the larger building societies converted to banks and an index limited to building society mortgages could no longer be assumed to be representative of all house purchases. So from 1993 the index was extended to include both banks and building societies and was re-named the All Lenders index. Since then the survey has been revised occasionally with the current version, the RMS, starting in August 2005.
Figures from the RMS are subject to random sampling errors. Approximate coefficients of variation of the average dwelling prices, advances and incomes in each series vary according to the sample size.
The Statistical Release features the latest house prices and house price annual rates of change figures based on completions data reported by a wide range of mortgage lenders. Tables include: house price indices; house price annual rate of change figures; mix-adjusted average house prices - both for the UK and for the regions, and by type of buyer at the UK level. A commentary is included to highlight the most significant aspects of the latest data.
The release is usually published at 09.30 hrs on the second Tuesday of each month with release dates given six months in advance. The release is shown in the Housing market document series.
Live tables
Live tables are available for the latest mix-adjusted house prices, house price annual rate of change figures and simple average house prices, mortgages and incomes using data from the RMS.
Live tables are also available for mean, median and lower quartile house prices based on HM Land Registry data providing comprehensive coverage of all residential property transactions in England and Wales. Data from HM Land Registry is also used, along with earnings data from the Office for National Statistics, to show affordability at the district level.
Live tables are available for other housing market data including house prices from Halifax, Nationwide and Land Registry, property transactions from HM Revenue and Customs and land prices from Valuation Office.
You will find all the live tables listed in the Housing market document series, or you can search by name if you know the table you are looking for.
Monthly house price index methodology
History of the price index
The mix-adjusted house price index is a weighted average of prices for a standard mix of dwellings. From 1969 to 1992 the index reflected dwellings mortgaged by building societies. From 1983 the standard mix of dwellings was updated annually to reflect the types of dwellings purchased during the previous three years.
In the early 1990s many of the larger building societies converted to banks and an index limited to building society mortgages could no longer be assumed to be representative of all house purchases. So from 1993 the index was extended to include both banks and building societies and was re-named the All Lenders index. It was based on a five per cent sample survey of all lenders.
Since 2001 a number of lenders (but not all) submitted details of all their completions instead of a five per cent sample. Data from 2003 through to August 2005 reflects the results of this enlarged sample. From September 2005, the survey became 100 per cent for those lenders taking part and changed to take data from the Financial Services Authority data collection.
Its name was changed to the Regulated Mortgage Survey (RMS). Data covering 1993 to 2002 are based on a sample size ranging from 26,000 to 36,000 per annum. From mid-2003 to August 2005 the survey covered about 25,000 mortgage completions per month. The RMS is larger and by the end of 2005 was covering about 40,000 mortgage completions a month. During 2006 about 50,000 mortgage completions a month were covered.
During 2007 there were around 50,000 records per month supplied by about 60 lenders. In the 6 months to May 2010 there were 32 lenders on average contributing to the house price index, providing around 23,000 records per month
The questionnaire on which the survey is based has been revised from time to time. For example, in 1982 when the question on the previous tenure of borrowers was extended to identify sitting tenants. The present method of analysis was introduced in the second quarter of 1968 and most of the detailed series now published have their origins in this period, although the mix-adjustment was modernised in 2003 when the monthly series was introduced.
Regulated Mortgage Survey
The Regulated Mortgage Survey (RMS) is a monthly Communities and Local Government and Council of Mortgage Lenders survey, which collects house price information from mortgage lenders for properties that reached completion during the month and for which they provided the mortgage.
Other data collected includes locality, type of dwelling, size, whether or not the property is new and whether the buyer was a first time buyer or not.
The data are used primarily to determine a monthly mix-adjusted house price index - but also provides valuable additional information on the housing market.
Monthly house price index
Communities and Local Government launched a new, monthly house price index in September 2003.
Thanks to electronic data transfer, many lenders have now switched to supplying 100 per cent of their completions instead of a five per cent sample. Consequently 25,000 completions per month are included in the new monthly index.
The new index covers purchases for owner-occupation and for buy-to-let. Re-mortgages are not included because the price of a property represents a valuation, which is not necessarily the same as a negotiated marketprice. Purchases by sitting tenants are also omitted because they are below market level.
DCLG has also improved the underlying index methodology (assisted by methodologists from the Office for National Statistics, ONS). The overall structure of the new index reflects the recommendations of a cross-cutting House Price Working Group (comprising representatives from Communities and Local Government, the Office for National Statistics, HM Treasury, the Bank of England and Land Registry).
For further information on the House Price Working Group, and its findings and recommendations, see the ONS article in the October 2002 issue of Economic Trends, An improved House Price Index - an update on developments (PDF, 3 pages, 15KB).
A regression model is used in the new index to produce price estimates for about 100,000 cells. The benefits of this approach are that it gives greater flexibility:
- no need for minimum sample per cell
- partial data can be used
- better mix adjustment
- more detail on location and dwelling size
- can add new variables
- in order to select the price estimation model, a range of alternative models were compared and tested to ensure that:
- house prices are log-normally distributed
- no key variables are omitted
- the modelled means are comparable with the actual means
The preferred model was a main effects model including seven explanatory variables and several two-way interactions between these variables:
- dwelling type
- is dwelling old or new?
- is buyer a first-time buyer?
- number of habitable rooms
- county (or London borough)
- type of local authority (based on an ONS classification)
- type of neighbourhood
Weights are calculated once a year (in January) based on the relative numbers of transactions during the previous three years. However, because these transaction weights are applied to average cell prices, the net effect is that the index uses expenditure weights.
The index is an annual chain-linked Laspeyres-type index, like the Retail Prices Index.
House price indices for the UK and the Government Office Regions are published in a monthly press release. A quarterly report will carry more in-depth analysis and a wider range of indices and other house price indicators. A seasonally-adjusted series will be constructed in due course, but a run of between two to five years of the new data will be needed before the parameters required for seasonal adjustment can be reliably determined.
The ultimate target is to create a house price index that reflects cash purchases as well as mortgaged purchases. Unfortunately the Land Registry is the only possible source of information on completed cash purchases - and there are two major deficiencies in the Land Registry data:
a) the excessive time lag between completion and the time details are received by Land Registry b) the limited range of information available for each property purchased.
Land Registry’s long-term plans for ‘e-conveyancing’ and the proposed development of a National Property Databank (by the Valuation Office) may help overcome these problems the Halifax and Nationwide house price indices still have an important role to play. Their indices are based on mortgage approvals - so reflect house prices at least one month ahead of completion. Moreover, since the Halifax and Nationwide use only their own in-house data they can process it immediately and do not have to await the receipt of data from other lenders. Consequently the Halifax and Nationwide indices continue to provide a much earlier indicator of house price trends than our new index.
Lenders participate in the Survey of Mortgage Lenders on a voluntary basis so we cannot insist either that they switch to 100 per cent or that they must do so by a particular point in time.
House prices data: Notes, definitions and methodology
For definition of a dwelling, type of dwelling, tenure and general definition of a household see Definitions of general housing terms.
First time buyer/former owner occupier
In the statistics presented a former owner occupier household is a household that was in owner occupation immediately prior to making a house purchase. First time buyers, in our statistics from the Regulated Mortgage Survey, are households that move into owner occupation without having to sell a property. Therefore numbers will include some buyers who have previously owned a property before, but are not in owner occupation at the time of purchase. Estimates from the CLG Survey of English Housing suggest that around 20 per cent of stated first time buyers may fall into this category. The other 80 per cent will be ‘genuine’ first time buyers (ie those who have never owned a property before).
Mix-adjusted house prices
The house price index is mix-adjusted to allow for the fact that different houses are sold in different periods. House prices are modeled using a combination of factors that produce a model containing a large number of “cells” (variable combinations such as first time buyer, old dwelling, and detached house in London). Once the monthly price estimates for all cells have been determined by the model, they are combined with their appropriate weights to produce that month’s mix-adjusted average prices for all the required output categories. Weights are calculated once a year based on the relative numbers of transactions during the previous three years, given by the Land Registry data. The index is an annual chain-linked Laspeyres-type index, like the Retail Prices Index as published by the Office for National Statistics.
New dwellings
New dwellings are defined as those that never previously existed. So conversions of buildings (e.g. out houses, barns) into living accommodation are not counted as “new” dwellings.
Land Registry
Her Majesty’s Land Registry (Land Registry) was established in 1862. It is a non-ministerial government department that collects data on transactions in property and land in England and Wales. The Land Registry data on residential house price transactions feeds into some of our live tables. This dataset only includes residential transactions at full market value. This means it excludes:
- All commercial transactions
- Transfer, conveyances, assignments or lease at a premium with nominal rent which are:
- Right to Buy sales at a discount
- subject to a lease
- subject to an existing mortgage
- to effect the sale of a share in a property
- by way of a gift
- by way of exchange
- under a Compulsory Purchase Order
- under a court order
- to Trustees
- Vesting Deeds
- Transmissions or Assents
- of more than one property
- leases for seven years or less
- Mortgage Lending
Income
The income of borrowers is the total declared income on which the mortgage is based. It may overstate/understate the borrowers’ true income and may include more than one income.
Buy to let mortgage
A buy to let mortgage is a mortgage vehicle that is used by a borrower to obtain money to buy a property in the private sector with the purpose of letting the property to a tenant.
Mortgage Payments
Mortgage payments results are analysed by Communities and Local Government from data collected in the Family Resources Survey (FRS), a survey commissioned by the Department for Work and Pensions. The published data are for England, as for the SEH data. Payments include both interest and the repayment of capital, if there was any. Insurance premiums relating to endowment mortgages are included, as are similar payments for pension or PEP mortgages and the like.
Care is needed in interpreting the figures as some are based on small samples and are therefore subject to substantial sampling error. The figures are shown grossed, using the Department for Work and Pensions own grossing factors.
Further information is published by the Department for Work and Pensions in FRS annual reports. Data from the Expenditure and Food Survey (EFS) also continue to be published annually in a report entitled Family Spending. Unpublished FRS and EFS data on incomes, rents and mortgage payments can be obtained from:
DCLG Housing Analysis and Surveys Division
Zone 4/J2
Eland House
London SW1E 5DU
Contacts
Email: housing.statistics@communities.gsi.gov.uk
Telephone: +44 (0) 303 444 2270