Chapter 1 Measuring Inflation

Jill Leyland

What is inflation and why do we measure it?

Why do we not have just one measure of inflation?

How do we decide what prices to use when we calculate inflation?

Does inflation accurately measure the increase in the cost of living?

How far back in time can we measure inflation?

Should inflation measures include house prices?

Inflation is a measure of how the prices of goods and services increase over time, usually expressed as a percentage. If those prices fall, it is known as deflation. If prices are rising exceptionally rapidly in a country it is experiencing hyperinflation. One example is Venezuela where prices started to rise rapidly in 2016, accelerated in 2017 and then inflation hit levels of over 1 million percent in late 2018, rising, according to some estimates, to over 2 million percent in early 2019.

When inflation reaches these levels wages have to be spent immediately and the currency becomes worthless. In August 2018 Venezuela slashed five zeroes off its currency to introduce the Sovereign Bolivar (equal to 100,000 old Bolivar). Such levels of inflation can cause bizarre problems too. Just prior to the currency reform, the amount of notes needed to buy a chicken would have weighed more than 14 kg—far more than the chicken.

Such levels of inflation cause obvious disruption and misery. But less extreme inflation, and how it is measured, affects us all. Measuring inflation can, however, be confusing. Did UK consumer prices rise in 2017 by 3.6%, as the Retail Prices Index (RPI) suggested, 2.6% as the Consumer Prices Index (CPI) showed, or 2.7% as CPI including owner-occupied housing (CPIH) showed?

Deciding how we measure inflation is one of the most contentious issues in economic statistics. The arguments combine history, politics, and legal issues with technical statistics. This is because official statistics are not produced in a vacuum, but must respond to—and are affected by—the messy realities of everyday living.

The example of Venezuela is not a one-off. The hyperinflation Germany experienced between 1922 and 1923 left scars on the national consciousness that still resonate, to some extent, today. Hungary, Zimbabwe, Greece and Peru are some countries that also experienced hyperinflation during the last 100 years.

It is thought that the worst ever hyperinflationary episode occurred in Hungary from 1945 to 1946. The best estimates suggest that, in not much more than a year, prices rose by 10, followed by 22 more zeroes, percent—that is, 100,000,000,000,000,000,000,000%. But who knows? Clearly it becomes virtually impossible to measure inflation at such times. By comparison, the better-known German hyperinflation after the First World War is thought to have involved a price increase of a mere 1,000,000,000,000%.

1.1 Why we measure inflation

Inflation matters to people. Years of even minor inflation can subtly erode your standard of living if your income does not increase sufficiently to compensate. And because consumer inflation measures are used to uprate pensions, rail fares, student loan repayments and tax thresholds, they affect our income and expenditure directly, unlike most economic statistics.

inflation rate
When people talk about the rate of inflation in a month, they are usually comparing prices with those of 12 months earlier. More precisely, it is the annual inflation rate, which is the increase in prices between the two dates as a percentage of the original figure. For example, if we are looking at monthly data and the index in January 2017 had a value of 106.5 and January 2018 had a rate of 109.2, then the inflation rate in January 2018 was 109.2 / 106.5 × 100 – 100 = 2.5%.

Rapid inflation, even well below hyperinflationary levels, can cause problems. Many people remember the problems caused by energy price rises in the 1970s or by mortgage interest rates in double digits in the early 1990s. Figure 1.1 shows the annual inflation rate (percentage change from one year to the next in the index) from 1901. (The index is a composite rate, piecing together the RPI with work done by economic historians for the earlier periods.)

Figure 1.1 Inflation since 1901

Annual inflation since 1901, UK RPI and historical data (% change in composite index)

Annual inflation since 1901, UK RPI and historical data (% change in composite index)

Office for National Statistics – ‘Consumer price inflation’

Figure 1.1 shows the sharp rise in prices during the First World War (when limited price controls and rationing were only introduced relatively late), followed by a dip afterwards, with a lesser rise during the Second World War. The high inflation of the 1970s also stands out. More recently, 2008 saw a dip due primarily to the effect of the sharp reduction in interest rates, imposed to soften the impact of the financial crash on mortgage interest payments.

The BBC website has a facility, based on research from the University of Warwick, for individuals to calculate their own personal inflation rate, based on where they live and the type of household they live in. It does not, however, adjust the interest rate depending on individual spending patterns.

It follows that accurate measuring of inflation is crucial and how it is done can be a matter of real public interest. Indeed, controversy about official inflation figures is common in most countries. People have different experiences of inflation, while the official index is a form of average. Different people have different expenditure patterns, with some items showing greater or smaller price changes than the norm. A person who habitually spends more on items that have risen faster in any period may indeed feel his or her personal inflation rate is greater than the official index shows. This sometimes leads to criticisms about the accuracy of official indices but, at least on these grounds, such criticism is misplaced.

1.2 Types of inflation

So far, we have covered what is technically called consumer price inflation, or inflation experienced by individuals and households. In public debate, this is often used as shorthand for overall inflation in the economy. But, of course, it measures inflation experienced by only one sector of the economy: households. Inflation in other sectors is also important to measure.

A very broad measure of inflation in the economy is effectively provided by what is known as the GDP deflator, discussed in more detail in Section 1.2.2. Other important price series are those for:

1.2.1 Using an index

How do we find a way of capturing millions of prices to come up with a single figure for consumer prices or producer prices in a country? The approach used is index numbers, which allow us to combine large quantities of data, and also to avoid the problem of having to convert them into pounds, euros or dollars.

The central concept in creating inflation indices is to take patterns of spending (or whatever is appropriate) along with representative prices for different items. These are combined using a weighted average of the different prices, with the weights accounting for the share of spending on that item. For example, if households typically spend 20% of their income on food, the price of food has a 20% weight in the calculation; if electricity costs typically account for one-tenth of an industry’s costs, then electricity prices have a 10% weight in the input producer price series for that industry.

In practice, it is not the actual prices that are combined. If they were used, then higher-priced items would have more weight in the index. Because the weights are based on actual household spending, they already take account of the different prices of different items. What we are interested in is the growth in prices. Therefore, the ratio of prices in the current period to the prices in the base period is calculated. This is known as the price relative; it is the price relatives that are weighted together to form the index. The process is examined in more detail in Section 1.3.

The index is a series that selects a particular period as the base, which is normally set at 100. If an index has, for example, 2015 as the base and 2016 is 105, then prices rose by 5% between the two years. If the index is 108 in 2017, then prices rose by 8% between 2015 and 2017 and by 108 / 105 × 100 = 2.9% between 2016 and 2017. Note that a change in the base, or reference, year does not alter the growth rate between one year and the next, although it alters the index figures. Only when there is a change in weighting or something else—such as a change in the composition of the index—do the growth rates change.

1.2.2 The gross domestic product (GDP) deflator

GDP deflator
The price of all the output that the economy produces. It is created using the components of real GDP, each deflated by the most relevant index of inflation.

One of the price indices with the widest coverage is the GDP deflator. It is essentially the price of all the output that the economy produces. It therefore covers not just the household sector, but all sectors of the economy.

Notwithstanding its importance, the GDP deflator is a derived or implicit deflator. The index is not constructed explicitly but falls out of dividing the series for nominal GDP by that for real GDP. The various components of real GDP that are added together to form the aggregate are each deflated by specific price indices constructed for that purpose. The rest of this chapter, therefore, considers some of the main explicit price indices available, starting with consumer price indices.

1.2.3 Consumer price indices

There is potential confusion in that the generic name for this type of indices is “consumer price indices”, whereas the UK’s Consumer Price Index (CPI) is a specific series. When we refer to “consumer price indices”, we mean the generic class. When we refer to the UK-specific series, we use the term CPI.

The Consumer Price Index Manual: Theory and Practice (2004), which is jointly compiled by a number of international organisations and taken as the most authoritative international guide on consumer prices, describes the aim of consumer price indices as measuring: “the rate of price inflation as experienced and perceived by households in their role as consumers”. It should cover the goods and services consumers buy “to satisfy their needs and wants”.1

Consumer price indices are used for a variety of purposes, for example:

How consumer price indices are designed, and what they cover, can depend on their primary purpose. For example, in the past, one of their main purposes was to guide wage bargaining. This explains why the consumer price index in many countries, including the UK, was originally the responsibility of the Ministry of Labour or Employment, and why the International Labour Organization (ILO) was the international organisation that originally took the lead in co-ordinating and spreading best practice.

Today, the Consumer Price Index Manual sets out several purposes for consumer price indices:

While the general purpose of a CPI [consumer price index] is to measure changes in the prices of consumption goods and services, there are a number of concepts that need to be defined precisely before an operational definition of a CPI [consumer price index] can be arrived at. The concept of consumption is an imprecise one that can be interpreted in several different ways, each of which may lead to a different CPI [consumer price indices]. It is also necessary to decide whether the index is meant to cover all consumers, that is, all households, or just a particular group of households. The scope of a CPI [consumer price index] is inevitably influenced by what is intended, or believed, to be the main use of the index. Compilers also need to remember that the index may be used as proxy for a general price index and used for purposes other than those for which it is intended.2

Does one measure expenditure when it is paid for, when the good or service is acquired, or when the item is used? Often there is little difference between the three. A piece of fruit, for example, is usually paid for at the same time as it is acquired and consumed (used) shortly afterwards. On the other hand, a piece of furniture can be acquired on one day, paid for over a period with financing and used for many years. A package holiday or a theatre ticket can be paid for in advance but “acquired” when the holiday is taken or on the day of the performance. Usually, and partly for practical reasons, consumer prices are based on the time of acquisition and the fact that something may be paid for over a period of time and used for much longer is ignored. When it comes to buying housing, however, the position becomes complicated because of the lengthy period of financing often required and the length of time the house is used.

A consumer price index is sometimes referred to as measuring the cost of living. The meaning of the phrase “cost of living” is not very clear, but for economists the concept means measuring the cost associated with maintaining a similar standard of living; it includes the fact that consumers will at times switch products to take advantage of goods rising less in price or goods that are newly cheaper. This itself raises many questions so in general consumer price indices aim to measure the cost of a typical “basket” of goods and services (although in the US the consumer price index officially aims to be a cost of living index on economic definitions).

Figure 1.2 shows how this works in practice, in the construction of just one group in the CPI. Follow the flow chart to see how many choices are made every time we compile data on a product that we give little thought to: envelopes.

Figure 1.2 Collecting prices to create an index

Location, shop and product selection for envelopes, UK CPI, 2014

Location, shop and product selection for envelopes, UK CPI, 2014

ONS (2014), Consumer Price Indices Technical Manual (2014 Edition), page 13, Office for National Statistics

1.2.4 The Retail Prices Index (RPI)

Retail Prices Index
An index originally introduced to expedite wage bargaining, intended to be typical of the mass of the population and, therefore, excluding high-income households and certain pensioners.

To explain the consumer price indices we have today in the UK, we need some history. The Retail Prices Index (RPI) as we know it today was introduced in 1956, with data starting from January of that year. It replaced the Interim Index of Retail Prices, which had been introduced in 1947. This in turn had replaced the (so-called) Cost of Living Index, introduced at the start of the First World War in 1914.

In 1956, an important intended use of the RPI—as with consumer price indices generally in most countries at that time—was in wage bargaining. Its stated purpose was, therefore, “to measure average price change[s] for households, which would include practically all wage earners and small and medium salary earners”.3 Steps were taken to exclude households not considered to be representative of this group and whose expenditure patterns were found to be very different from most households: high-income households and pensioner households with 75% or more of income coming from state pensions or other state support. These exclusions still exist, with such pensioners and the top 4% of households by income not covered.

Until 1997, the RPI and its derivatives were the only consumer price indices in the UK. In particular, a derivative, RPIX—which excludes mortgage interest payments, but includes other owner-occupier costs—was initially used as the target inflation rate when inflation targeting as a means of monetary policy was introduced in the early 1990s. It was also subsequently used as the target set by the Chancellor for the Bank of England to achieve.

1.2.5 The Consumer Prices Index (CPI)

The current inflation test for membership of the eurozone is for candidate countries to have a rate of inflation not more than 1.5 percentage points above the inflation rates of the three best-performing countries (i.e. countries with the lowest inflation).

In the 1990s, the EU Harmonised Indices of Consumer Prices (HICPs) were created as part of the preparation for the introduction of European Monetary Union in the Maastricht Treaty (1992). The aim was to provide a comparable measure of inflation among the EU member states. One of the tests for membership of the eurozone is for candidate countries to have a rate of inflation comparable with inflation rates of other EU countries. Thus, a measure of inflation based on standardised principles was needed. The HICPs for the combined eurozone countries are also used by the European Central Bank to set its inflation target.

It was not envisaged at the time that the HICPs would replace national consumer price indices. The text of the initial EU regulation setting out the framework made this clear:

Whereas there is a need for the Community and particularly its fiscal and monetary authorities to have regular and timely consumer price indexes for the purpose of providing comparisons of inflation in the macro-economic and international context as distinct from indexes for national and micro-economic purposes … 4

Consumer Prices Index (CPI)
The EU standardised index of consumer prices for the UK. The Bank of England’s target inflation measure.

The HICP for the UK was first published (along with that for other countries) in 1997, with data back to January 1996. It was renamed the Consumer Prices Index (CPI) in December 2003; at the time it replaced RPIX as the Bank of England’s target inflation measure. The UK’s use of CPI as the headline measure is in contrast to most EU countries, which still use their national indices for this purpose. It does, however, remain the UK’s HICP and as such is subject (at least at the time of writing) to EU law.

1.2.6 The Consumer Prices Index plus owner-occupied housing (CPIH)

HICPs (including the CPI) do not currently include any estimate of owner-occupier housing costs. This was due initially to differences in the way different countries treated owner-occupier costs in their national consumer price indices and the consequent difficulty of obtaining agreement on a method. While an agreed method was found in the 1990s, many countries have found problems in implementing it and there does not seen any prospect that such costs will be included in HICPs in the near future.

CPIH
A derivative of the Consumer Prices Index that includes owner-occupied housing costs.

Because of the delay in introducing a measure of owner-occupier costs, the UK decided to create its own series of CPI including such costs, called CPIH. This was first published in 2013. In addition, it also currently differs from CPI by including Council Tax. Since March 2017, it has been the headline index for the Office for National Statistics (ONS).

1.2.7 Differences between indices

ONS Resource

In 2013, the UK Statistics Authority invited Paul Johnson, Director of the Institute for Fiscal Studies, to conduct a review of UK price indices to consider what changes are needed to the range of consumer price statistics produced for the UK to best meet current and future user needs. The Royal Statistical Society commissioned a paper to answer some of the challenges it set.

Johnson, P (2015), ‘UK consumer price statistics: A review’

Astin, J and Leyland, J (2015), ‘Towards a household inflation index: Compiling a consumer price index with public credibility’

For both the RPI and CPI/CPIH, a large number of component indices are published. Additionally, there are a number of specific adaptations of the overall index:

Figure 1.3 compares RPI, CPI and CPIH after putting them all on a comparable base of 2005 = 100. Figure 1.4 compares the 12-month inflation rates shown by the indices.

Figure 1.3 Comparing CPI, CPIH and RPI

CPI, CPIH and RPI, UK, 2005 to 2017, (January 2005 = 100)

CPI, CPIH and RPI, UK, 2005 to 2017, (January 2005 = 100)

Office for National Statistics – ‘Consumer price inflation’

Figure 1.4 Comparison of the inflation rates produced by different indices

CPI, CPIH and RPI inflation rates over 12 months, UK, %

CPI, CPIH and RPI inflation rates over 12 months, UK, %

Office for National Statistics – ‘Consumer price inflation’

Before 2010, the main cause of the differences between the RPI and the CPI/CPIH inflation rates—both the higher RPI inflation rates of 2006 to 2008 and the sharply lower rates of 2009—was movements in mortgage interest rates (included in RPI but not in the other two). Two other causes were differences in calculation method—known as the formula effect—and the use of house prices as a depreciation proxy in the RPI. Since 2010, the increased formula effect has played a more significant part. The formula effect is discussed further in Section 1.3.

Because of its use in contracts, the RPI is never revised. In practice, this also applies to the CPI, although under EU law it would have to be revised if an error were found. CPIH has been revised on one or two occasions, due to changes in methodology. The most recent revision was when Council Tax was added in March 2017. It is not expected to be revised in the future, even if methodological changes are made.

Name What it measures Examples of uses
CPI The UK version of the EU HICP, widely used by government and also others, excludes housing costs Bank of England inflation target

Public sector pensions

Index-linked National Savings certificates (after May 2019)

Economic analysis
CPIH The ONS headline measure of inflation, includes housing costs Economic analysis
RPI The longest established consumer price index in the UK. It excludes high-income households and pensioners Student loan repayments

Rail fare increases

The majority of private pension schemes

Government bonds

Table 1.1 Common measures of consumer price inflation in the UK

Common measures of consumer price inflation in the UK

1.3 Compiling an index

Consumer price indices require two basic sets of information: weights and prices.

Figure 1.5 Steps to create a consumer price index

Using weights and prices in the construction of CPI, RPI and CPIH, UK

Using weights and prices in the construction of CPI, RPI and CPIH, UK

Figure 1.5a Collect data

The first step is to collect current prices, and discover how households we want to measure allocate their spending. Base prices will have been collected during the base period.

The first step is to collect current prices, and discover how households we want to measure allocate their spending. Base prices will have been collected during the base period.

Figure 1.5b The price relative

This is the ratio of current to base price for each item.

This is the ratio of current to base price for each item.

Figure 1.5c Calculate the index

Now we have weights and price relatives, which are sufficient to calculate a consumer price index.

Now we have weights and price relatives, which are sufficient to calculate a consumer price index.

1.3.1 Collecting information: weights

The weights represent the proportions that households spend on average on different types of goods and services— these determine the make-up of the basket of goods and services that are measured. In the case of the RPI, these come essentially from the annual Living Costs and Food (LCF) Survey—which asks people to record what they spend their money on over two weeks—supplemented by other information. CPI and CPIH use information from the national accounts system, much of which is based on LCF information, but adjusted so that the weights are more broadly based.

In the UK, the basket of goods and services is updated every year in March to reflect changes in household behaviour, as determined by the LCF survey or other information. The changes apply to the February index onwards—consumer price indices for any particular month are published in the middle of the following month. At this time, weights are updated to reflect the latest information available from the LCF and other sources, so some items have greater weight and others lesser. New items may enter the index and some items may leave.

ONS Resource

If you want to find out the surprisingly interesting reasons why the ONS included quiche in its basket but removed nectarines in 2018, the explanatory article for 2018 sets out the changes, and the reasons, in detail.

ONS (2018), ‘Consumer price inflation basket of goods and services: 2018’

Sometimes, items introduced are a straight replacement for another item. For example, in March 2018 a child’s sit-and-ride toy replaced a tricycle. Sometimes, though, items are added or removed because the class of expenditure they are in needs to be expanded or contracted. For example, in 2018 quiche was introduced to improve coverage of the pizza and quiche subclass within bread and cereals, and a soft play session was added to improve coverage of children’s activities. Raspberries replaced peaches and nectarines in the fruit class to improve coverage of soft fruits, with an offsetting reduction in the number of stoned fruits by omitting peaches and nectarines; this also meant that the weight of other soft fruits was reduced. The annual change is always accompanied by an article from the ONS.

1.3.2 Collecting information: prices

The second piece of information needed to compile an index is the monthly collection of a sample of prices. Some of this is still done by the traditional method of sending price collectors toshops once a month; some prices are collected by telephone or email; some are collected centrally from head or regional offices or taken from the internet. Altogether, around 180,000 price quotes for nearly 700 products or services are collected each month. Shops used for collection are spread around the country to take account of potential regional variations. Once prices have been collected, a number of checks and amendments are carried out.

1.3.3 Calculating elementary aggregates

Once price quotes for any price index have been corrected, some basic steps must be carried out to ensure that the set of quotes is fit for purpose. One step is to check any figure that seems out of line—for example, if a price seems to be substantially higher or lower than the previous quote.

Sometimes, prices for particular periods are missing and must be imputed. This is usually done by looking at the movements in prices of similar goods or services.

If products and services change, then a change in prices is due to both inflation and a difference in quality. This can often happen if there is a technical improvement in the product.

In general, price indices aim to measure the evolution of prices of goods and services of constant quality, so if a new and improved version of a product has a higher price than the preceding version, the question arises as to whether the rise is due to the improvement or to inflation or to a mixture of both.

If it is deemed that it is all due to an improvement, then the new price is reset to equal the previous one, with corresponding adjustments made to subsequent ones so that the impact of the technical improvement is nullified. If the price rise is due to a mixture of inflation and improvements to the product, then partial adjustments may be made. Clearly, this can require some fine judgement.

In practice, the ONS uses a variety of different methods. Adapting for quality change can be particularly difficult with very technical goods, such as computers, digital cameras and smartphones, which have models and characteristics that change rapidly. In these cases, a technique known as hedonic regression is used.

hedonic regression
Regression techniques are used to relate the price of a good to its characteristics. For example, the price of a computer could be related to its processing speed, the size of the hard drive and the amount of memory. Iterative regressions are run over a sample of goods in one month to relate the log of the price to different features, deriving a coefficient (or weight) for each feature. When a product is no longer available or a new product appears, the equation can be used to give a predicted price. The procedure is complicated and requires a good understanding of the products concerned.

How it’s done Quality adjustments using hedonic regression

The Consumer Prices Indices Technical Manual 2014 gives an example of hedonic regressions using computers:

“[T]he measurable characteristics may include the speed of the processor, the size of the hard disk drive and the amount of memory … The results of the regressions are used to value changes in quality when a product that is part of the sample is no longer available and is replaced by another product.”5

The regression uses the log of the price of the good as the dependent variable, and the log of the technology level of components to impute the impact that improving one component should have on price. So even if the price rises when technology improves, if this is by less than the hedonic regression predicts, we can argue that the price has actually fallen.

Hedonic regression is not the only method that the ONS uses. Others include option costing, quantity adjustments and class mean imputation. There are also bespoke methods for mobile phone contracts and used cars. This is an important part of creating indices, but clearly it involves subjective judgements.

1.3.4 Unweighted averages: Dutot, Carli and Jevons

After the basic information has been collected, the index must be compiled. It is a hierarchical process. Indices are first compiled for very detailed groups, known as elementary aggregates; this could be, for example, as detailed as white sliced bread, branded, in the north-east. Some elementary aggregates can also distinguish between small shops and chain stores.

As we have discussed, we would ideally calculate consumer price inflation by combining the prices of goods—or more precisely their price relatives—and their weight—how much is spent on them in the most recent period for which spending information is available. For some goods, however, we do not have information on how much is spent. The LCF will tell us how much is spent on potatoes, but not how much is spent on King Edward potatoes as compared with Maris Piper or Charlotte potatoes, for example. And there is no information as to how much is spent in a particular shop. So, at this most detailed level of prices, we face the problem that we do not have information on spending and, therefore, no weights. This applies to approximately two-thirds of the items in the CPI and a little less than 60% of the items in the CPIH and RPI.

unweighted averages
  • Dutot: the mean of all prices for the month under consideration, divided by the mean of prices in the base month.
  • Carli: The mean of the price relatives (where a price relative is the ratio of each price in the month under consideration), divided by that price in the base month.
  • Jevons: The geometric mean of the prices in the month under consideration, divided by the geometric mean for the base month.

What we do know, of course, is the price in any particular period, which can be compared with the price of the same product in a base period. This is known as a price relative. Unweighted averages of these price relatives can then be calculated at these very low levels of aggregation.

Three forms of unweighted average are used for this purpose. Two are arithmetic averages that calculate the mean of items ( if there are n numbers then the numbers are added together and divided by n). The third is a geometric average. To calculate the geometric mean of n numbers, multiply them together and then take the nth root.

The first form of arithmetic average is the ratio of averages, often called the Dutot after the man who invented it. The arithmetic average of all prices for the category is calculated for both the base month (January of each year in the UK) and for the month under consideration. The ratio of these two averages is then calculated.

Where It,0 is the price index of the elementary aggregate under consideration, n is the number of price quotes and pi,t is the price of item I at time t.

The Dutot is also known as the ratio of averages, for obvious reasons.

The second form of arithmetic average is the average of price relatives, or the Carli. The ratio (price relative) of each matched pair of prices in the base month and the month under consideration is calculated and the arithmetic average (mean) of these is then taken.

There are two versions of the Carli, the chained and the direct. The UK uses the direct version. The chained version is liable to serious upward drift: if prices rise and then fall back to the previous level, the index shows an increase. This would not be the case for the direct form of the index, which would show the same result as the original period.

The third formula, a geometric mean, is also known as the Jevons. This can be calculated as the geometric mean of the price relatives, as in the first formula below. Alternatively, the geometric mean of the prices in period t can be divided by the geometric mean of prices in the base period 0. It does not matter which calculation is used because the results are mathematically identical.

The RPI uses a mixture of the Carli and the Dutot. The CPI and CPIH use mainly the Jevons, with a limited use of the Dutot.

The differences in formulae underline one of the main controversies about the UK price indices. A geometric average is always less than or equal to the arithmetic average. Given the same set of price relatives, the Carli index (used only in the RPI) will always give a higher inflation estimate than the Jevons (used for most calculations in the CPI and CPIH). The greater the variability in the price relatives, the greater the difference.

The Dutot sometimes gives a higher inflation estimate than the Jevons and sometimes lower, depending on whether there is more variability in the prices of the base period or those of the current period.

Like all formulae used in index numbers, each of these has its problems.

Over time, most expert opinion has switched towards the Jevons and away from the Carli as the preferred method, with the Dutot as a second preference. The issue nevertheless remains controversial.

Worked example Comparing the Dutot, Carli and Jevons averages

Imagine that a chain of small food shops keeps track of its average selling prices once a quarter for 12 items. (Note: these are not real prices!)

As a small business, its prices would reflect movements in exchange rates, shortages of supply and seasonal changes. The prices are shown in Table 1.2.

Product Jan Apr July Oct
Rum 36.01 30.78 28.85 26.60
Wine 27.44 24.02 21.37 19.56
Coffee 12.12 15.00 18.03 13.13
Oranges 7.35 9.01 9.11 6.93
Butter 4.89 5.89 5.21 4.34
Quinoa 4.56 7.10 6.70 4.40
Ham 3.98 5.22 5.02 5.10
Bread 2.12 2.90 3.04 2.20
Apples 2.01 2.05 2.02 2.04
Chocolate 1.88 2.45 3.10 2.88
Nuts 0.50 0.02 1.20 3.96
Olives 0.44 0.46 1.34 3.78

Table 1.2 Food prices measured four times a year

Food prices measured four times a year

How do the indices calculated using the three averages reflect inflation, as experienced by the shop’s customers?

From Table 1.2, we can see that the imported exotic rum, wine and premium coffee are much more expensive than other items. Few customers buy them, but their prices fell during the year. Nuts and olives are in high demand at Christmas but there is a limited supply, so prices jump. However, the shop overstocked nuts in the second quarter and so discounted them by selling them for almost nothing to anyone who bought another item.

Table 1.3 shows the price relatives, using January as a base.

Product Jan Apr July Oct
Rum 1.00 0.85 0.80 0.74
Wine 1.00 0.88 0.78 0.71
Coffee 1.00 1.24 1.49 1.08
Oranges 1.00 1.23 1.24 0.94
Butter 1.00 1.20 1.07 0.89
Quinoa 1.00 1.56 1.47 0.96
Ham 1.00 1.31 1.26 1.28
Bread 1.00 1.37 1.43 1.04
Apples 1.00 1.02 1.00 1.01
Chocolate 1.00 1.30 1.65 1.53
Nuts 1.00 0.04 2.40 7.92
Olives 1.00 1.05 3.05 8.59

Table 1.3 Price relatives, using January as a base

Price relatives, using January as a base

Using these price relatives, we can calculate three indices for the four quarters.

  Jan Apr July Oct
Dutot 100.00 101.55 101.64 91.89
Carli 100.00 108.69 146.97 222.56
Jevons 100.00 87.88 135.83 141.57

Table 1.4 Dutot, Carli and Jevons indices

Dutot, Carli and Jevons indices

From Table 1.4, we can see that:

  • the Dutot index showed small changes, even though the prices of items that most people bought rose during the year. As an arithmetic average, it is sensitive to the falls in the prices of rum, wine and premium coffee, the three most expensive products.
  • because prices in the shop vary a lot, the Carli index gave extremely high numbers, even though the only price relatives that were greater than two were for the two cheapest items.
  • the Jevons index was sensitive to the discounting that occurred in the second quarter.

Note that this is a highly simplified exercise, designed to demonstrate the implications of the calculations. The prices are not real, and we have used only a small selection of goods. Also, there is no weighting or grouping: the indices assume that customers buy the same amount of rum as of nuts; as Figure 1.2 shows, in practice a statistician would combine prices into groups that represent the proportion of the household budget allocated to types of goods and services.

You might want to set up a more sophisticated example, perhaps using real prices, and manipulate the data to see if you can observe the same effects.

Our example does not imply that one method is best (you might have an opinion about which one is most suitable in this case), but it does show that the way in which we combine changes in price can matter.

1.3.5 Further aggregation

Once the elementary aggregates have been calculated, they are combined into larger and larger categories. For example, an index for branded sliced bread in the south-east would be combined with data for other regions to make an index for branded sliced bread overall. This would be combined with other breads to make an index for bread, which would then be combined with indices for other foodstuffs to make an overall food category, and so on, until the overall index is compiled.

Laspeyres index
The sum of each product’s price relative, multiplied by its weight.

At these higher stages of aggregation, more information about expenditure patterns is available, so a weighted index can be and is used. The index for each item is weighted by the share of household expenditure attributed to it. This form of index, called a Laspeyres index, is also used to compute many other index numbers. The procedure here is identical for both RPI and CPI, and the price relatives (the index for the current month compared with the base month) are weighted by spending in the latest year for which data are available.

There are many forms of weighted index numbers. When measuring how prices evolve between a base period and the current period being measured, should items be weighted by the base period or by the current period—or something in between? There are many options, but for practical reasons a base weighted index is usually used because not enough information is known about the current period at the time of compilation.

How it’s done The Laspeyres index

The weights are derived from the share of each category or type of product in the base period.

For each quote taken, a price relative is calculated—this is the ratio between the price in the current quarter and the price of the product in the base period.

To calculate the Laspeyres index, each price relative is multiplied by its weight. The results are added together to make the index.

Algebraically:

Where It,0 is the index for time t where the base period is 0

Pit is the price for the ith item at time t

Pi0 is the price for the ith item in the base period

wi is the weight of the ith item

Worked example Calculating a Laspeyres Index

Suppose households buy only potatoes, fruit and washing powder and these have weights of 50, 40 and 10 respectively. Three-quarters of expenditure on fruit is spent on apples and the remaining quarter on oranges. Suppose the price of apples for the month of June is 10% higher than in the base month of January and that for oranges is 20% lower.

  1. Calculate the fruit index:
    • The price relative for apples is 110 / 100 = 1.1.
    • The price relative for oranges is 80 / 100 = 0.8.
    • The index for fruit for June is then 1.1 × 75 + 0.8 × 25 = 102.5.
  2. Combine it with the other indices:
    • Assume the index for potatoes is 98.6, and the index for washing powder is 105.9.
    • Therefore, the overall index is (98.6 × 50 + 102.5 × 40 + 105.9 × 10) / 100 = 100.9.

1.3.6 Chain linking

Consumer price indices are unusual in that the calculation process uses a month, rather than a year average, as an interim reference base for the following 12 months. These annual fragments must be linked together, or “chained”, to form a continuous index.

The RPI is calculated each year with a January base month. The CPI has a December base month. For the RPI each index is calculated up to the following January, and for the CPI to the following December. These yearly fragments must then be linked together.

The annual chaining does not affect the reference base of the index. Since 1987, the RPI has been published on a January 1987 = 100 basis, while the CPI and CPIH base is revised every five years or so and is currently at 2015 = 100. This change is purely arithmetic—it does not affect the growth rates shown by the indices.

The procedure for the RPI is relatively simple. The procedure is not carried out on the elementary aggregates or at the item level of aggregation immediately above, but takes place at the next level, called a section. It is directly connected to the annual update of the basket of goods and services and the corresponding annual weighting adjustments.

How it’s done Chain linking the RPI

Consider what happens at the start of, say, 2018.

Indices for the different sections would be available from January 2017 to January 2018 on the 2017 weights. These would have been initially calculated as January 2017 = 100, but by being linked to previous years, would be on the RPI reference base of January 1987 = 100.

Suppose, therefore, that the index for the component being considered (bread, for example) for January 2018 is 278. This would have been based on the weights and items in the 2017 index.

This is the base covering the items and with the weights of the 2018 index. Suppose the index for February on this basis is, say, 101. To link this to the old series, a value would be created for February 2018, that is, 278 × 101 / 100 = 280.8. The index for following months would be calculated the same way; if the index for May on the January 2018 = 100 index is 102.4, then the chained result for May is 278 × 102.4 / 100 = 284.7. Thus, the index reflects the new weights from February each year.

The process of chaining enables the index to update weights and coverage to maintain its relevance to household expenditure, but it can result in a complex phenomenon known as chain drift, characterised by the index drifting away, in either direction, from the unchained index.

1.4 Controversies and new developments

1.4.1 The formula effect and the RPI

As mentioned earlier, the different formulae used at the elementary index stage give different results; this has been the root of recent controversies about the difference in inflation rates given by the RPI and CPI. RPI inflation is nearly always higher than CPI, with the wedge—the overall difference in annual inflation rates—averaging around 0.8 percentage points in recent years. Most of this is due to the formula effect (the result of the different formulae used in the two series).

The formula effect has always existed but it increased sharply (and unexpectedly) in 2010 as a result of changes made to the collection of clothing prices. The change, expected to be minor, was made in good faith to correct a previous underestimation of clothing prices—an underestimation particularly marked in CPI. The change made was a loosening of like-for-like price-matching criteria applied to data used for both families of index, and the result was rather dramatic.

Since 2010, the RPI–CPI formula effect (as calculated by the ONS) has widened to between 0.9 and 1.0 percentage points per year, compared to 0.4 to 0.7 percentage points previously. It became clear that the RPI overestimated clothing inflation as a result of the change, with some arguing that the use of the Jevons—which is sensitive to very low numbers—would cause the CPI and CPIH to underestimate clothing. This had an effect on the overall index.

The change coincided with the government’s decision to increase the use of CPI. Previously, the RPI had been widely used in both public and private sectors, apart from the Bank of England’s use of the CPI as its inflation-rate target. But in 2010, the government announced that the uprating of public sector pensions and most benefits would be switched from the RPI to the CPI. Because of the generally lower inflation rate given by the CPI, this was controversial and the increase in the formula effect added fuel to the flames.

As a result, the ONS started a major investigation in 2011 into the formula effect, testing whether alternative price collection methods for clothing (and some other items) would reverse the widening seen since 2010, assessing the statistical merits of the different formulae and investigating international practice. It became apparent that the Carli formula used in the RPI was no longer used in consumer price indices by any of the countries investigated. The ONS concluded from all its investigations that the use of the Carli was problematic and that it caused the RPI to overestimate inflation—a conclusion supported by most expert opinion, although not universally. As a result, the ONS proposed changing, to a greater or lesser extent, the Carli formula used in the RPI and carried out a consultation on this.

However, with the RPI still widely used in the private sector, notably for the majority of private sector pension schemes, and because inflation-linked government bonds (gilts) are linked to them, the consultation on the proposed change showed considerable opposition to the idea. In particular, there was lobbying of the Bank of England and the Treasury by gilts investors, making it clear that such a retrospective worsening of the terms of existing index-linked issues could cause the Treasury reputational damage. Further, many respondents made it clear that the continuity of the RPI was valued and that sudden major change would disrupt this. As a result, the then national statistician recommended to the Board of The Authority that no change should be made. She also took a decision to not make any further changes to the compilation of the RPI, other than routine updating. This remains current ONS policy.

The decision to keep RPI methodology largely unchanged is linked to constraints related to index-linked gilts. When these were first introduced in the 1980s, there was some concern in financial markets that government might change RPI to investors’ detriment. Consequently, a clause was added to their prospectus stating that, should RPI compilation or coverage be changed in a way that the Bank of England considered to be both fundamental and detrimental to holders of index-linked gilts, holders of these gilts could demand immediate redemption at uplifted par. This clause was dropped in 2002, but three gilt issues that include the clause are still outstanding (as of early 2019), with the longest-dated due to mature in 2030. The Statistics and Registration Service Act 2007 made it obligatory for the ONS to consult the Bank of England before making any such change to the RPI, as long as any issue with the clause is still extant. If the Bank deems any change to be both fundamental and detrimental to holders of relevant gilts, then the change can only be made with the consent of the Chancellor of the Exchequer. The RPI is the only series that the Act obliges the UK Statistics Authority (the governing body of the ONS) to produce, and the only series that has any political constraint on changes.

In 2013, the RPI had its National Statistic status—essentially a Kitemark of the statistics concerned as complying with the Code of Practice for Statistics—removed by the Assessment Division (now the Office for Statistical Regulation) of the Authority. The two reasons given were the use of the Carli being deemed not to be in line with good statistical practice and the fact that the national statistician’s decision meant that it was no longer open to the prospect of continuous improvement.

As a result, the RPI has been left in an uncomfortable position. It is very clear that it overestimates clothing inflation (possibly other categories as well). The hope in the ONS was that people would stop using it, but any such progress is very slow. The majority of private sector pension schemes still use it for uprating; many schemes are unable to move away from it (this sometimes being tested in the courts). Meanwhile, the longest-dated gilt linked to RPI matures only in 2068. CPI—the current alternative—is, not surprisingly, less popular with pensioners (and many pension fund trustees); there are some grounds for thinking that it underestimates inflation (albeit by far less than the RPI overestimates it).

Read the Economic Affairs Committee’s report.

House of Lords Economic Affairs Committee (2019), 5th Report of Session 2017–19 HL Paper 246: Measuring Inflation

The problems arising from this impasse led to an inquiry by the House of Lords Economic Affairs Committee (started in summer 2018) into the use of the RPI to measure inflation. The Committee’s report, published in January 2019, made a clear recommendation, among other points, that the problem with the RPI should be corrected, that the Chancellor of the Exchequer should allow the change, and that the series should then be managed normally. At the time of writing, the responses from the ONS and Treasury are awaited, but there is clear pressure building for RPI reform.

1.4.2. Measuring owner-occupier housing costs

How to deal with owner-occupier housing costs has long been one of the most controversial issues in consumer price statistics world-wide. Indeed, many countries still do not include them. While some elements of owner-occupier costs—for example, routine repairs and maintenance—are relatively uncontroversial, the cost of house purchase is different.

Part of the problem lies in the substantial difference in timing between acquisition, payment and use, as noted earlier. Part lies in the fact that a house is a capital good and an asset, something that will ultimately generate a substantial sum of money—and, in recent decades, more often than not a profit—for the owner. Economists argue that, as a house is a capital good, it should not be in the index as such, but instead the flow of services a person gets from it should be measured. On the other hand, shelter is a necessity and the cost of a mortgage is a major part of many household budgets.

Various ways have been devised to deal with the problem:

CPIH: rental equivalence

CPIH currently uses rental equivalence, the value of the use of the house is deemed to be equivalent to what the rent would be. Adherents of this approach say it is consistent with national accounts methodology and sits nicely with economic theory. Opponents criticise it as being unrealistic because the rental and purchasing markets are not the same and because it means that a large part of the index is based on an imputed number. The latter argument rules it out for the HICP because imputation is only allowed in the HICP for the minor purpose of estimating the price of a temporarily missing item.

RPI: mortgage interest

The RPI originally used rental equivalence but, following the postwar growth in owner-occupancy, it switched in the 1970s to using mortgage interest payments. In the 1990s, it was decided to include a proxy for depreciation (intended to be equivalent to the amount of money a prudent house owner would put aside annually to meet the cost of occasional major repairs, such as renewing the roof). Changes in depreciation are proxied by changes in house prices. Adherents of this approach say that this is more realistic and closer to what households actually pay than any other approach. Opponents argue that house prices are not a good proxy for depreciation costs, that the price of housing—as a capital cost—should not be in there anyway, that mortgage interest payments are the cost of financing rather than the cost of purchase and that the method does not fall neatly enough into either of the acquisition, payment or use approaches. In addition, the house price index includes the cost of land, which has risen particularly quickly in recent years.

Net acquisitions

It had been believed that the HICP (and thus the UK CPI) would ultimately use the net acquisitions approach. This includes house prices but only of houses new to the housing sector—new builds or housing acquired from outside the household sector, for example, if council-owned housing is sold. The reason for this is that other housing is bought and sold within the household sector, so that for the sector as a whole, the cost is zero. Adherents say that this includes an element of house prices and, since it relates to actual price, is appropriate for an index used for inflation targeting. Opponents argue that, as dwellings are a capital purchase, they should not be in at all or, conversely, that all housing should be in. The ONS, like most other statistical institutes in EU countries, already compiles a quarterly series of owner-occupier housing on this basis. Many countries have had difficulty in compiling these series on a sufficiently timely basis. Thus, the debate continues.

1.4.3 Household costs indices

household costs indices
Measures of inflation designed to reflect costs experiences by households. They may include interest rates, mortgage repayments and insurance premiums; they account for payments when they are made. Each household has identical weight.

The problems with the RPI and the seeming difficulty of improving it led to calls for a new index that would meet the original aim of the RPI—namely, measuring inflation as perceived by households. The ONS is now developing such indices, known as household costs indices. They will be calculated for different types of households, as well as an overall index. An important, but not the only, use will be to publish them alongside the indices for household disposable income for the same groups, so that it can be seen whether income for different groups is keeping pace with inflation.

The main differences between HCIs and indices such as the CPI/HICP, which are designed to meet macroeconomic needs, are:

The first provisional and partly developed HCIs were published in December 2017. Work to elaborate them further is ongoing.

In the UK, the ONS envisages the following three families of consumer price indices for the future:

This acknowledges the fact that different purposes can require different consumer price indices.

1.4.4 Web scraping and scanner data

There will be changes in how prices are collected. The ONS is already experimenting with web scraping—automatically collecting prices from websites. The advantage of this is that it gives much more data on which to base the index. The disadvantage is that the data collected are very erratic, and none of the elementary indices currently used appear to cope easily with the data. More complex forms of index are, therefore, being considered.

Potentially more valuable is the use of scanner or electronic point of sale data because this gives information about the quantity of purchases as well as the price; this could potentially eliminate the need for elementary indices and the problems they have caused. This is an exciting development. Until the passage of the Digital Economy Act 2017, the ONS had no power to oblige companies to share their data, but it is currently testing out datasets. It is also able to learn from a few countries that are more advanced in this area.

Like web scraping, scanner data require different handling and different formulae. Both are, of course, “big data” and there is much to learn about handling techniques. There must clearly be a learning process. But the potential for progress at a brisk rate is substantial. In Australia, some 20% of the consumer price index is now based on scanner data.

1.4.5 Inflation in the very long run

Download the Bank of England’s long-run data dataset

The first official series of consumer prices in the UK dates from 1914, although the first series of producer (wholesale) prices was published in 1903, using data going back to 1871. For periods before that, economic historians have recently done a lot of work in estimating indices for various periods from available reports of prices in different markets, purchasing records of institutions, and so on. Such indices have been combined into longer series. The ONS calculated consumer prices back to 1750 by chaining available indices but the Bank of England’s ‘A millennium of macroeconomic data’ research database goes back much further, to 1209 (as shown in Figure 1.6).

Figure 1.6 Consumer prices since 1209

Price index composite series 1209 to 2015, UK, 2015 = 100, logarithmic scale

Price index composite series 1209 to 2015, UK, 2015 = 100, logarithmic scale

The figure shows the series plotted on a logarithmic scale, so that growth rates over time can be compared. It can be seen that the sharpest rise in prices has occurred over the past 100 or so years. Before that, prices rose fairly rapidly during the 16th and early 17th centuries (the great Tudor inflation). The Napoleonic wars and the First World War also brought spikes in inflation—for a number of reasons, wars can create shortages of consumer products so that prices rise—with prices falling afterwards, at least partly because of economic depression.

So it does appear that the last century has been more inflationary than the norm in previous centuries. However, these long-term charts hide shorter-term variations. As the main text shows, different periods have seen very different inflation rates within the last century.

Note: the series appear much smoother over the last century, without the erratic oscillation of previous periods. In part, this may be due to a more comprehensive database becoming available so that estimates are less affected by erratic price movements in individual products. However, it may also reflect reality to the extent that the modern economy is less reliant on erratic factors, such as variable harvests.

1.5 Other measures of inflation

1.5.1 Producer price indices

Before retail price indices, statisticians compiled wholesale price indices. They are some of the oldest official statistics in the UK. They were first published—albeit with limited coverage—in 1903, with data going back to 1871.

producer prices
The prices at which firms buy and sell to other firms; also known as factory gate prices. The prices are divided into two indices: the price of inputs used in production processes, and the price of outputs produced.

Today they are known as producer price indices (PPIs). Producer prices measure the prices firms pay, and the prices at which they sell their goods and services to other businesses. The basic principles and concepts of the current series were essentially set in the 1950s.

PPIs help us understand the buildup of inflationary pressures in the economy. If firms face higher costs, they may in turn increase the prices they charge consumers, which increases consumer price inflation. PPIs can, therefore, give us a clue about how consumer prices might develop in future.

Two basic types of PPIs are compiled: input prices that measure prices of materials and fuels bought by UK manufacturers, and output prices (also known as factory gate prices) that measure the price of manufactured products. As well as being an aid to forecasting inflationary pressures on consumers, PPIs can also be used in contracts for uprating quoted prices. Output prices can also feed into deflators used to measure the output of parts of the economy.

The factory gate price is the amount received by UK producers for the goods that they sell to the domestic market. It includes the margin that businesses make on goods, in addition to costs such as labour, raw materials and energy. It also covers interest on loans, site or building maintenance, and rent. The indices measure the price of goods produced by UK manufacturers at the point they enter the distribution chain. Note that the indices do not include any distributor (wholesaler or retailer) margins.

The input price measures the price of materials and fuels bought by UK manufacturers for processing. It includes prices of imported and domestically sourced materials and fuels. It is not limited to materials used in the final product, but includes what is required by businesses in their normal day-to-day running, such as fuel.

A further division is made in each case between gross and net price indices. Gross indices for each sector include prices of products sold by one business to another within the same sector; net prices exclude intra-sector transactions. Net prices, therefore, give a better picture of the impact of producer price inflation on the rest of the economy.

As with other price indices, PPIs work on the basket of goods concept. The basket of goods is a wide collection of representative products and their prices, collected each month. The movements in these prices are weighted to reflect the relative importance of the products in a chosen year (known as the base year). In this, the principles are analogous to those of consumer prices: we take the prices businesses pay or charge and weight them together. Instead of using expenditure weights, we use the share of a particular item used in production or sold to other businesses.

Most of the basic data are provided from a sample of manufacturing companies. Prices are collected under the Statistics of Trade Act 1947, so responding is compulsory. Currently, around 6,750 price quotes are collected monthly from around 4,000 respondents. The sampling framework is designed both to ensure adequate cover (in terms of both sales and respondent number) of each industrial group, as well as to provide good overall coverage and ensure coverage of different size companies. Normally, each respondent provides only one price quote to avoid over-reliance on the pricing policies of any one company. Quotes must adhere to the following rules:

Sampling is rotational to avoid excessive burden of reporting on individual firms; a fresh sample is selected each year.

A small number of products are collected from other sources where this is appropriate: for example, prices of meat products are taken from Smithfield Market, and the Forestry Commission provides prices for forest products.

Once prices have been collected, they are subject to the usual checks to examine any changes that appear unusual. The UK Manufacturers’ Sales by Product (PRODCOM) survey is conducted annually to measure the production of industrial products. Weights for the individual products, and for combining products together into higher-level categories, are derived from PRODCOM inquiries.

The procedure for aggregating follows the normal process for price indices, with quotes for products being combined initially into indices for individual products, then into groups of products at lower-level categories, next into more aggregated (higher-level) categories, and finally into the overall indices. (See Section 1.3 on compiling consumer price indices.)

Important series are seasonally adjusted. After publication, PPIs are subject to revision, typically for up to five months, to take account of later information.

As can be seen from Figure 1.7, the input series is considerably more erratic than the output series. This is because it is directly influenced by commodity prices, which are often highly variable, and also by exchange rate movements. The chart shows how the movements in the input series are typically reproduced in a very muted form in the output series.

Figure 1.7 Input prices are more erratic than output prices

Input and output PPI, 2003 to 2018, UK, 2010 = 100

Input and output PPI, 2003 to 2018, UK, 2010 = 100

Office for National Statistics – ‘Producer price inflation’

1.5.2 Services producer price indices

In the UK, as in other countries, statistics on service industries were for many years the poor relation of those on production industries. Indeed, they still are, although there have been substantial and ongoing improvements. There were two basic reasons for this. First, in the past—and, indeed, until relatively recently—services were often regarded as an adjunct to the production sector, rather than as industries in their own right. (This was somewhat curious in retrospect because the UK services sector exceeded the size of the production sector as long ago as 1950.) Second, collecting information on services is often more difficult than for production industries. Services may not be easily defined, may not be standard, and may change more frequently.

services producer prices
The prices at which services, which make up 80% of the UK economy, are sold to other businesses. This is difficult to calculate, as services can be hard to define and tend to change frequently in non-obvious ways. The index is calculated quarterly.

This relative lack of coverage applies to prices of services as much as to any other services sector statistic. Prior to the current programme of improvement that started in 2017, services producer price indices (SPPIs) were based on 4,200 price quotes—compared to 6,750 quotes for the PPIs—and covered only 59% of services industries, despite the services sector accounting for nearly 80% of the economy. The data are compiled on a quarterly basis only, compared to monthly data for PPIs.

The current programme of improvements will see the number of price quotes rise to around 6,000 per quarter. Sample structure is also being improved as a result of the ability to use improved data on services sector turnover. The improvements have been carried out in waves, with the final wave due to be incorporated in the April 2019 release.

The improvements also include changing the scope of the survey to measure business-to-all activity, instead of business-to-business. Currently, the SPPI includes only services sold by UK businesses and purchased by UK businesses and government. The change will broaden coverage to include transactions between UK businesses and all sectors of the economy. This will require, not only the addition of domestic consumers, but also export of services where appropriate.

The aggregate SPPI is produced on both a gross and a net sector basis. The gross sector aggregate is calculated using weights based on sales from all business to business transactions within the UK and so reflects the services-sector inflation experienced by all UK businesses. The net sector aggregate is created using weights based on sales from transactions to businesses outside the service sector, thereby giving a measure of inflation showing the impact of services-sector inflation on businesses in other sectors.

There are essentially two methods of price measurement used in the SPPI. The first, and most commonly used, method requires respondents to regularly provide a price for the same clearly specified service each quarter. Quotations are supplied for services that have the following characteristics:

As with other price indices, it is crucial to ensure that the price collected each quarter is for exactly the same service. This ensures that only genuine price movements are recorded, rather than price changes that are due to a change in the service provided. But, of course, this is easier said than done. Services constantly change their nature under the pressure of innovation and changing circumstances. Coping with the challenges of measuring such changes is one of the major areas of interest in economic measurement research.

The second method of collecting prices measures the cost of time spent providing the service as a proxy for an actual transaction price. This is only used where it has been identified that providing a quarterly price for a repeatable, clearly defined service is not possible, and that the cost of time spent in service provision is a reasonable proxy.

Once they join the group of firms supplying information to compile the SPPI, the contributors are asked to identify the service areas in the industry aggregation structure where their businesses are represented. They provide a list of grades/personnel that provide services in this area, the number of hours they have worked in the quarter, either a standard or realised charge-out rate, and the total fees received for the service area during the quarter.

The hours worked by each grade in the period when the firm first joined the reporting group are then considered to be the base hours for those grades. These are used as the standard hours for each grade or member of staff in future quarters. The respondent is asked to consider the same service characteristics as stated for returns based on the pricing of a clearly specified service. Changes to hourly rates are then monitored each quarter. The hours provided in the base period are essentially used as weights, to ensure that any changes in rates have a greater impact on the price change measured for each contributor than grades/personnel that contribute fewer hours. This should reflect the impact of these rate changes on the prices received by these contributors.

Once the basic data have been collected and validated, the process of aggregation follows, essentially the same steps as for the PPIs.

1.5.3 Export and import prices

export and import price indices
Exports are manufactured in the UK and destined for firms in foreign markets. Imports are manufactured overseas and purchased by UK firms. To compile the indices, the prices paid are measured directly.

These series have been produced since 1993 and are compiled in a similar way to PPIs. In this, they differ from import and export unit value indices, which are derived by dividing imports or exports in nominal terms by the corresponding volume measures. Instead, the export and import price indices are the results of surveys that measure prices directly. (Further discussion is to be found in the later chapter on trade statistics.) Like PPIs, they are collected under the Statistics of Trade Act 1947, so response is compulsory.

Export price indices (EPIs) cover prices of products manufactured in the UK but destined for export markets. Quotes are requested in the currency the overseas customer pays and are, ideally, collected on a free on board basis (excluding the costs of transfer from the UK to the overseas destination).

Import price indices (IPIs) cover the prices of imported raw materials and semi-manufactured products. Respondents are selected from a list of traders supplied by HM Revenue and Customs (HMRC). They can either be manufacturers in their own right or importers acting as intermediaries. In either case, the price collected is that charged to or paid by the UK manufacturing company.

As with PPIs, some prices are collected from central sources.

Until 2017, the EPI was based on around 2,100 price quotes and the IPI on around 1,900. Improvements to sample sizes and the sampling framework are underway so that both surveys will be based on around 6,000 price quotes. This will increase coverage from 57% to 76% (EPI) and from 41% to 67% (IPI).

1.5.4 Construction output price indices (OPIs)

construction output prices
The costs related to construction projects, covering materials, plant, labour and an allowance for companies’ margins and profits.

Construction output price indices (OPIs) estimate costs related to different types of construction projects, and provide aggregated measures for all new work, repair and maintenance, and other construction work. Collecting data from construction companies is problematic because of the nature of the sector; there are many small-scale or single-person businesses and such businesses are sometimes not very long-lived. OPIs are, therefore, compiled in a different way from other business price indices. Indices are compiled for different forms of construction—for example, housing, infrastructure and private industrial—and separately for new work and for repairs and maintenance.

For all types of construction, three types of input costs are estimated: materials, plant and labour. A markup is applied to allow for companies’ margins and profits. Materials costs are estimated using individual producer price indices (PPIs); plant costs are measured using the services producer price index (SPPI) for construction plant hire; labour costs are measured using the average weekly earnings (AWE) index for construction. These are weighted together to provide an index to estimate changes in construction output costs for the product.

Revisions are subject to the revisions policies of component series; for routine revisions, the series remains open for a period of five months, in line with PPI—its main component.

At the time of writing, these statistics are considered experimental.

1.6 House price indices

There are a number of house price indices published in the UK. The ONS publishes a series—in conjunction with HM Land Registry (which covers England and Wales), Registers of Scotland, and Land and Property Services Northern Ireland—known as the UK house price index (HPI). In addition, there are important and well-followed indices published by private organisations. These include the two oldest series—those published by the Halifax and Nationwide Building Societies. Other well-known series are published by Rightmove and LSL Acadata.

According to an article published in the Daily Telegraph in August 2018, there are more than 20 measures of changes in house prices in the UK.

These series have different characteristics, based not only on the method of compilation, but also on the source of the data.

It is possible to follow trends at different stages of the housing market from initial offer, to sale, through to completions. Together, the different indices provide a more comprehensive view of the housing market than would one index alone.

1.6.1 The Rightmove house price index

This is based on dwellings advertised on the Rightmove website; thus, it reflects asking prices for properties. It covers Great Britain (it excludes Northern Ireland). As well as a national index, it provides indices for Scotland, Wales and regions of England, with a particular focus on London.

The Rightmove index is published during the month in question. Rightmove only publishes a seasonally unadjusted series and does not revise.

The Rightmove index is the most timely, but reflects asking prices—which are often not the actual sale price. It may also include dwellings offered for sale but that are not, in the end, sold.

Rightmove updates weights every quarter. It uses a mix adjustment technique, calculating the average price for each property in areas defined by postcode.

1.6.2 The Halifax and Nationwide house price indices

These are based on their own mortgage data (with adjustments to compensate for bias in their coverage) and reflect mortgage approvals. Both publish a monthly index for the UK, with periodic information on specific areas, regions, towns or local authorities. Both these series necessarily exclude houses purchased by cash transactions.

Nationwide and Halifax publish one week after the end of the month. They do not revise their seasonally unadjusted series, although their seasonally adjusted series are revised. The indices reflect mortgage approvals, which are more up to date than completed sales, but may also include approvals that do not come to completion. They also have limited coverage, being based on their own data only—effectively around 12,000 transactions per month (Nationwide) or 15,000 per month (Halifax).

Nationwide updates its weights every two years, while Halifax weights refer to 1983. They both use hedonic adjustments, defining each property in terms of characteristics—such as location, number of rooms, age of property, etc—and then using a regression model to estimate the price of each feature in a particular month.

1.6.3 ONS HPI

The ONS HPI uses Land Registry information, based on actual completed sales data, and has wide coverage—about 100,000 transactions a month. Because of the delays that sometimes occur in sales being actually registered, it is published six weeks after the end of the month. The HPI is open for revisions for up to 12 months to take account of delays in reporting to the land registries. Consequently, the HPI has effectively complete coverage, including cash purchases, once revisions have been made. It reflects actual sales prices, but is the least timely of the series.

Indices are published for the separate countries of the UK, for regions of England, counties and local authorities.

Weights are updated annually. It also uses hedonic adjustments.

1.6.4 The LSL Acadata index

The LSL Acadata index also uses Land Registry information, about 80,000 transactions monthly. LSL Acadata uses modelling techniques to forecast the latest month. The LSL Acadata series is reasonably complete for England and Wales and is somewhat more timely, but latest data are part-estimated. There is a separate series for Scotland. Data are also compiled for Wales, England and regions of England, as well as information for counties, London boroughs and unitary authorities.

You can find data for these series online:

Due to its forecasting technique, Acadata publishes its initial estimate for each month, subsequently revised, two to three weeks after the end of the month. Afterwards, the LSL Acadata series is open to full revision.

Weights are updated annually. It uses a mix adjustment technique, similar to Rightmove, but adjusts by local authority area.

Index Data source Number of transactions Coverage Stage of recording transaction Adjustment methodology Weights
UK HPI Registration data from HM Land Registry, Registers of Scotland, and Land and Property Services Northern Ireland Around 100,000 a month UK (all transactions) Registration of sale Hedonic regression Updated annually
Nationwide Nationwide mortgage lending Around 12,000 a month UK (mortgage transactions) Mortgage approval Hedonic regression Updated every 2 years
Halifax Halifax mortgage lending Around 15,000 a month UK (mortgage transactions) Mortgage approval Hedonic regression Last updated 1983
LSL Acadata HM Land Registry price paid data Around 80,000 a month England and Wales (all transactions) Registration of sale Mix adjustment Updated annually
Rightmove Advertised properties on Rightmove portal Around 100,000 a month Great Britain (all transactions) Advertised date Mix adjustment Updated quarterly

Table 1.5 Summary of HPI characteristics

Summary of HPI characteristics

HM Land Registry (2018)

1.6.5 Average house price indices

All these sources produce—in addition to an index of average house prices (measured in pounds)—a range of other information. One piece of information they all publish is average house prices. Rightmove and Acadata use arithmetic means for this, while the other series use geometric means. As explained earlier, geometric means give lower results than arithmetic means because arithmetic means are sensitive to higher values.

Figure 1.8 (produced by HM Land Registry) shows average house prices for the five series described. It can be seen that the series for Rightmove and Acadata are higher than the other three, because of the use of arithmetic and geometric means. In addition, the Rightmove series shows seasonal variation and covers asking prices that are frequently higher than actual sales prices. It should also be noted that Rightmove and LSL Acadata refer to England and Wales only (Scotland was only added to Rightmove data from January 2018), whereas the others cover the whole of the UK. Nevertheless, all series show broadly similar trends.

Figure 1.8 House price indices do not coincide

Average House prices indices for the UK, 2005–2017, £

Average House prices indices for the UK, 2005–2017, £

Gov.uk – ‘Comparing house price indices in the UK’. Note: this graph is a reproduction of the original. (Contains public sector information licensed under the Open Government Licence v3.0.)

1.7 Summary

1.8 Further reading

Notes

  1. ILO, IMF, OECD, Eurostat, UN, The World Bank (2004), Consumer Price Index Manual: Theory and Practice, International Labour Office, page xix 

  2. ILO, IMF, OECD, Eurostat, UN, The Word Bank (2004), Consumer Price Index Manual: Theory and Practice, International Labour Office, page 17 

  3. Ministry of Labour and National Service, Cost of Living Advisory Committee: Report on Proposals for a New Index of Retail Prices, March 1956, Cmd. 9710, page 16 

  4. Council of the European Union – ‘Council Regulation (EC) No 2494/95 of 23 October 1995 concerning harmonized indices of consumer prices’, Office Journal of the European Communities, No L 275/1 

  5. ONS (2014), Consumer Price Indices Technical Manual (2014 Edition), page 50, Office for National Statistics