Appendix 6 Measurement challenges during the Covid-19 pandemic

Sumit Dey-Chowdhury

A6.1 Introduction

The coronavirus pandemic has led to the largest contraction in the UK economy in more than 300 years. It has also led to conceptual and practical challenges in how national statisticians measure the economy, including GDP, inflation and the labour market. Their statistical frameworks have helped to understand how the pandemic has affected the economy. They have also highlighted where further research might be needed on how we measure it.

Below we detail some examples of the measurement challenges that statisticians have faced as a result of Covid-19.

A6.2 Non-market output in GDP

ONS Resource

This note from the ONS explains how its approach to measuring education output evolved during Covid-19.

ONS, Coronavirus and the impact on measures of UK government education output: March 2020 to February 2021

Non-market output comprises the production of goods and services by the government that are either supplied for free, or at prices that are not economically significant. For example, the provision of health and education would be primarily considered as non-market output in the UK.

Traditionally, we have estimated education output in the UK using volume indicators, such as the number of full-time equivalent students. This would typically be a reasonable estimate of how much teaching is provided over the year. However, over the course of the pandemic, when schools were often closed as part of the public health response, estimating the volume of education output based on the number of students would not capture the impact on the provision of education.

Between March and July 2020, schools were largely closed in the UK, with teaching primarily being delivered by remote learning. Between September and December 2020, schools reopened for in-person attendance, although not at pre-pandemic levels. There were large-scale school closures again between January and March 2021. At the time of writing, there has been a return to in-school learning with some social distancing measures still in place.

The ONS coped with this using estimated full-time equivalency (FTE) values of remote learning while schools were closed. This accounted for the percentage of normal teacher input, with the proportion of instruction and teaching given by parents. An FTE of 1 would assume that remote learning is a perfect substitute to in-school learning, whereas an FTE of 0 would assume no output at all, with real-world FTE values between these extremes acting as a way to “discount” the time that students spent studying.

To calculate these FTE discount values, the ONS collected timely and relevant information from teachers (Figure A6.1). This estimated the percentage of normal teacher input and the proportion of instruction provided by parents. To calculate the FTE factor for each remote learner:

\[\%\text{ of normal input} × \left(1−\text{proportion of instruction dependent on parents}\right)\]

This meant that the effects of the pandemic were reflected more accurately than would have been the case if the ONS had reported only volume indicators. Figure A6.1 shows that remote learning has become more equivalent to in-school, teacher-led learning during the pandemic.

  Pre‑pandemic Apr to Jun 2020 Sept to Dec 2020 Jan to Feb 2021
In-school attendance 95 1 − 10 52 − 94 3 − 19
Primary School FTE   29 − 40 33 − 37 57 − 59
Secondary School FTE   56 − 63 62 − 67 83 − 86

Figure A6.1 FTE of educational output in England, percentage

FTE of educational output in England, percentage

Note: There was a pre-pandemic absence rate of 5%

A6.3 Inflation

Approximately 80% of the price quotes in the consumer prices index including owner occupiers’ housing costs (CPIH) basket, 45% by weight, were collected in stores across the UK monthly, prior to the pandemic. Measuring headline inflation measures during the pandemic became problematic because it was impossible to do in-store price collection when the stores were closed, and so many price quotes were collected from the websites of retailers or by telephone.

Not all goods or services were “available”. Available items are those for which the goods or services could still be purchased by consumers, although it may not have been possible to collect their prices. These prices were estimated based on the index immediately above it in the classification structure; the price movement of a similar item; or carrying forward its price.

In contrast, “unavailable” goods and services were those for which consumers could no longer access the market because they had effectively been shut down, for example, hairdressers. Those price movements were imputed so that they would have no impact on the headline CPIH figure, that is, the CPIH reflected only the price movements of those goods and services that consumers could purchase.

The consumer price index (CPI) is based on the relative change in prices, which is then weighted by its share of expenditure in the base period. Each year the expenditure weights are updated, alongside the contents of the basket of goods and services. As such, the CPI estimates for 2020 were based on pre-pandemic spending patterns for 2018:

\[I_{t,0} = 100\times\frac{\frac{\sum}{i}\left(\frac{P_{it}}{P_{i0}}\right)w_{i}}{\frac{\sum}{i}w_{i}}\]

where:

Typically, expenditure weights do not change considerably from year to year. However, the impacts of public health restrictions and voluntary social distancing led to unprecedented changes in patterns of consumer spending. For example, haircuts, foreign holidays, and theatre admission were no longer available, while households were spending a higher proportion of their income on food and non-alcoholic beverages. Figure A6.2 provides experimental estimates of the change in consumer spending from April to December 2020, indicating large and volatile shifts in consumer spending through 2020.

  Normal weights1 Apr May Jun Jul Aug Sep Oct Nov Dec
Food and non-alcoholic beverages 79 135 127 109 88 87 81 85 100 94
Alcohol and tobacco 32 65 60 51 40 37 30 36 49 46
Clothing and footwear 51 30 35 49 53 56 53 56 55 72
Housing and household services 296 453 420 362 310 304 288 302 328 291
Furniture and household goods 50 39 45 59 62 59 59 70 78 67
Health 22 14 15 18 25 23 23 25 27 26
Transport 120 50 69 108 125 116 161 120 107 106
Communication 17 27 25 21 18 18 17 18 20 17
Recreation and culture 136 66 73 86 103 103 102 104 103 115
Education 24 40 35 30 25 25 24 24 28 24
Restaurants and hotels 96 10 12 20 67 88 83 75 17 57
Miscellaneous goods and services 77 72 84 86 83 83 78 85 89 86
  1. “Normal” weights refer to the official consumer prices 2020 weights. 

Figure A6.2 Experimental estimates of the change in consumer spending from April to December 2020

Experimental estimates of the change in consumer spending from April to December 2020

Notes: These are the 2020 annual CPIH expenditures, based on 2018 Household Final Consumption Expenditure data. They have been uprated by the monthly change in expenditure in 2020 based on a range of timely information for this period.

ONS Resource

This note from the ONS explains the effects of reweighting the UK consumer prices inflation basket accounting for changes in consumption patterns during the coronavirus (COVID-19) pandemic on the annual rates of the Consumer Prices Index including owner occupiers’ housing costs (CPIH).

ONS, Effect of reweighting the consumer prices basket during the coronavirus (COVID-19) pandemic: October to December 2020.

Through the pandemic, official estimates continued to be measured under a fixed basket approach in line with international guidance. Items that were unavailable to consumers during periods of movement restrictions have been imputed so as to make a negligible contribution to the all-items inflation rate.

However, experimental estimates were also published, which aim to understand the effects of these large changes in consumer spending patterns. The latest estimates show that the annual rate of the reweighted basket was very close to a “re-chained” version of the official rate, although “there remains considerable variation in the scale and make-up of consumer spending compared with the pre-pandemic period” (ONS 2021).

ONS Resource

The latest international guidance recommends that in compiling CPI estimates for 2021, expenditure weights are adjusted so that these better reflect base-period spending patterns, where lockdown restrictions have had an impact in 2020, rather than using pre-pandemic 2019 expenditure, which would have been the case under normal practices. ONS updated expenditure weights reflect the unprecedented changes in consumer expenditure for the most heavily affected spending types through the pandemic, such as restaurants and cafes.

A6.4 The labour market

Estimates of employment, unemployment and inactivity have traditionally been based on the Labour Force Survey (LFS). These household survey responses are then weighted up to the official UK population. First interviews of the LFS have traditionally been conducted face-to-face, with subsequent follow-up telephone interviews. Due to coronavirus restrictions, face-to-face interviewing was suspended in March 2020. The shift to telephone interviewing increased non-response in the sample (See Appendix 1), especially among the non-UK born population.

The LFS does not provide a count of the number of migrants in the UK, but a proportion of the population that are migrants. This population value is fixed to a 2018 estimate, which was taken from before the pandemic, so does not reflect any changes on migration flows from the pandemic. If a smaller share of the population is estimated to be migrants, there will be a decrease in the migrant population and an offsetting increase in the UK-born population.

Therefore, a fall in the migrant share in the LFS might reflect a relative change in the rate at which overseas-born people participate in the LFS. This fall in response rate of the non-UK born population and the fixed population explains why the LFS estimates a decrease in the migrant population of 1 million between Quarter 1 and Quarter 3 of 2020, as well as a similar rise in the number of UK-born residents.

Throughout the pandemic, HM Revenue and Customs’ (HMRC’s) Pay As You Earn (PAYE) Real Time Information (RTI) figures have provided the number of payrolled employees. Unlike the LFS, the RTI figures record the entire population that is on payroll, which has showed a much larger fall in the number of employees (Figure A6.1). Between March and June 2020, RTI estimated a fall of 650,000 employees, whereas the LFS initially estimated a smaller fall of 180,000 between March and May 2020. Part of this is likely to be explained by potential overestimates in the size of the population, on which the LFS figures are based.

Given the uncertainties in the LFS estimates, the RTI payroll information has underpinned a recent reweighting of the LFS. This will better reflect changes in international migration and other effects of the coronavirus pandemic. In July 2021 new modelled population totals for UK, EU and non-EU country of birth were introduced into the official labour market figures. Evidence from HMRC RTI implied that the actual population may not have moved in line with the mid-year population estimates used previously. The findings show that the impact on the headline measures of rates has been relatively small, but that there has been a larger impact on estimates of levels and changes in levels (Figure A6.3).

Figure A6.3 LFS and RTI employee estimates before and after the onset of Covid-19 pandemic, UK
Index: 2019=100

LFS and RTI employee estimates before and after the onset of Covid-19 pandemic, UK Index: 2019=100

Notes:

  • These estimates are based solely on the number of employees from the Labour Force Survey (LFS) and the HM Revenue and Customs’ (HMRC’s) Pay As You Earn (PAYE) Real Time Information (RTI) system. Other employment group estimates from the LFS are not included.
  • Index figures are based on three month rolling averages of employment figures (the month denotes the first month included in the rolling average).
  • Another factor might be the large pickup in the inflow of workers from self employment to employees, which might be because of how respondents to the LFS have changed their perceptions of the employment status.