Chapter 12 Hard-to-Measure Statistics

Diane Coyle

Which statistics are hard to measure?

How should global supply chains be measured?

How do statistics account for rapid technological and quality change?

Do we need to think beyond GDP?

How can measurement be improved?

12.1 Introduction

The standard treatment for wet age-related macular degeneration (wAMD), a serious eye condition, has for many years been an injection of Lucentis into the affected eye. The cost is over $2,000 per injection, and some patients need an injection each month. The treatment is highly effective. However, another medication, Aventis, made by the same company, Genentech, was found in the early 2000s to be a less invasive and far cheaper effective treatment, costing less than $150 per injection. However, Aventis has only been licensed for treatment of certain cancers; its use for wAMD is “off label”. Genentech has tried to limit the use of Aventis for eye treatments, citing the safety issues in repackaging the drug. A trial comparing the two, though, found they were equally effective.

In these times of consumer excess, gifts involving experiences are becoming more popular. These can range from theatre tickets to skydiving. It is even possible to spot quirky gift opportunities while scrolling through Instagram. How about a diet plan from @dietplanadvice, @mealplanmagic or @healthyfoodadvice to set a loved one up for their New Year resolutions?

Many people know the challenge of explaining exactly what they do at work all day to their elderly relatives. Many job titles, especially if linked to digital technology, are new: Product Manager Personalisation; Data Scientist; Social Media Manager; Backend Java Developer; EKS Implementation Consultant; Lead Ruby on Rails Engineer. And the world of work is still continuing to change rapidly.

What links these three phenomena? Each reflects in a different way the profound changes in the economy due to technology. Technological innovation has been the driving force in economic growth since the Industrial Revolution.

It is difficult for economic statistics to keep up when the pace of change is very rapid:

There have been other eras when the statistical framework for the economy needed a significant update. For instance, the 1886 Annual Yearbook of Statistics for the UK shows that most of the official figures collected then – at the height of the Industrial Revolution, and more than 30 years after Charles Dickens had written Hard Times about the terrible conditions in the Manchester cotton industry – related to agriculture. The government had almost no information about mills, mines, canals and railways.

That sense of a gap between the modern economy and its measurement was underlined by a review of economic statistics conducted in 2016 by Sir Charles Bean. As he said at the time, “The faster the pace of change, the bigger these measurement issues will become, unless something is done. Measuring the economy has never been harder”.

You can watch this video of the conclusions of the Bean report.

The Bean report called for new approaches to measuring the digital transformation. One motivation for exploring the challenges is the disappointing productivity performance of the UK economy since the mid-2000s.

productivity gap
The productivity puzzle refers to the flatlining of UK productivity since the financial crisis of 2008 and 2009. The productivity gap refers to international comparisons of productivity, specifically why UK productivity growth has historically lagged behind that of developed countries.

Read about the UK’s productivity gap, and the competing explanations for it, in Chapter 6.

real GDP
Nominal GDP, adjusted for the effects of changes in prices. Also known as GDP in volume terms.

While there are many possible explanations for this, it makes sense to explore the extent to which measurement issues play a role in this disappointing performance, particularly as there seems to have been substantial digitalisation of the economy over that period, a point stressed in Chapter 2 of this book. A second motivation is a growing appreciation that although real GDP growth is often used as a rule of thumb for the change in economic welfare, it has significant limitations; there is growing interest in broader approaches to measuring economic welfare.

12.2 Which statistics are hard to measure?

This book has set out the frameworks that are used to measure the economy. It has discussed how economic output is measured, how we gauge the change in prices, and the metrics we apply to international trade. Increasingly, however, the economy is changing in ways that challenge those measurement frameworks.

The table below breaks down different goods and services depending on whether their price, quantity and quality is observed or not. In general, the easier it is to observe those characteristics of price, quantity and quality, the easier it is to fit them into our measurement frameworks.

Price observed Price unobserved
Quantity & quality observed Market goods
Some market services
Barter transactions
Quantity observed, quality unobserved Other market services Imputed transactions (owner occupied housing, FISIM)
Public services
Household production
Free digital services
Quantity & quality unobserved New delivery models (such as online travel agents) Crypto-currencies

Figure 12.1 Which parts of the economy have observed prices and quantities?

12.2.1 Measuring the price of goods

In the second half of the twentieth century, economic measurement was simpler than it is now. A higher proportion of consumer spending went on goods rather than services. In relative terms, the proportion of consumer spending accounted for by goods rather than services was higher, though as a matter of fact, the UK services sector exceeded that of goods as long ago as 1950. There was also less variety in the goods and services people were purchasing. On the whole, this meant it was easier to measure revenues and prices. In addition, it was more reasonable to assume that the quality of most goods was not changing all that rapidly, and it was therefore easier to calculate a “real” or volume terms output as well. Volume terms output takes out the effect of price changes, as discussed in Section 2.2.3. This situation is represented in the top row of Figure 12.1.

Economic measurement has always involved some challenges, of course. For instance, barter transactions do not involve a price; but they are small scale in a developed economy like the UK.

12.2.2 Imputed transactions in GDP

imputed transaction
The process of “inventing” a transaction where, although no money has changed hands, there has been a flow of goods or services. It is confined to a very small number of cases where a reasonably satisfactory basis for the assumed valuation is available. For example, imputed rent is the amount that an owner-occupier would have to pay in rent to achieve the same consumption of housing services. It can be thought of as the amount that owner-occupiers (that is, non-renters) pay themselves for the housing services that they produce. While this concept is important when measuring economic output, it is not directly observed expenditure by homeowners.

As we move down the table, we find some imputed transactions that are also included in GDP, the most important being the notional rent that owner-occupiers are assumed to face – a figure calculated with reference to market rents. It can be thought of as the amount that non-renters pay themselves for the housing services that they produce. The rationale is to ensure that shifts in housing tenure, as has happened recently in the UK with a move from owning to renting, do not in themselves affect the GDP growth figures.

Imputed rental is calculated as follows:

Measurement of imputed rental is important to ensure that, for comparisons over time and between countries, the valuation of housing services is calculated on a consistent basis. Different countries and periods will have a different composition of owner-occupied housing and rentals. For example, two homeowners would still consume the same amount of housing services as if they both rented their homes to each other, and we need to include these services in the national accounts.

12.2.3 Non-market output

Public services are another prominent issue in existing GDP estimates, as they do not have a market price by definition – they are supplied free or at prices that are not economically significant. In total, such public services account for around a fifth of GDP. The conventions that are widely followed in estimating the volume of non-market output include:

ONS resource

ONS publishes estimates of some public services such as healthcare which do include allowance for quality change. These are published at Public service productivity on the ONS website. However, the European System of Accounts does not currently allow quality adjusted estimates to be included in GDP, for comparability reasons.

In the UK, we follow the principle of direct volume indicators in producing estimates of healthcare and education output. For volume estimates of healthcare output, the numbers of activities and procedures that are carried out are weighted together by the cost of each activity. The volume of education output is based on weighting together the number of full-time equivalent students in each type of educational setting by the costs of providing that education. 

12.2.4 Transactions without a price tag

The second row also has some important categories of market services we would want to include in GDP that have no explicit prices. One is a set of financial services known as Financial Intermediation Services Indirectly Measured (FISIM). Financial intermediaries charge for their services explicitly via commissions and fees, but also implicitly via an interest margin. These implicit transactions are financial market transactions without a price tag, conducted on the basis of a spread; money changes hands but the “price” has to be imputed.

Another category in the second row is household production – valuable activities carried out in the home, such as childcare and cooking. These are not included in the GDP figures, although they can substitute for marketed alternatives, such as nurseries and ready meals. The Household Satellite Account accounts for, and values, unpaid production activity. This includes childcare, adult care, household services, as well as unpaid nutrition, transport, laundry, and volunteering services. Measuring unpaid production and consumption provides a more complete picture of the activities that affect people’s well-being.

12.2.5 Zero-price digital services

Finally in the second row, and rapidly growing in usage, are zero-price digital services, such as online search or social media, which people pay for by consuming advertising but also indirectly through the prices charged for products advertised via these services. Much about these services is hard to observe and measure, including sometimes location of activity (domestic or overseas) as well as price and quality – although there are some indicators of the volume of usage such as volume of data used or number of users. How much of these services should be counted in GDP is also unclear.

12.2.6 The sharing economy and cryptocurrencies

sharing economy
The sharing of under-used assets through completing peer-to-peer transactions that are only viable through digital intermediation, allowing parties to benefit from usage outside of the primary use of that asset.

The last row includes some new digital services whose quantity and quality are both unobserved, at least with present methods of data collection, and whose price might or might not be observable. In these services, such as sharing economy platforms or cryptocurrencies, there is seemingly rapid growth in consumer use, switching between services both new and conventional, a lack of clarity about cross-border transactions and significant gaps in data collection. What’s more, some of the new digital services blur the boundary between what is in the market (and therefore GDP) and what is instead part of household production.

Figure 12.2 is a useful tool to illustrate the concept of the sharing economy.

Figure 12.2 Is a service part of the sharing economy?

Is a service part of the sharing economy?

The sharing economy takes place through the sharing of under-used assets by completing peer-to-peer transactions that are only viable through digital intermediation, allowing parties to benefit from usage outside of the primary use of that asset. Measurement considerations include:

Peer-to-peer activities such as car-sharing used to be regarded as “personal”, yet using an online platform to organise this activity can commercialise it. The platforms can also blur the difference between full-time and casual labour, as well as between employee and self-employed.

This chapter takes you through economic transactions such as those described above where, for various reasons, it is difficult or impossible to distinguish price, quantity and quality. These are intangible services or activities. Market prices of the services themselves are sometimes zero, due to the revenue model of their provider. Some of them overlap with household production, so there are some substitutions between activities inside and outside of the production boundary. And finally, it is not always clear where the transactions are taking place, or where the services are being provided.

12.3 Measuring things that are hard to measure

Earlier in the book, we described the approach used to measure GDP, and the central idea that we are trying to capture, which is value added. Prices provide the weight to attach to different goods and services to come up with a measure of the overall level of output of an economy.

12.3.1 The changing character of the economy

There are arguably two different challenges posed here:

To make things even more challenging, some phenomena span both categories.

The measurement challenges described here have come about because of the changing character of the economy, including the shift to services and intangibles, increased variety of goods and services, rapid technological change and digitalisation, and globalisation. Some of these challenges have been known for years, but have become more acute over time.

For example, in his 1994 address to the American Economic Association, Zvi Griliches designated a number of sectors as “hard to measure” because of the intangible nature of their output and the importance of quality as well as quantity:

“Our measurement and observational tools are becoming increasingly inadequate in the context of our changing economy,” he wrote. In earlier work he had addressed the problem of trying to account for quality change in constructing price deflators (see below).

12.3.2 Measuring GDP better

Government services are one example of an activity that does not fit easily into the simple framework, but where there are good reasons for including it in GDP. This is because public services can be close substitutes for private services (such as education), and for pragmatic reasons to allow comparability between countries or the same country over time. Furthermore, if GDP is about generation of added value, as Chapter 2 suggests, public services clearly do have value and should therefore be included.

Public services often have no market price, and historically, their output was measured by their inputs, mainly the amount spent on salaries in the public sector. This inevitably makes it seem that there is no labour productivity growth in public services, which is clearly incorrect – just think of the vast improvements in medical outcomes over the years thanks to improved know-how. To value public services better, it is therefore necessary to understand quality changes, which is often challenging.

ONS Resource

Public services are not covered in this chapter, but more information on the work that has been done to understand quality change, and therefore public sector output and productivity, can be found on the ONS website.

Another example of a hard-to-measure sector within GDP is the financial services sector. Although some financial products are reasonably standardised and have an observable price and quantity, such as the premium on a household insurance policy, many do not. For instance, in the UK, most retail bank accounts do not charge account holders any fees, so the banks cover costs by charging a higher interest rate on lending to customers than on borrowing from them. Although transactions are clearly occurring as money is deposited, withdrawn and loaned, there are no standardised units of volume or prices. Likewise with investment banks, there is just the spread between the “buy” and the “sell”.

The approach to measuring financial services in the System of National Accounts has been changed with every major revision of the definitions. The current approach is known as financial intermediation services indirectly measured (FISIM). It captures the value of those services by calculating the difference between the (sometimes implicit) rates of interest that financial institutions charge and the rates of interest they pay.

ONS Resource

The article ‘Financial intermediation services indirectly measured (FISIM) in the UK revisited’ explains developments in FISIM methodology.

This indirect approach has been criticised, not least because it indicated the financial sector was in a buoyant state just before the 2008 financial crisis, leading some observers to point out that it predominantly measured speculative trading. Andrew Haldane of the Bank of England observed in a 2010 speech, provocatively entitled ‘The Contribution of the Financial Sector: Miracle or Mirage?’, about the financial crisis, “At a time when people believed banks were contributing the least to the economy since the 1930s, the national accounts indicated the financial sector was contributing the most since the mid-1980s”.

While the methods to best reflect public and financial services in GDP are continuing to develop, there are areas where debate continues about exactly what should and should not be included. Below, some of the most challenging are discussed – all are areas of active research among economists and statisticians.

12.3.3 Accounting for intangible assets in GDP

The traditional conception of investment is spending on machinery and buildings, that is, investment in physical – or tangible – assets. But there is no reason for us to limit ourselves to these assets, as the definition of investment used in the national accounts is for “assets used in the production process for more than one year”, according to the European System of Accounts. This could clearly involve, for example, investment in research and development that gives rise to new products or new ways of producing existing products.

intangible asset
These are assets that do not have a physical or financial embodiment, capturing knowledge assets or intellectual capital. They include software, branding, design, and research and development, which contribute to the long-term accumulation of a business’s knowledge capital.

While therefore meeting the definition of investment, spending on intangible assets – those that have no physical or financial embodiment, but nonetheless meet the criteria for being assets – is hard to capture as we need to distinguish between what is “current” and “capital” spending. Expenditure on new assets is not “expensed”, i.e. not treated as intermediate consumption, which means it is not subtracted from GDP. As such, getting the allocation of spending between “current” and “capital” is important.

In the current national accounting guidance (see Figure 12.3), only spending on some intangible assets (also known as knowledge assets) is treated as investment. Intangible asset expenditure categorised as investment includes:

Many of us would consider other forms of spending on intangible assets to be investment, but in the national accounts they are treated as current spending.

Intangible asset spending categorised as current spending includes:

Broad category Type of intangible asset Description (from CHS) Capitalised in the national accounts?
Computerised information Software and databases This includes knowledge embedded in computer programs and computerised databases. Yes
Innovative property Research and development Yes
Mineral exploration and evaluation Yes
Entertainment, literary and artistic originals This includes knowledge acquired through scientific research and development, product development and non-scientific inventive and creative activities. Yes
Design No
Financial product innovation No
Economic competencies Branding No
Organisational capital This includes knowledge embedded in firm-specific human and structural resources, including brand names. No
Firm-specific training No

Figure 12.3 Framework for measuring intangible assets

While only spending on some intangible investment is included in the national accounts, estimates of all intangible investments are calculated by ONS and other research bodies. Chapter 2 sets out the framework for the output measure of GDP, which can be summarised as:

intermediate consumption
This refers to the goods and services that are consumed as inputs by a process of production.

GDP = Output – Intermediate Consumption

To calculate the investment in intangible assets, we need to understand how the data we collect fits into this equation. This relies on two approaches:

How it’s done Estimating own-account investment

When assets are created but not sold on the market, valuing them is difficult. This can happen with tangible assets (for instance, a company doing their own building renovation) but is more common for intangibles. Businesses and government regularly invest in bespoke software in-house, using software developers and other specialised workers, and also incurring overhead costs and using other assets (such as IT hardware) in the process.

To estimate the value of in-house investment (known as “own-account investment” in the national accounting terminology), the costs of production are estimated:

  • It is assumed that the labour costs are the largest portion of the cost, and data on this aspect are more readily available through labour force surveys.
  • Relevant occupations (types of workers) are identified through research, and their wages and salaries form part of the cost. Even the most relevant workers will not be producing in-house intangible assets all of the time, so wages are scaled down accordingly.
  • These costs are then scaled up to account for non-labour costs, such as overheads, goods and services, and cost of using other assets.
  • Finally, certain industries will produce the asset for sale, rather than to use in-house (for instance, the software industry will produce software mostly to sell), so an adjustment is made for this.

In the UK, the value of investment in tangible and intangible assets has been about the same in most years over the past two decades (see Figure 12.4). However, this is only true if you include the intangible assets outside of the national accounts, which accounts for around two-thirds of total intangible investment. The largest intangible investments are in training and organisational capital – both outside the national accounts. High-tech industries, such as information and communications and professional, scientific and technical industries, tend to invest more in intangibles than others. Some industries are more tangible capital intensive, such as the utilities and transportation industries.

Figure 12.4 Market sector intangible and tangible investment, current prices, UK, 1997 to 2016

Market sector intangible and tangible investment, current prices, UK, 1997 to 2016

12.3.4 Accounting for the value of data in GDP

Data is a particular type of intangible asset that has been growing in importance as the economy becomes ever-more digital. While there is doubtless some hype, it is clear that business and government organisations are increasingly using data to produce improved services, and people are purchasing growing amounts of data to consume online services such as social media, streamed or downloaded films, music and online games. As noted above, some investment in databases is captured in the measurement of intangible assets: the cost of preparing data in the appropriate format (but not the cost of acquiring or producing the underlying data) should be included, valued on a sum of costs basis. This may understate the importance of data in the modern economy, when data usage has soared, and many companies monetise the data they collect, for example by selling targeted advertising or by customising offers.

One option would be to include more of the costs of creating a usable database, such as data acquisition and storage. Often data is created by businesses as an asset they are producing themselves and will use over time; this treatment would increase measured GDP, and is similar to the treatment of software in the statistics. It might in fact be difficult to separate out the investments in data and in software and other intellectual property. But if the data is not used over time, instead being an input to the company’s production process – improving a service to customers, say – it is an intermediate good, and this treatment would not increase GDP.

12.4 New business models

New technologies are enabling companies to organise their production in radically different ways. Many of these new business models are difficult to measure.

12.4.1 Global supply chains

supply chain
The network of organisations that cooperate to transform raw materials into finished goods and services for consumers. Trade liberalisation, decreased restrictions on capital movement, and technology advances have allowed supply chains to become increasingly global.

As foreshadowed in the chapter on the balance of payments, Chapter 9, a business model that is often in the headlines but difficult to measure, given current data collection, is the creation of longer supply chains as firms increasingly specialise in one part or a few parts of the entire production process. This aspect of globalisation has been enabled by cheap and reliable information and communication technologies, as well as the building of physical logistics infrastructure and shipping.

Much trade in goods – around two thirds – consists of intermediate goods or components rather than finished products. Many familiar manufactured products are no longer “made in” one country. The iconic example is Apple’s iPhone, which is “designed in California”, assembled in China, and includes components from many other countries, from Germany to Korea. But the outsourcing of some production to a sub-contractor, either in the same or another country, has become widespread.

value added
The value of output minus the value of all inputs (called intermediate goods).

How widespread, though? Although data on value added in world trade, enabling tracking of trade in intermediates and final goods separately, has become available, its analysis is in its early days, although it will ultimately enable the tracking of production through international chains. Bart Los and Marcel Timmer of the Economic Statistics Centre of Excellence (ESCOE) – set up to implement some of the recommendations of the Bean report) – have attempted to do this.

Los and Timmer describe their report as “cleaning up terminology, standardizing concepts and more generally providing clear guidelines on which measure to use for what type of questions”. It helps policymakers (and us) understand the value added of intermediate goods.

12.4.2 New models of manufacturing

In addition to trade statistics, the classification of activities between manufacturing, distribution and even retail and other services is affected by the phenomenon of extended supply chains.

factoryless goods production
An extreme case of goods sent abroad for processing, where the physical transformation of the goods is 100% outsourced. This takes place when a resident firm owns the intellectual property (technology, know-how, product design, etc.) used in the production process, but fully outsources the material transformation process required to produce the output.
The process of adding one or more services to a manufactured good, such that services are in fact supplied by manufacturing firms. Selling products-as-a-service is a whole and distinct value proposition offered to customers where they are promised life-long support for periodical, monthly or yearly, subscriptions stated in service contracts.

Figure 12.5 shows traditional manufacturing, with three alternative models: factoryless goods production, contract manufacturing, and servitised manufacturing.

Figure 12.5 Extended supply chains blur the boundary between goods and services

Figure 12.5a Extended supply chains blur the boundary between goods and services

Figure 12.5b Extended supply chains blur the boundary between goods and services

Figure 12.5c Extended supply chains blur the boundary between goods and services

One estimate, based on scraping the text from business websites, indicates that depending on the sector, up to 18% of manufacturers in the UK and US are using contract manufacturers. Chemicals and life sciences are at the top of the list in the UK and electricals and life sciences/pharma in the United States.

12.4.3 Where do transactions occur?

The interaction between new forms of assets such as intangibles and new business models poses particular problems. For instance, if a Germany-headquartered car manufacturer contracts out production to a plant in the Americas, has a subsidiary in a low-tax country where it registers its designs and intellectual property or perhaps its finance arm and treasury department, and stores data via a cloud service provider such as Amazon Web Services or Microsoft Azure with a data centre in Belgium, then it is next to impossible to be sure that all the intangible flows and intra-company transactions are recorded in the “right” place.

The complexities of multinational operation have thrown up some striking statistics in recent times. For instance, Ireland’s real GDP increased by over 26% in 2015, an extraordinary and implausible figure. The main reason was that Ireland is an attractive low-tax jurisdiction for many companies to allocate their intellectual property, and the revenues its use generates.

The OECD note ‘Irish GDP up by 26.3% in 2015?’ explains Ireland’s jump in GDP in more detail, and concludes that intangible assets make it “much more important not to derive incorrect conclusions from the developments of GDP … For this purpose, one should rely on other indicators from the system of national accounts and look at broader measures of well-being”.

Even in the context of new trade wars, it is hard to see the global production chains created by multinationals being snipped apart once again, particularly in the case of intangibles. Globalisation in combination with digitalisation therefore poses a significant set of challenges for statisticians.

12.5 Quality change in technological products

In fact, the international guidance on constructing price deflators is clear that they should allow for quality change as well as observed price movements. But this is easier said than done and, in practice, many if not most deflators make no such allowance.

A final area of hard-to-measure phenomena that need to be accounted for in GDP is rapid technological and quality change, and the implications for calculating price indices. This has become a feature of the modern economy in which continual innovation has led to improvements in the quality of the goods and services produced. The measurement challenge is that technological products tend to change at a rapid pace over time – for example, mobile phones. It is, therefore, not always straightforward to estimate a like-for-like comparison of volumes of a product over time. We need to adjust for this quality change in producing accurate volume estimates of output. That is, we capture the price change in the deflator only. As noted before, some of the conceptual and practical challenges arising from digitalisation and rapid technological change are not new, but have become more pressing.

12.5.1 Collecting price data

Some are easier to deal with in principle, if not always straightforward in practice. For example, in Chapter 1, there is a description of some methods used to account for the changing quality of some goods. In addition, outlet substitution bias (when price indices do not collect data from new types of retailer such as online retailers or digital platforms) requires statisticians to stay on their toes about keeping up with such behavioural changes. Data collection for price indices has traditionally involved an intensive process of collecting prices from stores and using expenditure data to create appropriate weights for combining individual item or category prices into an overall index.

The fact that many prices are now available online offers an alternative and potentially faster (and cheaper) data collection method. But in this case the challenges may include not having revenue weights to construct the index, and having to cope with a vast number of slightly different or new products.

The ‘Billion Prices Project’, founded in 2008 by Alberto Cavallo and Roberto Rigobon, collects online prices daily. Using the data from this source, Cavallo argues that although price levels are not identical online and offline, they overlap sufficiently that statistical offices can consider using novel methods to collect price data and construct deflators in future.

However, even faster collection of data does not help with the problem of how to include new goods in a price index. The conceptual challenge is that a price index is meant to capture the change in prices of an array of goods such that deflated or real income keeps consumer utility constant. If new – or disappearing – goods are not taken into account, the price index will be biased. The standard approach statistical offices take is to update the “basket” of included goods annually, and compare prices for “matched” goods that are available in both periods.

12.5.2 Adjusting for quality

A similar issue occurs with goods whose quality is improving rapidly, so that the same price paid will buy a superior product in the later period.

hedonic adjustment
The approach currently used in the CPI to quality adjust prices of rapidly changing ICT products. This uses data on the characteristics of a product to identify changes in the implicit value of a comparable product through time, even where comparable products may not be available.

One approach to these is to calculate their ‘hedonic’ price, in other words to use regression analysis to relate the price to specified and measurable characteristics, such as the processing speed or memory of a computer or the screen resolution and camera quality of a smartphone.1 Hedonic regressions are, though, rarely used in practice by statistical offices as they involve additional data collection and processing, so the method tends to be confined to a few product categories such as certain consumer electronics or cars.

It is possible, though, to describe the biases in price deflators due to new products and quality change.2 The biases depend on the extent to which new products are good substitutes for old ones – how “new” are they really?

scanner data
Detailed data on sales of consumer goods obtained by scanning the bar codes for individual products at electronic points of sale in retail outlets. The data can provide detailed information about quantities, characteristics and values of goods sold as well as their prices. Scanner data provide a great deal of information on the number and type of products sold.

Researchers are also taking forward work on improving the incorporation of new and better products using scanner data.3

However, the methods being considered require a considerable amount of estimation work and still do not cope well with the introduction of substantially new products or services with no older comparators. The fundamental problem is the conceptual basis of a deflator: what does it mean to leave consumer utility unchanged from the previous period when there has been significant technological advance? The underlying economic theory makes the assumption that consumers have preferences over all goods and services, past, present and future, but this is clearly not the case. Economic measurement may be able to improve price indices in practical ways, but it will not solve the conceptual puzzle.

How it’s done The price of telecoms and cloud computing

Some recent research has started to deliver improvements in specific price indices. One example concerns the price of telecommunications services. The previous ONS deflator indicated that this price was flat for years, whereas telecoms engineers pointed out that this failed to take account of substantial quality improvements (in compression, bandwidth, latency) and massive increases in the volume of data communicated across telecoms networks.

In producing a telecoms deflator, we look to consider the various services that are offered. We estimate the price change in each one and then weight these together to reflect the composition of telecoms services. These include a mix of traditional services such as voice calls and newer data-based services. The challenge is that the existing output deflator gives higher weight to traditional voice and text (SMS) services, whose prices have not changed as much, while broadband and mobile data prices have fallen more rapidly. The effect is that this implies a slower rate of fall in the prices of these telecoms services than is the case in practice. This in turn leads to a slower growth in the real-terms output and productivity of the sector, in contrast to the considerable usage growth and experience of service improvements.

Mo Abdirahman and other authors have proposed alternative telecoms deflators that capture these “new” data services and increased data usage that comprise the full range of output produced by the telecoms industry. These look to adjust for the quality changes by considering how these services are priced, and using value or volume of data usage as alternative weights. This helps to allow for the improvement in the telecoms services that are being offered, and so better capture the price change over time.

The range of deflators show a 35% to 90% decline over seven years to 2017, rather than zero. The more weight each index puts on volume of usage rather than revenues from the component services, the faster the estimated price decline, as “traditional” non-data services remain more costly per bit of data, and thus provide a relatively high share of revenues in the sector.

Figure 12.6 The range of potential telecoms deflators

The range of potential telecoms deflators

Another instance is the price of cloud computing, when businesses run software and store data with a cloud provider such as Amazon Web Services, Google, Microsoft or IBM rather than investing in their own hardware, software and technical personnel. Data on cloud usage are not collected, but the prices of the multiple different products are available on the providers’ websites. Without knowing the revenues earned from each type of product, it is not possible to construct a traditional price index.

Diane Coyle and David Nguyen estimated that there have been substantial declines in the prices of cloud services over time, especially adjusting for the quality improvements (such as continual software updates and enhanced services). In the example below, the adjusted price has declined by nearly 90% in eight years.

Figure 12.7 Price index, large instances, Linux, Q1-2010 to Q3-2018

Price index, large instances, Linux, Q1-2010 to Q3-2018

Coyle and Nguyen 2018, using data from Amazon Web Services

As the price of cloud use plummets and businesses grow more familiar with it, it is hardly surprising that more and more companies and government organisations are using the cloud rather than investing in their own ICT equipment and capabilities.

12.6 Beyond GDP

All of the phenomena above have been viewed through the lens of the current framework for GDP (and its related concepts). But do we need to think beyond GDP?

12.6.1 The spectrum of alternative measures

economic welfare
Economic welfare is a measure of the utility or satisfaction the society derives from the consumption of goods and services.

It has long been said, as stressed in Chapter 2 of this book, that GDP should not be used as a measure of economic welfare, as it has the more limited aim of measuring market-based transactions in the economy. But not only has it always included non-market activities such as public services and the imputed rent homeowners pay themselves for their accommodation – nearly a third of total GDP – it also becomes a measure of welfare when it is deflated from nominal to volume terms. This is because the concept of a price deflator is that it measures what level of spending would give people the same level of utility in this period as in previous periods. And of course growth in this “real” GDP is widely used by economists as an indicator of economic welfare.

This suggests thinking about a spectrum of measurement of the economy as a whole, from a narrow approach to conventional GDP to increasingly broad measures of economic welfare. The options here are set down in a paper by Heys, Martin and Mkandawire (2019), starting with the current definition of GDP.

Figure 12.8 A simple spectrum from market activity measurement to welfare

A simple spectrum from market activity measurement to welfare

12.6.2 Broader measures of economic welfare

These alternative measures have been explored in depth by economists. They include:

Democratic GDP is set out in this paper by Andrew Aitken and Martin Weale.

Jones and Klenow’s paper also includes a downloadable data set.

household account
The household account captures the non-financial and financial transactions of households, including the disposable income, spending, savings, debt and financial assets of households.

Joseph Stiglitz, Amartya Sen and Jean-Paul Fitoussi, the three economists who led the Commission, wrote a report setting out which indicators they thought were useful, and how they should be reconciled. A decade on, Joseph Stiglitz, Jean-Paul Fitoussi and Martine Durand followed up on the report in an OECD publication, ‘Beyond GDP: Measuring What Counts for Economic and Social Performance’.

ONS Resource

The ONS publishes many indicators of wellbeing, which can be combined into a pluralistic dashboard of the kind recommended by Stiglitz, Sen and Fitoussi.

12.6.3 Zero-price digital services in GDP

The use of digital services – involving the consumption of data – poses a particular conundrum. It acts to highlight some of the challenges described above: should we change how we measure GDP, or would it be better to step outside the framework of GDP and consider wider measures of welfare?

For the most part, people do not pay an explicit price for digital services such as social media, although they are purchasing data bundles as part of their telecommunications services and their devices such as smartphones, laptops or TV sets. Consumers are also paying indirectly for the costs of the online advertising that supports the free services, as advertisers will recoup costs in their prices. But is that enough to capture the economic value of the digital services?

One approach has been to think of what is happening as a sort of barter transaction. Work by Leonard Nakamura, Jon Samuels, and Rachel Soloveichik points out that people receive free services in return for being served advertising and marketing. The “free” content is a payment in kind for household attention: “In our methodology, households are active producers of viewership services that they barter for consumer entertainment”.

Nakamura, Samuels and Soloveichik estimate that US real GDP would have grown a tenth of a percentage point faster each year (1.53% a year rather than 1.42%) from 2005 to 2015 if this adjustment is included, which is not a negligible amount.

Erik Brynjolfsson, Avinash Collis, and Felix Eggers applied survey-based methods widely used in estimating the value of zero-price environmental goods to the zero-priced digital goods. In the context of both online surveys and lab-based experiments they found that consumer valuations of “free” digital goods are large. For instance, in the US, the median value placed on access to free online search for a year in 2017 was approximately half of median income. The authors advocate adding the total consumer valuation estimated using this approach to GDP to give an enhanced “GDP(B)”, although it is still early days for testing the method and the robustness of the estimates.

A third approach gives another alternative conceptual framework for treating digitalisation, focusing on consumption rather than production. Charles Hulten and Leonard Nakamura argue that much contemporary technological change can be thought of as an advance in the technology of consumption rather than the technology of production, so that produced outputs are transformed into basic commodities satisfying consumer needs and wants.

In a second paper, Hulten and Nakamura suggest that the growth rate of what they call “expanded GDP” has exceeded that of GDP by an amount “too large to be ignored”.

For example, the production of TV sets and fibre-optic cable is now delivering greater value to consumers than it used to because of the increased utility people get from being able to stream their favourite TV series whenever they like, or watch YouTube videos. Similarly, taking a mini-aspirin a day to avert cardio-vascular disease – not just to relieve a headache – is a new consumer technology applied to an existing product. The authors suggest creating an expanded GDP to take account of the copious consumption of free digital and information goods and the fact that the same level of consumer utility can be produced with less output.

It is evident from these very different approaches that the economic statistics community has not reached a settled view on how to handle the measurement of these kinds of services – even setting aside the complications arising from the fact that the providers are often located in other countries, route traffic between data centres in different countries, and route their financial flows across borders as well.

Indeed, there are some approaches suggesting the need for a more profound rethink of the purpose of measurement in this context.

12.6.4 Time-use accounting

The use of online services is blurring the “production boundary” between what is counted in GDP because a market transaction occurs, and what is counted in the household account because it is outside the market.

Read more about how these production boundaries are being blurred in the papers ‘Do‐it‐yourself Digital: The Production Boundary, the Productivity Puzzle and Economic Welfare and ‘Towards a Framework for Time Use, Welfare and Household-centric Economic Measurement’.

In the mid-twentieth century a good deal of activity crossed the production boundary the other way: more and more women opted for paid employment and purchased household durables and services, rather than producing services such as cleaning and meals themselves at home. Now, this is reversing. People are doing more of their own banking and travel and check-out services themselves as they transact online, or are providing “voluntary” labour in uploading amusing videos, or are even writing open-source software.

ONS Resource

Explore the ONS 2020 time-use survey, carried out exclusively under coronavirus (COVID-19) restrictions.

time-use accounting
Accounting for the value of unpaid household service work, not only in physical units, but also in monetary values. Time-use surveys measure the amount of time people spend doing various activities, such as paid work, household and family care, personal care, voluntary work, social life, travel, and leisure activities.

This observation leads to the thought that a time-use accounting framework will provide a useful lens on the economy, and particularly on the utility or well-being that people derive from different possibilities for spending their time. ONS has conducted occasional time-use surveys, and is currently undertaking a new one that will provide additional insight into people’s use of online services.

Time-use accounts underpin the construction of the household accounts satellite to the national accounts. They may now reveal new insights about the shift in activities across the production boundary.

12.7 Controversies and improvements in measurement

This chapter covers a territory that is made up entirely of controversies. What hope is there for improvement? More than you might think.

12.7.1 Administrative data

Apart from the active research in terms of the underlying concepts and approaches, one key improvement will come from new forms of data collection. Digital activities create many difficulties, but can perhaps also help with solutions: digital services create huge amounts of data that can be used to understand them and the wider economy.

The use of big data techniques is in its early days with many national statistical offices, but there are some early uses that suggest this might be a promising approach.

administrative data
Data extracted from other organisations’ administrative or management systems (including commercial organisations). The administrative and management data used will have been collected initially for the delivery of services or for operational purposes.

The most developed use of new data sources is from the increased use of administrative data, that is, data held by government and derived from its administrative processes, such as tax collection. Some of this data can help shed light on the wider economy. For example, in the UK, the use of VAT data from the tax authority is also offering the prospect of faster, more precise estimates of transaction activity.

ONS Resource

For instance, from 2017, ONS has been using VAT data in constructing GDP data.

12.7.2 Scanner data

The automatic recording of prices and quantities of goods purchased when scanned at store checkouts provides a potentially rich source of data and could allow price statistics to be based on a much wider range of goods and services in the future.

ONS Resource

Read about the latest progress by the ONS in exploiting scanner data.

12.7.3 Web scraping

The use of new data sources is not limited to overall economic activity or prices. We can also use it to understand how new technology and business models are changing the economy. For example, a novel data approach is the use of web scraping to derive an up-to-date classification of economic activity.

Jyldyz Djumalieva, Antonio Lima, and Cath Sleeman, researchers from NESTA, proposed a methodology for classifying occupations based on skill levels and specialisations provided in online job adverts, rather than the conventional Standard Occupational Classification categories. They used semi-supervised machine learning techniques to a dataset of 37 million UK online job adverts collected by Burning Glass Technologies. The resulting occupational classification is based on skill types, specialisation and salaries (see Figure 12.9), and so provides a methodology for grouping jobs into occupations based on skills. The method produced 33 domain-specific skill categories, some with sub-categories.

Figure 12.9 Classifying occupations by the skill requirements

Classifying occupations by the skill requirements

12.8 Conclusion

Not surprisingly, the use of hard-to-measure indicators is a work in progress. There is active research on all the issues described in this chapter, with significant progress needing to be made on both the conceptual challenges, and the practical data gathering and data processing ones. Even so, policymakers have an intense interest in the economic statistics concerning digitalisation, intangibles, globalisation and technical change. Across this waterfront, there are pressing policy challenges: Why is productivity growth so weak? To what extent are people getting better off anyway thanks to quality improvements? What is happening to trade flows and to the tax base as multinationals allocate their activities around the globe? If data is so important for future innovation and growth, how much of it is flowing within and across borders, and can we say what its contribution to the economy is?

This chapter has covered two related sets of issues: tracking the impact of digitalisation and other aspects of structural change on economic measurement, particularly GDP and price deflators; and considering the impact of these changes on economic welfare. Real GDP growth has never been a perfect indicator of changes in economic welfare. But the gap between real GDP and economic welfare in a broader sense has grown wider, and so, alongside work on measuring digitalisation and globalisation (which is necessary so the statistics can better inform policymakers and decision-makers), economists and statisticians are asking some deep conceptual questions about economic measurement.

12.9 Further reading

  1. Griliches Z (1964), ‘Notes on the Measurement of Price and Quality Changes’, in Models of Income Determination, Conference on Research in Income and Wealth, Princeton, NJ: Princeton University Press, pages 381–418 

  2. Diewert WE, Fox KJ, Schreyer P (2018), ‘The Digital Economy, New Products and Consumer Welfare’, ESCoE Discussion Paper 2018-16 

  3. Diewert WE (2019), ’Quality Adjustment and Hedonics: A Unified Approach’, Vancouver School of Economics Discussion Paper 19-01