A proposal to enhance national capability to manage epidemics: The critical importance of expert statistical input including official statistics

1.Introduction

Given the high level of global mobility, pandemics are likely to be more frequent, and with potentially devastating consequences as has been demonstrated by COVID-19. This article describes what needs to be done to develop the statistical information needed to manage pandemics effectively. Whilst it is based on our experience in Australia, we believe our proposal applies more generally.

Whilst parts of Australia experienced a second wave of COVID-19, the country is in relatively better shape than most others and, at the time of writing this article, is virtually virus free. That said, we believe there is a critical need for improvements in the strategic statistical oversight of the whole process of anticipating, managing and analysing pandemics, using the experience of the COVID-19 to improve the quantitative information and advice available to policy makers. The proposal in the article outlines the quantitative aspects of a plan to enable policy makers to deal more efficiently and effectively with future such events, thus enhancing both the social and the economic welfare of its people.

Whilst the standard epidemiological data was available for the COVID-19 pandemic, and appeared to be of good quality, a dispassionate assessment of Australia’s health and economic response to the pandemic revealed some important inadequacies in available data, and the statistical analysis and interpretation used to guide Australia’s preparations and actions. For example, one key shortcoming was the lack of data to obtain an early understanding of the extent of asymptomatic and mildly symptomatic cases and the differences in reproduction characteristics across age groups, occupations or ethnic groups. In our view, this has meant that that Australian Governments were impaired in their ability to carry out their responsibilities.

We believe that managing the health, social and economic impacts of a novel virus concurrently requires a risk management (as distinct from risk avoidance) approach. This cannot be done without understanding the risks in quantitative terms and this depends critically on the ongoing acquisition, integration, analysis, interpretation and presentation of a variety of data streams to inform the development, execution and monitoring of appropriate strategies. Figure 2 (later in the article) captures the essential components grouped into four basic phases (Initial Detection; Prediction and Accumulation of Knowledge; Monitoring; and Post-pandemic Analysis), and the critical steps in each phase to plan for and acquire the diverse data needed. It is referred to as a Pandemic Information Plan (PIP) in this article. A broad range of statistical skills, knowledge and know-how is needed to support most aspects of the PIP. Official statistics have an important role to play and, given the audience for this Journal, these are discussed separately.

Furthermore, it is our strong recommendation that a multi-disciplinary Task Force be established to develop the PIP in more detail in anticipation of the event of a pandemic. As well as those involved from the policy side, the membership should include a statistician with expertise in statistical modelling and analysis, an official statistician, an epidemiologist, a medical researcher, an economist, a social psychologist and a public health official.

This article is organised as follows. Section 2 discusses what we have learnt from the COVID-19 experience in Australia. Each country has tackled COVID-19 differently but we believe the lessons learnt in Australia apply more generally. Section 3 presents an epidemiological model used to support the discussion in the article. The Section could be easily adapted to other epidemiological models. Section 4 discusses the importance of understanding heterogeneity and dispersion for managing the virus. In our view, this is not being given sufficient attention. Section 5 discusses the proposal to develop a PIP in more detail and presents the key elements of a PIP. Section 6 considers the return to normalcy as the pandemic abates. Section 7 provides a discussion on the role of official statistics; then we make some concluding remarks.

2.Learning from the COVID-19 experience

Despite the criticisms below, the following important steps were taken to inform the government’s management of COVID-19 and also to keep the public abreast of the progress of the virus.

What could be improved?

Figure 1.

(This diagram has been slightly modified from the corresponding diagram in Hao et al. 2020.) Illustration of the SAPHIRE model. Hao et al. extended the classic SEIR model to include seven compartments and their relationships: S: susceptible; E: exposed; P: Infective; I: ascertained infectious; A: unascertained infectious; H: isolated; Rem: removed. Two parameters of interest are: pt the ascertainment proportion (or ascertainment rate); bt the transmission proportion (or transmission rate); with the subscript t indicating that they change over time. In particular, pt changes quite a bit with the level of testing.

3.Epidemiological models

Epidemiologists use a variety of models but the information needed to support the models is broadly consistent. Many of them are variants of the classic SEIR (Susceptible, Exposed, Infectious, Removed) model. For the purposes of illustrating the information requirements, we have used the model from Harvard University [6].

At present, in Australia, only I is known with reasonable accuracy, and then only to the extent that testing is accurate. However, all compartments are important for understanding the path of the virus. The reproduction number should be based on bt, but in fact is currently being based on pt×bt. The ascertainment rate will vary with the level of testing and how closely targeted it is to high prevalence areas. Increases in pt×bt could be for this reason, rather than because the underlying number of infections has increased i.e. positive cases will increase as testing increases even if the prevalence remains the same and possibly as prevalence reduces.

The unascertained (A) component contains a number of different categories: pre-symptomatic, asymptomatic, and symptomatic but not yet ascertained (tested). Knowledge of each is important for understanding current infections levels as well as the efficacy of the testing protocols.

Regular scientifically random surveys of sufficient size will enable both A and I to be estimated with a reasonable degree of accuracy and hence pt and bt. If the samples are large enough they will enable the compilation of odds or prevalence ratios (as in the UK), as shown in Table 2 (see below), thereby providing insights into how the risks vary by sub-population and so offering guidance about the necessary policy initiatives. It will also provide estimates of most of the currently unknown basic compartments and sub-compartments in the Model, as shown in Table 1. Figures produced by health authorities cannot provide crucial information [3].

As stated by Georgiou [5],

The Rem component of A includes two main categories – infected and deaths. Some of the infected potentially may also be re-infected although the evidence suggests this is rare.

4.Heterogeneity and dispersion

A noteworthy aspect of the impact of COVID-19, which is policy relevant, is its differential impact on different sub-groups of the population. It is vitally important to be able to characterise this heterogeneity in order to manage a pandemic effectively. Customising strategies according to geography, age, industry/occupation, ethnicity or some combination of these, may enable the development of smarter interventions that have

far less impact on people’s well-being and on the economy. There are several ways in which information about heterogeneity can be used to advantage, including:

There are different ways at looking at heterogeneity. It may refer to (a) the probability (odds) of obtaining the virus, (b) the reproduction characteristics, or (c) the risk of serious illness or death.

With respect to (a), Table 2 shows the prevalence of swab-positive cases for different sub-populations in England, UK. They are derived from a series of scientifically random surveys of 100,000 people conducted to understand COVID-19 infections. Only the first and January 2021 rounds are shown.

The key messages from Table 2 are the highest prevalence is in young adults and the lower prevalence is with seniors. There is little difference by gender. There is a high prevalence with care home and health workers although the relative difference has declined over time presumably as protections for these workers has improved. Persons from an ethnic background have a higher prevalence.

In Australia, the anecdotal evidence suggests that the age and occupation patterns of COVID prevalence may have been similar to the UK when case numbers were high in the first and second waves. Furthermore, it also appears that the relative prevalence in migrant population may have been significantly higher than for the rest of the population. The high number in migrant numbers may have been associated with higher risk through employment, larger family sizes, and communication difficulties.

It is also important to understand the heterogeneity in reproduction characteristics. It may enable answers to the following questions and development of appropriate interventions:

Knowledge of the heterogeneity characteristics of fatality rates is essential in order to make sensible decisions about protections for those most at risk (e.g. aged care residences). Failure to address this adequately was the cause of most COVID-19 deaths in Australia. There were protections for visitors to care facilities but the main risk was actually with employees who might have contacted COVID-19 externally and spread it to the care facilities. Furthermore, many of these employees had part-time jobs across a number of care facilities.

To provide a simple example of the impact of heterogeneity on the effective reproduction number (R), if 20% of the population has an effective reproduction number of 2.5 or higher say (defined by geography, age, occupation/industry etc.) and 80% of the population has an effective reproduction number of less than 1.0, then the overall reproduction number will be a weighted combination of the two with the weights dependent on the number of active cases in each sub-group. R will be close to that of the first sub-group if that also, as expected, has the highest number of active cases. Reducing R below 1 for the population as a whole requires (1) targeted interventions to the first sub-group, (2) less stringent interventions for the second sub-group but sufficient to keep their reproduction number below 1, (3) interventions to minimise leakage between the two sub-groups (e.g. border restrictions, protective clothing) and (4) steps to reduce the risk for those most at risk of fatality (e.g. the elderly). If this can be achieved, it should result in the desired health outcomes but with reduced impact on the overall economy and well-being.

One limitation of the reproduction number is the averaging nature of the models underpinning the estimates. To partially overcome this, many of the models also incorporate what is known as the dispersion factor (k). It is a measure of how much some infected people transmit the virus and how little others do and therefore the relative importance of super-spreaders and super-spreading events. There was not much work in Australia in understanding dispersion with COVID-19, possibly because of the lack of suitable data to understand it, although there has been a lot of focus in interventions on avoiding super-spreading. A possible data source is test-and-trace data, obtained during contact tracing, but it has to be retained in a way that it can be used subsequently for statistical analysis. For COVID-19, there was a lot of reliance on anecdotal evidence.

Endo et al. [4] estimated a dispersion factor of 0.1 for COVID-19, which suggests that 80% of secondary transmissions may have been caused by a small number of infectious individuals. They concluded that “as most infected individuals do not contribute to the expansion of an epidemic, the effective reproduction number could be drastically reduced by preventing relatively rare super-spreading events.”

A low value of k means that a relatively small number of infected individuals are driving transmissions. It follows that if you can identify and reduce the situations that disproportionately drive transmissions, then the focus can be on these situations and other measures can be less disruptive. This is particularly important in the latter stage of a pandemic when the focus is on preventing future outbreaks.

In summary, data collections should be put in place to enable heterogeneity and dispersion to be estimated. Data scientists with statistical skills need to be involved in the design of these collections and the information models to set up the data so that it can be used for analysis.

5.The proposal for pandemic information

Epidemiological data (tests undertaken, number of positive cases, deaths) were the main data sources for learning about the COVID-19 pandemic and how it was tracking. These were supplemented from time to time by mobility indicators from sources such as Google GPS to understand the impact of interventions on mobility. Occasional small-scale surveys and research studies have also been conducted to throw some light on specific issues. More recently, waste-water surveillance has been used but only to detect the potential presence of the virus. The ABS also conducted regular surveys of the impact of the pandemic on households and businesses, with the capability to vary the subject content from survey to survey.

However, there are several other important, indeed essential sources of data that could also be tapped to inform intelligent management of a pandemic and are included in our proposed PIP. These sources include:

Figure 2.

The diagram represents management of a pandemic in four main phases, focusing on the requirements for data acquisition, expert analysis and interpretation, and presentation in each phase. Each box outlined in red involves tasks requiring statistical expertise. A variety of statistical skills, knowledge and knowhow is essential for efficient and effective management of the pandemic. Each of the numbers corresponds to more detailed information in Appendix 1. NPCC = National Pandemic Coordination Commission; see Appendix 1. The top sequence of boxes 2.1, 3.1, … represents the other major planning processes not requiring input of significant statistical expertise.

Table 3 summarises the range of data that might be used to inform a pandemic. It refers to four phases:

Table 3 begs the question: Why wasn’t all this information available for COVID-19? Important reasons include:

Pandemics such as COVID-19 lead to crisis. And in a time of crisis, it is essential that people and groups who are generally in (healthy) competition work collaboratively and collegially. Data, information, models and approaches need to be made available for learning, exploitation, critical peer assessment and comparison, so that the Government can be confident that it is receiving the best possible advice available. The PIP (see Fig. 2) seeks to correct these deficiencies.

Figure 2 displays the four basic phases for managing a pandemic. The figure clearly omits a large amount of complex activity that does not depend directly on data and analysis, although statistical and operations research techniques might be relevant to ensure these activities are performed effectively. This is captured in the single line of steps across the top of the chart.

It is immediately evident from the figure that there is a very large requirement for expert statistical advice. For the COVID-19 pandemic, very few of these steps were in place. The figure also omits data related to understanding the economic and social impacts of the interventions to control the pandemic and what is required to support the economic recovery. In these areas, the ABS has the main responsibility for providing the required data and, based on past performance, it would be expected that they would collaborate closely with the key users such as Treasury on their statistical requirements as they did with the COVID-19 pandemic.

The numbers associated with the process steps in Fig. 2 correspond to subsections of Appendix 1 to this document which, with varying levels of detail, provide a description of these steps. Table 3 also provides a discussion of how the data sources relate to the pandemic management phases. It includes the data sources used to help manage the COVID-19 pandemic as well as potential new data sources.

There needs to be agreement on the information requirements to support the responses to a pandemic. In fact, this should be done as soon as possible, utilising the COVID-19 experience, so there is a ‘blueprint’ before the next pandemic eventuates. This would include addressing questions such as:

There will no doubt be other information requirements that should be considered by the proposed multi-disciplinary Task Force we propose be set up for the purpose of determining the information requirements. Furthermore, measures of uncertainty are also important to enable sound interpretation of the data for policy analysis purposes. Once the information requirements are decided, the Task Force should consider the most appropriate data collections, and who should be responsible, so that development work can commence and they can be deployed at relatively short notice if required.

The effective reproduction number (R𝑒𝑓𝑓) and dispersion number (k) play important roles in understanding the path of epidemics and enabling a risk management approach to be taken. The information requirements needed to support these require particular attention.

We believe that the whole-of population-estimate of R𝑒𝑓𝑓 is of limited use for policy determination even if available at the State/Territory level. R𝑒𝑓𝑓 may be below 1 at this level but it might be much higher for some groups. It needs to be studied and estimated at a sub-population level, as it will vary considerably across different sub-populations. This type of analysis will help identify the groups where customised interventions and messaging are likely to result in the most beneficial impact. Surveys in the UK show infections are much higher for young adults, certain occupation groups (such as health and care workers) and certain ethnic groups. Had this information been available for use in Victoria for managing the second wave of COVID-19, there might have had a more nuanced response to the management of the risk, with better social and economic outcomes. Knowledge of the dispersion number k is also important for targeting interventions to prevent super-spreading.

To conclude this section, we emphasise that a well-designed scientifically random survey of the relevant population plays a very important role in satisfying many of the information requirements, including more accurate estimates of the effective reproduction number (as shown by the work of the REACT study [12] in the UK in respect of the COVID-19 pandemic. The (unidentified) survey data can also be used to support micro-simulation models developed to better understand likely patterns of transmission. Furthermore, such surveys will enable more detailed data to be obtained on the progress of the vaccine roll-out.

6.Return to normalcy

A key decision for governments is whether the underlying strategy is to be suppression or elimination. In many countries, elimination was not an option but it was an option for island countries like Australia and New Zealand. For COVID-19, the National Cabinet in Australia agreed on a suppression strategy although the interventions of the State governments suggest they are actually pursuing elimination strategies. Monitoring (Phase 3 on Fig. 2) is crucial to both strategies. A return to normalcy requires early detection of new cases so that testing, isolating and tracing can be put in place quickly and health systems do not get overwhelmed by large case numbers. If detected and controlled early, the interventions do not need to be as economically and socially damaging. In particular, it could avoid the huge damage due to disruptions and uncertainty caused by moving in and out of different levels of restrictions over time.

For COVID-19, in Australia the reliance was on epidemiological data for early detection (based on an extensive self-selected testing program) but there can be important lags in detection which can make its management more difficult. Waste-water surveillance and testing provided an opportunity to detect viruses much more quickly. Studies at Yale University (see https://globalnews.ca/news/6958321/sewage-coronavirus-ottawa-yale/) estimate that it is a 5-day leading indicator of people showing up positive in testing.

Waste-water surveillance is a key element of Box 5.2 (Fig. 2) but it needs to be done well. A good spatial sample design is crucial to optimise the probability of detecting traces of the virus whilst ensuring the tests are being used efficiently. The spatial sampling design would need links to data about the population (e.g. Census data) from which the waste-water has been derived. Adaptive sampling techniques could be used whereby positive tests at a relatively broad catchment area could be used to initiate tests in smaller catchment areas.

As with all tests there is the risk of false positives, depending on the degree of sensitivity of the tests, and false negatives, depending on the specificity of the tests. If the sensitivity is not 100%, multiple tests should be taken to minimise the chance of and expenses associated with reacting to what might be a false positive test. Likewise, the testing strategy needs to account for any lack of specificity. There can also be technical failures and anomalies, leading to measurement error, which will require statistical models to detect and correct.

There were many positive waste-water tests that have not led to the identification of positive COVID cases. It is commonly thought that this might be due to people who have previously had the virus still shedding fragments. Another explanation may be lack of sensitivity in the tests. There is a need to calibrate the strength of the test results to the likelihood of actual positive cases.

A suitable vaccine is also key to the return to normalcy. One would hope that experienced statisticians are involved in the development and testing of the vaccine. They should also be involved in the design of the roll-out (when operations research techniques such as queueing theory come to the fore). They also need to be involved in the design of the monitoring system, including information system design. For countries with population registers, these could provide the framework for understanding who specifically has had the vaccine and enable links to health system data bases. For other countries, a well-designed household survey may be required. It could be the same survey we have proposed for better understanding the prevalence of the virus. A survey may also be required, even for countries with registers, if some of the required information is not available from linked registers (e.g. reasons for not being vaccinated).

7.The role of official statistics

We think the following steps undertaken by the ABS in respect of COVID-19 were excellent and a good starting point for this discussion (see https://abs.gov.au/abs-responds-covid-19).

They produced a range of statistical products providing relevant insights on households, employment and industry to inform government, business and community responses to the pandemic. These included:

Access to confidentialised microdata for Australian businesses was made available to researchers through a remote access facility, known as TableBuilder,66 so that researchers can produce their own tables, graphs and maps.

They also explored the use of new data sources. These included:

The ABS produced a range of analytical products, based on ABS data, that improved understanding of the impact of COVID-19. These included analytical products on topics such as economic and labour market impacts.

Importantly, the ABS rapidly introduced new monthly surveys of the impacts on households and businesses of COVID-19. Estimates from these surveys were produced very quickly. Also, the content could vary from month to month, depending on issues of interest.

There are some other initiatives that could be undertaken with future pandemics. The ABS infrastructure could be used more extensively. This includes its household survey framework and household interviewers. In particular, the infrastructure could be used to provide the additional data we believe is vitally important for informing pandemics, as discussed earlier in this article.

Furthermore, the official statistical agency could assist with the integration of data from a range of different sources to provide a more coherent picture. This could be by means of analytical articles aided by an integrating framework, should a suitable one be available. For example, how can you get a balanced view of the real death rate taking account of death registration data, health data, excess death calculations and data on co-morbidity?

International comparisons are extremely important. For the comparisons to be valid, they need to be compiled in accordance with agreed international standards. Each pandemic will have its own characteristics but the underlying variables and parameters that are used to drive the epidemiological models will be the same (see Fig. 1). Work could be done now on identifying and defining them. They should be accompanied by estimates of the level uncertainty. For example, data on positive cases needs to be disaggregated by the type of test (e.g. swab or serological) as the specificity and sensitivity of these tests vary. There would be other areas where international standards would be important – such as the core demographics to be used, and definitions to be used for the effective reproduction number (R𝑒𝑓𝑓) and the dispersion co-efficient (k). The development of international statistical standards is core business for the UN Statistical Commission and it is suggested that this is where the work should be undertaken but in very close co-operation with the WHO.

8.Concluding remarks

There is a critical need for strategic statistical involvement in the whole process of learning from the COVID-19 pandemic to identify how to improve the quantitative information and advice provided to policy makers for future pandemics. The costs of collecting and analysing additional data are small compared with the human and economic costs of a pandemic. The additional data will provide important insights that were not available from the data used during the COVID-19 pandemic and help provide better economic and social outcomes.

Tests based on an ongoing national, scientifically random survey are one important response (undertaken in some Western European countries for COVID-19) but were rejected in Australia seemingly because of initial concerns about wasted tests but later because it was felt that ‘self-selected’ tests provided sufficient information: that is, the value provided by random surveys was not understood. They are not wasted if they provide information about the actual level of infection with socio-demographic characteristics (e.g. children) including asymptomatic and symptomatic cases that have not been tested. They are not wasted if they can provide timely information about the efficacy of interventions. And they are not wasted if supplemented by data on characteristics such as the number of close contacts in the previous 24 hours and other information that is useful for better managing the pandemic and the vaccine roll out. As it turned out, more than 15 million COVID-19 tests had been conducted at the time of writing and a household survey would have only required a small proportion of these tests. Countries with population registers may be able to use them in lieu of sample surveys.

Test-and-trace is a vital response to pandemics. Potentially, it also can provide a rich source of data for understanding the path of the virus but the data bases need to be established and managed in a way that it can be used for subsequent statistical analysis.

There is much to be learnt, at relatively low cost, from less direct data sources such as Waste-Water Surveillance to provide early alerts to the presence of a virus. These tests will not be available at the beginning of a pandemic but their development should be a high priority with statisticians intimately involved in this development work.

There were various interventions used to help manage COVID-19 (e.g. lockdowns of various types). These have significant economic and social effects and may or may not be effective. We need to learn from the COVID-19 experience. The analysis is complex and requires sophisticated statistical methods. However, the benefits to managing future pandemics are so strong, this work has to be done.

Finally, we strongly recommend a multi-disciplinary Task Force be established to determine the information requirements for managing a pandemic in more detail and how they might be met utilising the experience of the COVID-19 pandemic. The membership should include a statistician with statistical modelling and analysis expertise, an official statistician, an epidemiologist, a medical researcher, an economist, a social psychologist and a public health official.