Closing schools: how to make a rational decision?

Original Content: The Conversation, in partnership with the AXA Research Fund

This is the purpose of a PNAS publication co-authored by 8 top-tier scientists, including two AXA professors : Prof. Massimo Marinacci, AXA Chair in Risks at Bocconi University, and Prof. Itzhac Gilboa, former AXA Chair in Decision Theory at HEC, currently member of the AXA Research Fund Scientific Board.

An article published in The Conversation presents the case of school closure decision in France in the context of the current health crisis. Policymaking during a pandemic can be extremely challenging. Given that Covid-19 is a new disease and its global impacts are unprecedented, decisions must be made in a highly uncertain, complex and rapidly changing environment.

Policymakers have been (and continue to be) charged with taking actions to protect their populations from the disease. Yet they lack reliable information on the virus and its transmission mechanisms, as well as on the effectiveness of possible measures and their direct and indirect health and socioeconomic consequences.

One particular issue that highlights the difficulties of decision-making is that of school closures. It’s a topic of much debate. According to UNESCO, schools were fully closed for an average of 3.5 months (14 weeks) since the onset of the Covid-19 pandemic. Their interactive map shows that one year into the COVID-19 pandemic, more than half the world’s student population (over 800 million students), still face significant disruptions to their education.

For policymakers, the problem involves finding the right balance between protecting people’s health and preventing economic and social disruptions by choosing whether and for how long to keep schools closed.

This decision involves a series of trade-offs. On the one hand, school closures may slow the pandemic and its impact by reducing child-to-child transmission. This would help to reduce the number of infections and the associated mortality rate. If this is the case, such interventions would bring clear health benefits to society. What’s more, reducing case numbers to below a given level would mean that some economic sectors could reopen, providing socioeconomic benefits.

On the other hand, closing schools can have high health and socioeconomic costs, both direct and indirect. For example, it may increase child-adult transmission (especially if grandparents help with childcare) and reduce the ability of essential workers to work (including health care workers). The economic costs of lengthy school closures are also high, generated for example through absenteeism by working parents, loss of education, etc.

Assessing the optimal policy implies weighing up the costs and benefits of this measure by comparing it to other possible courses of action.

This is particularly difficult, however, due to the lack of coherent scientific evidence and the high level of uncertainty surrounding the problem. At the time of decision, there is for example uncertainty due to the lack of data on the relative ability of school closures to control the spread of the virus, as well as to the contradictory projections from modelling studies.

There is no single view

Let’s take the example of a decision-maker who wishes to base themselves on the projections from the most recent epidemiological models and the advice of experts to assess the effectiveness of such a measure.

Imagine three different (hypothetical) scenarios:

• Model 1 is based on the only evidence we have, which is the one coming from influenza outbreaks for which the majority of transmission is between children. In these cases, closing schools is the biggest contributor to reducing the reproduction number (the famous R value) to below 1. In fact, it may well be the only measure that could do so.
• Model 2 relies on data from previous coronavirus outbreaks, for which evidence suggests minimal transmission between children. Here, R cannot be reduced to below 1 by closing schools. This measure does not affect the size of the epidemic and will not bring any benefits.
• Model 3 projects some child-to-child transmission. So, closing schools contributes to reducing R to below 1 and reduces the size of the epidemic. However, this only works in combination with other measures (without which R will remain above 1). As an isolated measure, school closures will not have a very significant effect. Under this scenario, the effectiveness of closing schools is important at the beginning, but declines as time goes on.

This uncertainty across models and their consequent predictions may be difficult to handle by policymakers, especially as one approach is not necessarily superior to another but simply adds another perspective. So, there is no single view.

The policymaker’s problem consists in finding the right balance between protecting the health of the population and preventing economic and social disruptions, given the scarce scientific evidence and the disagreement that may exist across model projections.

This means that determining the optimal time a policy such as closing schools should be implemented may be exceptionally challenging – lengthy closures bear very high costs, thereby substantially reducing any benefit to health systems and populations, but relaxing the measure earlier increases the risk of transmission surging again, leading to a second peak.

Using decision theory

In such a context, in which human lives and the economy are at stake, using ideas and constructs from modern decision theory could help governments face the uncertainty inherent in decisions related to the Covid-19 pandemic and justify decision-making. This will lead to policymaking that is more responsible and transparent.

These were the conclusions of a January 2021 study published in the journal PNAS, undertaken by a multidisciplinary international team in collaboration with Nobel Prize in Economics laureate Lars Peter Hansen.

Decision theory provides a way to organise a large amount of potentially conflicting scientific knowledge, and rules for evaluating response options in order to turn them into concrete decision-making.

One such rule involves calculating the expected utility of the potential courses of action in each scenario (or model in this case). To do so, we compute the consequences of closing schools for different periods of time, including both the benefits and costs, in various states of the environment. This concept is directly connected to the uncertainty that is beyond the decision-maker’s control, such as the baseline transmission dynamics of the virus.

The expected utility for each scenario is calculated by translating economic consequences into utility levels (via a function capturing the decision-maker’s attitude towards risk) and calculating the weighted average in the different states of the environment. A second weighted average is then computed by accounting for the degree of confidence that the decision-maker has in each individual scenario.

However, this approach still requires the decision-maker to express probabilistic beliefs on the correctness of a given model. In other words, they cannot say, I don’t know which scenario is the right option.

Because of the disagreements that may exist across different model outputs or expert opinions, another path may be to acknowledge one’s ignorance and relax the assumption that we can associate precise probabilities to any event. One rule in line with this approach involves acting very cautiously, forcing the decision-maker to only consider the scenario offering the lowest expected utility. In this case, only the worst-case model (here, Model 1) is being considered.

Avoiding reasoning mistakes

Making a decision based on the aforementioned rules is equivalent to complying implicitly with a set of general consistency conditions or principles governing human behaviour. Using decision theory as a formal guide will lend credibility to policymaking by ensuring that the resulting actions are coherent and defensible.

To illustrate how decision theory can serve as a coherence test, imagine the case of a policymaker trying to determine the optimal duration for school closures. The decision-makers can make up their minds by whatever mix of intuition, expert advice, and quantitative model results they have available, and then check their judgment by asking whether they can justify the decision using a formal decision rule.

Conceptually, it can be seen as a form of dialogue between the policymakers and decision theory, in which an attempt to justify a tentative decision helps clarify the problem and, perhaps, leads to a different conclusion.

Used this way, formal decision rules may help policymakers clarify the problem they are dealing with, test their intuition, eliminate strictly dominated options, and avoid reasoning mistakes and pitfalls that have been documented in psychological studies, e.g. confirmation bias, optimism bias, etc.

Finally, because committees might investigate the way decisions were taken during the crisis (for example, how lockdown measures were implemented and lifted), policymakers are held to account for the actions they took. A formal decision model can play an essential role in defending one’s choice and generating ex post justifiability.

The efficiency of transparent communication

To make the decision-making process under uncertainty more efficient, we also suggest acknowledging and communicating the various uncertainties transparently. For example, illustrating, quantifying and discussing the multiple sources of uncertainty may help policymakers better understand the potential impact of their choices.

Furthermore, while policymakers are responsible for making decisions, they are also responsible for communicating to professionals and the public. The way individuals respond to advice and chosen measures is at least as vital as government actions. Communication should thus be an essential part of the policy response to uncertainty.

As government strategies have been extensively debated in the media and models have become more scrutinized, one lesson learned from the Covid-19 management experience may be that policymakers and experts must increase their approaches’ transparency. Using the constructs from decision theory in policymaking, even in an informal way, will help ensure prudent navigation through the uncertainty that pervades this pandemic.