Some political leaders — the Jair Bolsonaros and Donald Trumps of the world — have given the public the impression that they’re at war with the scientific community. They’ve wilfully ignored evidence and trafficked in misinformation in ways that have led to botched pandemic responses and cost the lives of millions.
But over the last two years, I’ve come to believe that the Bolsonaro or Trump approach is the exception, not the rule. In most countries, and across the United Nations system, it’s ignorance about how to use evidence properly to inform decision-making that has led to missteps.
In my 25 years working with governments, I have never before seen such demand for evidence to address a societal challenge — across countries and the political spectrum — as I have during COVID-19.
Heads of government may have made decisions that gave more weight to some types of considerations than experts might have, but there is typically not a straight line between evidence and action in most circumstances. And these policy-makers were democratically elected to make such tough decisions on our collective behalf.
I have also never before seen such rapid and extensive deployment of new evidence tools, including what’s known as living evidence syntheses and living guidelines, to meet the needs of political leaders.
New tools versus old habits
Living evidence syntheses add new evidence as it’s made available, based on its quality, so that we have a continually evolving picture of what the entire evidence base, not just the newest study, tells us. Good ones also describe how much certainty we have about particular findings.
Living guidelines extend the approach to provide continually evolving recommendations for particular countries or groups.
An additional tool can help to find the “best” evidence synthesis on a given question within minutes.
These tools do for policy-makers what product review websites do for shoppers — they test all the evidence out there and recommend only the best.
But the problem is many political leaders and their staff still don’t know these tools exist. Instead, they continue to rely on the latest study, turn to squeaky wheel experts or one-off expert panels, or simply look at what other countries are doing.
This unsystematic approach to COVID-19 evidence is a major reason why the steps taken by heads of government to address the pandemic yielded limited benefits, caused significant harm or had high economic and social costs. Examples include questioning the effectiveness of masks, promoting ineffective or harmful medicines like hydroxychloroquine and ivermectin and not supporting the infection prevention and control measures needed to keep the risk of transmission low in schools and day-care centres.
Solutions
The good news is there are solutions, as we document in the report of the Global Commission on Evidence to Address Societal Challenges. Here’s how:
-
We must stop chasing the latest study, and instead focus on the information emerging from living evidence syntheses. Such syntheses use a consistent approach to assessing the quality of all studies addressing the same question. They don’t accept a journal’s peer review as synonymous with quality.
-
We must stop accepting the personal opinions of high-profile experts and instead seek out experts who can back up their statements with a description of how they identified, assessed and interpreted the evidence they’re drawing on. Experts should speak in a way that makes it possible to judge their accuracy, as American think-tank president Richard Hanania has argued.
-
We must stop accepting the recommendations of old-school and one-off expert panels and instead push for more living panels that develop truly evidence-based recommendations. These panels should convene people with the right mix of issue-specific knowledge, evidence-appraisal expertise and lived experience. They should follow rigorous processes to develop their recommendations, and adjust them as the evidence and situation evolve.
-
We must stop engaging in the groupthink that can come from simply asking what other countries are doing and instead evaluate what they’re doing (and incorporate these evaluations in living evidence syntheses) so we know whether the benefits outweigh the harms and justify the costs.
-
We must build the processes into government and into the UN system so using evidence is a consistent feature in decision-making, just as political, economic, legal and communications considerations are always examined.
In December 2021, the World Health Assembly agreed to launch a process to develop a historic global accord on pandemic prevention, preparedness and response.
As the WHO and other parts of the UN system convene political leaders in the months ahead, they should complement preparations for the next pandemic with a commitment to use evidence in addressing the full range of societal challenges we face.
John N. Lavis receives funding for his work with the Global Commission on Evidence to Address Societal Challenges from the American Institutes of Research, the Canadian Institutes of Health Research, the Health Research Board, Healthcare Excellence Canada, and Michael Smith Health Research BC. He receives funding for his work with the COVID-19 Network to support Decision-making (COVID-END) from the Canadian Institutes of Health Research. He is the co-lead of the Evidence Commission and COVID-END.
