On Finding Answers in Research

I recently posted, on social media, a list of studies showing the effectiveness of masks and of mandates. One commenter said he doesn't trust studies because they could be biased, and instead he has used raw data to come to his own conclusions. 

A few problems here:

• If a group of scholars in a field work together to analyze data until they reach a consensus, and then the paper undergoes a peer review process in which no less than three impartial and anonymous reviewers scrutinize the methodology and then further analyze the gathered data to ensure they come to the same conclusions, if all that can be warped by bias to the point that all these studies coming to the same conclusion are flawed, then what makes it likely that Joe Blow, basement data collector, has no bias in his data analysis?
• Part of the problem is that Joe thinks he can consciously recognize and avoid bias, as if it's something within our awareness, as if all researchers and their anonymous reviewers are biased consciously due to their some benefit they hope to get from nefariously leading the data to match their hoped for conclusions. But often bias is covert, which is why the scientific method has so many rules and systems to undermine any possible subconscious confirmation bias, like double-blind studies. Can Joe be sure that he harbours no unconscious biases??
• If studies come to different conclusions, we can evaluate the quality of the studies to deduce which is more accurate by looking at the ranking of the journals that have accepted them, the confidence levels of the data, and then get into the background of the lead authors, and then further into the sample size and whether or not it's blinded and/or a matched-pair analysis. We can find the answers to scientific questions. 
• Joe's raw data compares a school with mask mandates in one state to another school without mask mandates in another state, and shows that the Covid rate in both counties are similar. Immediately it's obvious that the differences in the counties are going to create confounds in the data. Ideally, we want to compare two very similar places (or match participants) that have most things in common except for a mask mandate in order to see the effect of the mandate. The research showing a significant difference in similar communities in Alberta and another that compared schools with different rules in Boston (in my previous post) are a better measure of effectiveness. Dr. Lucky Tran addressed why mask mandates don't always lead to lower rates of infection:

"If masks work, then why did places with mandates and high mask wearing like NYC and South Korea experience Omicron surges? A big reason: bars, restaurants, and large events where masks come off were still open. Mask policies don't exist in a vacuum. It's why we need layered measures. So if we are keeping places where masks come off like bars, restaurants and large events, what's the point of mandates? Universal masking works, and it helps ensure everyone, particularly our most vulnerable, can access essential services like healthcare, transportation, and groceries. People have a choice to go eat out or attend entertainment events, where there are higher risks of transmission. But people shouldn't have to make the impossible choice between going to work or getting healthcare and risking getting COVID. Masks keep these spaces safe for all. I'm seeing a lot of misinformation saying mask mandates don't work. You can't use a graph like this (below) to conclude causation because there are many confounding factors: e.g. places where people eat and drink were fully open. Mandates didn't apply to all public spaces."​

This is an on-going issue in my classes. Too many students don't understand the scientific method or how it weeds out bias. And too many believe that it's not possible to find the most accurate or reasonable or scientifically verifiable answer to a questions like whether or not masks work to stop a virus or whether or not mask mandates effectively reduce transmission. Some things can be known. Trust the science; wear a mask.

ETA: today's headlines:

Global News headline: "'We cannot live with 15,000 deaths a week': WHO warns on rise in COVID fatalities"

Inside: "Learning to live with COVID-19 does not mean we pretend it's not there. It means we use the tools we have to protect ourselves and protect others. . . . Wear a mask when you can't distance. And try to avoid crowds, especially indoors."

And Globe & Mail headline: "To address Canada's health care crisis, start by containing COVID-19." 

Inside: "High-quality masks are a low-burden and effective intervention to slow spread of all seasonal viruses. Masks should be used in indoor public settings wherever broad mingling occurs."