Sin of Omission: Alberta's Paxlovid triage study.

I want to briefly go over the Alberta Health Services study released on medRxiv a couple of days ago.

Generally, I am not particularly interested in studies attempting to justify the vaccination anymore but this study is something else: This is a triage study. The analysts involved in writing this study are interested in finding a way to justify a “rational allocation” of Paxlovid in the wider population.

Supply based health care services, rather than demand based, are the norm in Canada. Indeed, that is one of the reasons our health care services consistently rank amongst the most dysfunctional in the developed world, so I am not particularly surprised this study is being done. What irks me about this study, rather, is how much the authors are leaving out.

This study uses observational data from positive cases in January 2022 and labels itself as a “case-control” study. Both cases and controls are those that tested positive for the virus. The cases are those that had “severe outcomes”, which is defined as hospitalization, ICU, or death. Whereas, the controls are those who recovered without any of the above outcome.

Additionally, the authors have health data on individuals in the study and control for age, sex, vaccination status, and the “number of underlying conditions”. Finally, the study excludes those under 18, those who have previously tested positive for the virus, and those that had “non-COVID related outcomes”.

Personally, I am extremely underwhelmed by the methodology, outcome and control variables used, and the exclusions.

Methodology

There are hidden assumptions in this study design. In order for an unbiased estimate, it would be essential to assume that individuals in the population are testing positive for the same reason (as explained below if that sounds strange). This is similar to the assumptions of a test negative design, which has been thoroughly abused by public health officials during the last year and a half.

Simply put, this assumption is not satisfied. During the observation period, there was still mandatory testing requirements in Alberta. These requirements were heterogeneous between populations meaning the unvaccinated individuals would be testing at different times than vaccinated individuals. While vaccinated individuals would be testing for purposes like international travel, something they would presumably only be doing when feeling healthy, unvaccinated individuals would not.

During the month of January, in fact, Alberta switched their testing methods for PCR tests to “be focused on those with clinical risk factors for severe outcomes and those who live or work in high-risk settings”. Now, who do you think Alberta considered a person at risk for severe outcomes, a vaccinated or unvaccinated individual upon entry to the hospital? The unvaccinated person would be tested. Every time. Thus, even if the unvaccinated person was hospitalized for a different reason, they would be considered a positive case and a severe outcome.

Furthermore, health care workers may also test, vaccinated or not, but the amount of unvaccinated health care workers in Alberta was disproportionately small. The fact is that the groups testing positive were not equal and the unvaccinated individual had a higher chance of being tested upon entry to the hospital.

There are a lot of reasons to assume the data is heavily biased here and the authors of this study did not even bother to include data that shows time from testing positive to severe COVID. A disproportionate amount of unvaccinated individuals ending up hospitalized on day one would clearly show that these tests were driven by testing-at hospitalizations rather than COVID leading to hospitalizations.

The authors did exclude about half of the severe outcomes as they were “not from COVID”, but unfortunately, the accuracy of that information can safely be questioned. The public health messaging alone has created bias in society. According to the “experts” at the time, vaccinated people don’t die from COVID and it is a death sentence for unvaccinated people. After so long inundating health care workers with that messaging, how can we trust them to make that call? We simply can’t. It would be more interesting to see who, in the entire data set including those who did not test positive at all, is winding up in the hospital and at what rate.

The outcome variable and control variables

Additionally, the authors grouping of “severe outcomes” is insane. To use a baseball analogy, a walk is not a grand slam. Nor is a hospitalization a death. These things should not be grouped together¹. That introduces a huge amount of bias into the analysis.

Likewise, the number of underlying conditions should not be an explanatory variable. The authors had full data on all individuals but chose to group a bunch of unrelated illnesses. This introduces the same level of bias. Being clinically underweight is not the same as having cancer. It simply does not make sense to group these things together. Instead of number of underlying conditions as a variable, the underlying conditions themselves should have all been viewed separately.

In fact, the choice of cases and controls introduces bias in and of itself. The cases are, essentially, the chances of having a “severe outcome” if you test positive for the virus. Interestingly, unvaccinated people were less likely to test positive for the virus in this analysis. 92.26% of all cases are in the vaccinated. Even according to the Alberta numbers, which were out of date population estimates and almost certainly underestimates of the unvaccinated population, about 10-11% of those 18+ were unvaccinated.

Exclusions

To that point, the authors also don’t provide any data on the characteristics of the exclusions. Both the “not from COVID” and the “already had COVID” group could significantly change the outcome of the analysis.

The former exclusion is subjective depending on the biases of the health care worker. The latter exclusion doesn’t eliminate all those who have had the virus—just those who tested positive under varying circumstances. It’s unclear how much this would change the underlying analysis, but considering we are talking about quite small numbers in the case of severe outcomes, the exclusion may be significant.

There were other notable exclusions, which should not have occurred given the amount of data available to the authors. These are not individuals excluded from the study, but rather, variables excluded from the regression. Ethnicity, for example, is often associated with an large increase in poor outcomes. I find it unlikely that this information was not available to the authors of the study, yet it is absent. This is just one of the many variables that are clearly missing from the study. There may be a dozen other variables that the authors chose not to use.

I’m not impressed at all by the gatekeepers of the data releasing their poorly done studies. Any first year statistics student can run a regression. A well designed study should aim to be as careful as possible while working to eliminate or reduce bias. That was not done here. Not even close.

Honestly, they need to make this data completely open to the public. What is stopping them? Surely not the same privacy legislation that they ignored when implementing a vaccine passport system, right?

1. Interestingly, it appears that a single individual could have all three severe outcomes.