By Andrew McKay 07/20/20
Preface 1: If you take only one lesson away from this article, I want it to be that the test positivity rate for COVID-19 is a very, very difficult thing to accurately figure out…but it still matters a lot.
Preface 2: This should finally clarify whether and when people who test more than once have those results counted in the data.
Preface 3: By way of warning, this article is very mathy. If you hate math, you may not enjoy reading this. But if you really want to understand some of the numbers driving public policy in our state and locally, well…here you go.
This morning, I noticed something irritating that I had sort of vaguely been aware of before but hadn’t quite bothered to really pay attention to: there are a lot of different numbers floating around for test positive rate in Florida and locally. After some investigation I now know why.
By my count, the Florida Department of Health uses three very different formulae to calculate test positive rate, and therefore different sources might well refer to or report widely different numbers.
FDOH METHOD 1 (Cumulative Positivity, also Andrew McKay’s method)
According to my data, the test positive rate on July 18 for Florida was 17.6%. That’s the total number of new positive cases (12,478 includes all persons) divided by the total number of test results (70,769). That’s the number I reported on my daily report dated 7/19 (because the reports are always for the prior day), just like I always have. In this method, a person only ever counts for one unit of any kind. No matter how many times someone is tested, if that person ever tests positive, they are forever a single positive. If they test 50 times negative, they count as one negative. This is the way FDOH calculates the cumulative test rate of all cases over the entire history of tracking COVID-19 on the Dashboard under the Florida Testing tab This is the simplest and cleanest way to handle the data. So far so good.
FDOH METHOD 2 (Percent Positivity for New Cases) (Maroon Bar Chart)
For the very same day of July 18, the FDOH state report in one place says there were 12,490 positive residents (not all persons, but close enough for now) and a total of 105,612 test results (12,490 positives and 93,122 negatives) for a test positive rate of 11.8%.
Just to be clear, 17.6% and 11.8% are wildly different numbers for the same day. Also 70,769 and 105,612 are massively different numbers of total tests for the same day. So where did the extra 34,843 tests come from?
It turns out those are all negative results…from people who have only tested negative before. As FDOH explains right in the chart, “These counts include the number of people for whom the department received PCR or antigen laboratory results by day. This percent is the number of people who test positive for the first time divided by all the people tested that day, excluding people who have previously tested positive.”
In this particular chart, anyone who has ever tested positive is excluded entirely from the data set, which is good. However, someone who has tested negative on (for instance) July 4 and has a result again on July 18 is still counted again on the 18th. Although it may not seem possible for second or third or fourth tests on the same person could account for a difference of 34,843 results, this is the reality. It comes from the fact that tens of thousands of people in Florida are being tested not just twice but multiple times. Mostly this is certain populations such as health care workers and long term care residents and staff, but there are some others as well.
So, if you understand the math so far, you will understand that Method 2 will give you a picture of how many positive tests are showing up on any given day among people who never tested positive before. You will also understand that this snapshot will always give a far lower positive rate than Method 1 because it includes duplicate negatives but not duplicate positives. It is important to note that on the county-by-county report issued by FDOH every day, this is the method used for positivity in any given county. Again, it’s the method which produces the lowest number from the data.
FDOH METHOD 3 (Number and Percent of Positive Labs) (Gray and Maroon Bar Chart)
Yet a third positivity number is designed to help us see what the positivity is of all the reported test on a given day. Again using our example of July 18, we see 16,423 positive results and 98,725 negative results for a total of 115,148 for a positivity rate of 14.3% (which is in between Method 1’s 17.6% and Method 2’s 11.8%).
If you’ve been following the news and the numbers for Florida, your first response should be, “Why didn’t anyone report a 16,423 positive test day? That would have been a record high and national news, right?” It’s because reliable news outlets don’t report just positive tests. We report new positive people or cases. In this instance, 3,933 of these were duplicate tests from people who had already tested positive in the past (16,423 total positives – 12,490 new positives). Once again, FDOH explains all this by saying, “These counts include the number of people for whom the department received PCR or antigen laboratory results by day. People
tested on multiple days will be included for each day a new result was received. A person is only counted once for each day they are tested, regardless of whether multiple specimens are tested or multiple results are received. If a person has a positive specimen and a negative specimen in the same day, only the positive result is counted.”
In this method, like Method 2, a person may only count once per day, but he may count as a positive or a negative again and again on different days. As you might suspect, this method of calculation is useful for seeing what percentage of people who had results on a given day had a positive test.
NOBODY USES METHOD 4 (All positives over all results)
A fourth method that isn’t currently used by FDOH may seem a bit silly, but stay with me for a moment to understand why it’s worth understanding. In this method, you simply take the total number of positives over the total number of tests, allowing individuals to count multiple times in a single day or across days if that happens. Clearly, this is a problematic method precisely because of the way it handles multiple tests for a single person. But seeing why should help us see the real problem with Methods 2 and 3.
Let’s imagine a person gets swabbed every day for five days. No, it doesn’t usually happen this way. Normally someone isn’t tested more frequently than once a week, but let’s pretend for the sake of illustration. Now let’s imagine those five tests come back negative, negative, negative, positive, positive on five consecutive weekdays:
Method 1 would count it as one positive case and one test result. On Monday it would show up as a negative, but on Thursday it would switch to a positive without adding a new test resulted because this person had already been reported on Monday as part of the negatives and total tests.
Method 4 would count it as three negatives, two positives, and five test results.
Method 2 would count it as a negative and a result on Monday, a negative and a result on Tuesday, a negative and a result on Wednesday, a positive and a result on Thursday, and wouldn’t count the fifth test when it comes back on Friday because prior positives are excluded. This means you get four days of test data from one person.
Method 3 would count the same for the first four days and then also count it as a positive on Friday; five days of data from one person.
HOWEVER, what if all five results came back on Friday instead of over the course of 5 different days?
Methods 1 and 4 would not change. But both Methods 2 and 3 would count it as only a single result, a positive. The importance of this is that because Methods 2 and 3 are willing to include data from a person more than once but not more than once on a single day, their data is at the mercy of the luck of the return date. That’s not a good way to manage data, in my opinion.
A FINAL NOTE ON METHOD 1
I almost hate to throw this in, but it’s true, and someone good at math will realize it, so here goes. There actually is a flaw with Method 1, but that flaw will not show up until we get to a point where lots of Floridians have already been tested at least once. The reason is that if you have already tested a lot of the population (half, say), suddenly you have a numbers problem.
Every test result will only count as a “new test” if it comes back for someone who hasn’t been tested before, which will be a dwindling pool the more people get tested. However, every new positive will still count as a new “case.” This means that the number of potential new people tested for the denominator will shrink but new positive cases will still be showing up.
As a result, when we start to get toward really large numbers of people already tested, this method will produce artificially high test positive rates. It’s even conceivable you could go above 100%. For example, on a day when 1000 people who had previously tested negative showed up positive along with 500 people who had never tested before and showed up negative, your positivity rate would be 200% by this method.
I don’t have a solution for this problem, but since we are currently only around 15% of the population tested, I’m not overly concerned. Nevertheless, there may come a time when Method 1’s positivity rate will have to be understood with this problem in mind.
For now, I believe Method 1 is still the best way to track positivity, and I will continue to report it this way. If other sources go by the FDOH county-by-county reports, they will likely report numbers much lower than mine. It’s not really a matter of which numbers are right so much as understanding them and just being consistent. If you look at the cumulative numbers FDOH reports on the Dashboard (under the Florida Testing Tab), those are done by Method 1. But if you listen to the Governor in his speeches, he tends to talk about total tests and recent positivity rates that are from either Method 2 or 3 (I haven’t checked to see). Mayor Robinson usually refers to the number from Method 2 number in his presentations. Since I believe in trying to keep numbers as consistent as possible, I’m going to stick with Method 1.