Mass testing for COVID-19 is having a large impact on everything in our lives in the UK. Case numbers are being used as the basis for the government to implement further restrictions; someone receiving a positive test case is now legally required to remain isolated in their residence for 14 days; schools send whole year groups home as soon as just one person receives a positive test; and university students are essentially being confined to their campuses whenever a case is detected.
While all these things are important in their own right and do warrant further discussion, what is perhaps more pressing is the following: are these COVID-19 tests actually telling people what we think they are?
RT-PCR testing – the importance of the cycle threshold
This article will only be considering the RT-PCR (or simply PCR) tests (i.e. the swab tests – the tests currently being used for diagnosis); other types of testing, such as antibody testing, are limited in their usage and tend to be reserved for research purposes (i.e. Public Health England’s weekly surveillance reports) and do not generally influence public policy.
RT-PCR, which stands for reverse transcriptase polymerase chain reaction, detects whether or not genetic material of the virus (RNA) is present in the sample taken from the individual. The RNA is first converted to DNA by enzymes. However, this can be a very small strand of genetic material; in order to be detectable, the DNA in the sample must be amplified. The amplification is done by a special machine, which amplifies the sample in a series of complex temperature cycles, resulting in doubling of the presence of DNA after each cycle. After 10 cycles, a single DNA strand becomes 1,024 strands. After another 10 cycles, there will be over a million DNA strands in total.
A threshold is determined at which point the DNA can first be detected (using fluorescent imaging) and where the test is declared positive for the virus. This introduces the crucially important concept of a cycle threshold (Ct). As described by Prof. Heneghan and Dr. Jefferson of the Centre of Evidence Based Medicine at Oxford (CEBM), “the number of cycles required before the fluorescence threshold is reached gives an estimate of how much virus is present in the sample. This measure is called the cycle threshold (Ct).”
“The higher the cycle number, the less RNA there is in the sample; the lower the level, the greater the amount in the initial sample.”
The PCR tests are incapable of determining whether someone is infectious
PCR tests were designed for use in a laboratory setting, and can only tell that there is viral DNA present. PCR tests were not designed to provide a definitive diagnosis of infectious disease. They cannot prove presence of an active infection, nor can they prove that the individual from whom the sample was taken is actually infectious. This is because the tests are capable of picking up inactivated, non-infectious RNA from individuals who may have been infected weeks ago but have since recovered. People can shed detectable, non-infectious RNA for a very long time; it has been observed to happen even up to 63 days after symptom offset. Crucially, the chance of finding these fragments depends on the aforementioned cycle threshold. If too many cycles are performed, the smallest fragment of inactivated RNA can be amplified to the level at which one would say the sample is positive.
Live viral culture must be taken to prove the sample is from a potentially infectious individual
In order to know whether or not someone has an active infection and is actually infectious, an attempt to culture the live virus from the sample must be made to prove this. A positive PCR test alone does not suffice.
The higher the cycle threshold (Ct) for a positive test, the less likely it is that person is infectious, and greatly so.
As reported in the American Association for the Advancement of Science last week (Sept. 29th, 2020):
“In a study published this week in Clinical Infectious Diseases, researchers led by Bernard La Scola, an infectious diseases expert at IHU-Méditerranée Infection, examined 3790 positive samples with known CT values to see whether they harbored viable virus, indicating the patients were likely infectious. La Scola and his colleagues found that 70% of samples with CT values of 25 or below could be cultured, compared with less than 3% of the cases with CT values above 35. “It’s fair to say that having a higher viral load is associated with being more infectious,” says Monica Gandhi, an infectious diseases specialist at the University of California, San Francisco.”
The figure below is from La Scola’s paper – the percentage of all samples providing live virus is shown by the bold black line. As can be seen, the percentage of samples with live virus falls from 100% (for positives at a Ct of 11) to just 2.7% of samples testing positive at a Ct of 35. It should be noted that the genetic material in a sample that needed 35 cycles to test positive needs to be amplified almost 17 million times to be as detectable as the genetic material in a sample which was positive with just 11 cycles.
In a recent study shared last week by CEBM suggested that “a cut-off Ct > 30 was associated with non-infectious samples”, and drew similar conclusions about the inadequacy of relying on a PCR positive, saying, “a binary Yes/No approach to the interpretation RT-PCR unvalidated against viral culture will result in false positives with possible segregation of large numbers of people who are no longer infectious and hence not a threat to public health.”
A positive at a lower Ct value indicates much more genetic material of the virus is present and dramatically increases the chance that the individual is infectious. It should therefore come as no surprise that those hospitalised with lower Ct values typically had a higher risk for severe disease and death.
From the same AAAS article mentioned above:
“A report in June from researchers at Weill Cornell Medicine found that among 678 hospitalized patients, 35% of those with a CT value of 25 or less died, compared with 17.6% with a CT value of 25 to 30 and 6.2% with a CT value above 30. In August, researchers in Brazil found that among 875 patients, those with a CT value of 25 or below were more likely to have severe disease or die.”
Unfortunately, there is no transparency or information regarding the cycle threshold used by the various private labs processing the tests for the NHS, so it is difficult to determine exactly what is in use in the UK. However, given the huge problems in the United States reported by the New York Times with laboratories commonly using test cycle thresholds above 30, and most other countries in Europe, such as Belgium, it seems unlikely the same has not occurred in the UK.
Lessons from the past also show the danger of solely relying on PCR test results – in 2007, the New York Times reported on a hospital in New Hampshire in the US where the staff believed they had become the centre of an epidemic of the bacterium Bordetella pertussis, which causes pertussis (whopping cough). There was mass panic; vaccines were quickly administered, hospital staff furloughed, and extra bed capacity was produced.
However, it appeared later that not a single sample taken from anyone at the hospital could actually grow the bacterium in a laboratory; it turned out that the PCR tests had been providing positive results based on fragments of genetic material that did not even come from Bordetella pertussis, and that the illnesses people experienced at the hospital were in fact caused by regularly encountered respiratory pathogens. Their feared epidemic of pertussis never happened.
PCR samples are highly susceptible to contamination
Due to how sensitive PCR tests are to small fragments of genetic material, laboratories have often found it an extremely onerous task to ensure samples are not inadvertently contaminated before they are analysed. This is particularly evident in cases where genetic material from previous tests can be left behind on laboratory equipment, which can in turn contaminate future test samples with residual DNA.
It would also require the individuals taking the samples (i.e. in test centres) to observe forensic infection control measures, and ensure their environments are free from any exogenous sources of DNA from SARS-CoV-2. Given that many test centres in the UK are freely available for members of the general public to walk into (some of whom may be shedding SARS-CoV-2 RNA), and only have a limited number of testing personnel who must be in contact with hundreds of different people, the challenge of maintaining a sterile environment becomes even more difficult.
A quote from a paper on PCR-based diagnostics, published in The Lancet Infectious Diseases by two researchers (one of whom consulted a US pharmaceutical company manufacturing PCR tests) explains some of the limitations:
“The widespread use of PCR in clinical settings has been hampered largely by background contamination from exogenous sources of DNA. In most pathogen-specific assays, the predominant source of contamination is derived from “carryover” products from earlier PCR reactions, which can be harboured and transmitted through PCR reagents, tubes, pipettes, and laboratory surfaces. Coupled with the robust amplification power of PCR, even very minor amounts of carry-over contamination may serve as substrates for amplification and lead to false-positive results.“
This contamination phenomenon could explain certain “super-spreader” events; where hundreds of individuals associated with a particular venue/institution receive a positive test result, but it appears nobody ever developed any symptoms whatsoever. It is likely that the laboratory that processed the samples (or perhaps the individual carrying out the swab test) accidentally contaminated them, producing false positives.
In spite of this, however, it is constantly communicated to the public that a PCR test is sufficient to both diagnose a disease and to indicate an active infection by SARS-CoV-2, even if the individual has no symptoms of illness. This is completely illogical. Diagnosis of disease cannot be done without the presence of clinical symptoms. Definitive evidence of active infection must come with a viral culture. For example, Moderna Therapeutics, one of the pharmaceutical companies with a leading candidate for a COVID-19 vaccine, uses the following stringent criteria for what counts as a COVID-19 positive for any individual taking part in the trial. They require evidence of telltale COVID-19 symptoms AND sufficient evidence that SARS-CoV-2 is present through the use of a PCR test. This is because of the standards which must be met in vaccine trials to show efficacy of the product:
“To be considered as a case of COVID-19 for the evaluation of the Primary Efficacy Endpoint, the following criteria must be met:
• The participant must have experienced at least TWO of the
following systemic symptoms: Fever (≥ 38ºC), chills, myalgia, headache, sore throat, new olfactory and taste disorder(s),
• The participant must have experienced at least ONE of the following respiratory signs/symptoms: cough, shortness of breath or difficulty breathing, OR clinical or radiographical evidence of pneumonia;
• The participant must have at least one NP swab, nasal swab, or saliva sample (or respiratory sample, if hospitalized) positive for SARS-CoV-2 by RT-PCR.“
Why these more robust standards are not applied to the criteria for determining a positive among the general public is unclear.
The testing issues extend far beyond cycle thresholds – the serious statistical issue of testing when disease prevalence is low
No laboratory test is 100% perfect – the same is true for the PCR tests. These tests have two crucial values, known as sensitivity and specificity, which can be described as follows:
Sensitivity: the probability that the test correctly identifies an individual who does truly have the virus as positive (true positive rate).
Specificity: the probability that the test correctly identifies an individual who does not have the virus as negative (true negative rate).
While false negatives are of interest, the implications of false positives are currently far greater in their impacts on wider society at this moment in time, for reasons that should become clear.
In June 2020, the UK Government conceded that it did not know the operational false positive rate of the tests in use, estimating at that time a rough rate of 2.3%:
“Results of 43 EQAs were examined, giving a median false positive rate of 2.3% (interquartile range 0.8-4.0%).”
Matt Hancock has since argued that the false positive rate is closer to 0.8% (summer testing in hospitals suggests the rate for some tests could even be as low as 0.4%), though no available government publication can confirm this.
Taking Matt Hancock’s assertion, this means the tests would have a sensitivity of 99.2%.
Using the assumption that the tests have a 0.8% false positive rate, mathematicians in the Department of Risk and Information Management at Queen Mary University of London published a very insightful paper calculating the probability of finding a true positive.
Using Bayesian analysis, they showed that, if you had received a positive test on the 11th of April (three days after the national daily deaths associated with COVID-19 peaked in the UK), the (mean) chance that test was 97% (with little variation around this point), meaning one could be extremely sure that they were indeed infected. This makes sense; most people getting tested in April had symptoms, so it would stand to reason that the confidence they were infected would be high. On top of this, the prevalence of SARS-CoV-2 in the UK was indeed remarkably high in April, so it would have been hard not to find true positive tests.
However, repeating this for the 17th September reduced this chance to 47%. Not only this, but the confidence interval, defined as a range of values that a measurement could plausibly take (given uncertainties in data), was extremely broad – from 3% to 87%. Statistically, this means the value of receiving a positive test case now would be close to meaningless; as the paper describes, “it is worse than tossing a coin”.
The main reason for this is that the prevalence of the disease in the general community is so much lower now than it was in April (~30% compared to ~0.41% in England between 25th Sept. and 1st Oct.)
The PCR tests lack the sensitivity to be useful for mass testing of the general public at this point; the false positive rate (about 0.8%) exceeds the natural prevalence of the virus (about 0.4%). This means, in a fully random sample of individuals being sampled, around 67% of positives found will be false*.
(*An important statistical caveat: it is not true that testing is done on a random sample of individuals, where you can safely assume only 0.2% of them have the virus. Many of those who present themselves for testing believe they have symptoms of COVID, making the chance they do actually have the virus a lot higher than the average. However, given that prevalence of other respiratory viruses that cause similar symptoms, such as rhinovirus, was as high as 25% in September (PHE: Weekly COVID19 Surveillance Report week 40), it is still the case that the number of people who truly have COVID is miniscule. The effect of the sample not truly being random will lower the 67% estimate above, but not to a significant degree).
Further evidence of this fact is provided by pathologist Dr. Clare Craig, who made an eye-opening observation in an article addressing false positives. She noted that the current death rate of patients testing positive for COVID-19 in hospital is about a quarter of it was in March and April. Granted, medical treatments have improved drastically, but the rate is now almost equal to the normally expected death rate of hospitalised patients. All patients for non-elective treatments are screened for COVID-19 on arrival to hospital in England. This suggests that, right now, COVID positives are not likely to be identifiers of truly sick individuals (as they were in March). They are more likely false and are artificially creating a subset of hospital patients; the death rate for all of whom is exactly the same (around 1.5% for all hospitalised patients).
Unless someone definitely has COVID-19 symptoms, it makes no sense to test them. Mass testing is not providing any meaningful information right now; it only serves to fuel mass panic, and encourage the government to implement knee-jerk policies to restrict people’s lives and livelihoods in response to “rising figures”, which are mostly false. Given the huge limitations of the testing, the use of these PCR tests for the general public should be discontinued immediately, and reserved mainly for those working in high-risk areas, or for those with definite symptoms and in need of medical attention, such as in hospital or care home settings.
- PCR tests do not distinguish between old and active infection; they only indicate presence of the RNA of the virus, even if it has been inactivated.
- PCR tests done with too high a cycle threshold are much more likely to pick up inactivated, “dead” fragments of virus and thereby identify individuals without an active infection as “positive”.
- A PCR positive does not mean the individual has a disease (that requires clinical symptoms), nor does it mean that they are infectious.
- In order to determine an infectious individual, a successful attempt to culture the live virus must be done (or, in the absence of this, a PCR test must be done with low enough a cycle threshold where it would be deemed unlikely to be both positive and non-infectious).
- When the prevalence of the virus is lower than the operational false positive rate of the test (as it is now), any positives that are found by random mass testing of the community are more likely to be false than true, and the results of such an endeavour therefore lack any meaningful use.
The limitations of PCR tests are well-established in the medical literature, for the technique has been used in laboratory settings for decades; the actions of Matt Hancock and others in the government who are regularly involved in providing information about testing, all of whom do not inform the public of these basic scientific facts, can be described as nothing other than fraudulent. This is especially true given the huge implications positive test cases have according to government policy, not only for the individuals receiving a positive test case, but also for the whole nation.
NEW (26/10/20): Kevin McKernan, R&D lead for the Human Genome Project at MIT, has added his expertise on PCR in a very insightful thread on Twitter, showing more evidence that these tests are not being used properly. The thread can be found here.