The following three sub-sections were included in a draft (30 August 2015) of a seroprevalence guidance document (a WHO-sponsored publication).
9.4.1 Re-testing specimens and selecting outcomes to use in prevalence calculations
A randomly-selected stratified subset of specimens should be re-tested immediately for purposes of quality assurance. The study protocol should specify whether the second test should use the same EIA kit as the first, or a different kit. In addition, the study protocol may direct that the second testing should be performed in a different laboratory. The specimens selected for re-testing should include:
- All equivocal results from the first run
- A subset of negative results from the first run
- A subset of positive results from the first run
After the second test, a 3x3 concordance table (sometimes called a confusion matrix) should be constructed to assess the concordance of the results. Table 9.1 shows an example of this type of table. Note that in this example, most specimens that were negative in the first run were also negative in the second run. Although some positives had an equivocal result in the second run, most specimens that were positive in the first run were positive in the second test. There were no positive specimens that re-tested as negative, or vice versa. These results are highly concordant and do not give any indication that a quality control problem existed in the first run.
In contrast, the data in Table 9.2 show discordant results. There were 18 specimens with results that changed categories, from negative to positive (n=10) or positive to negative (n=8). This casts doubt on the quality of the first run, and indicates that the entire set of specimens from the first run (not just the subset) should be re-analyzed.
For seroprevalence calculations, each specimen must ultimately be assigned an outcome value: positive, negative, or equivocal. There are two issues to be determined regarding the outcome values:
- What value should be assigned to specimens with discordant results?
- What value should be assigned to specimens with repeatedly equivocal results?
9.4.2 Handling discordant results
As described above, some specimens should be tested at least twice to confirm that there was not an important quality problem in the first run. If the valid run indicators show a problem, or the concordance analysis between the first and second runs shows a problem, then all the specimens in the first run will need to be re-analyzed, and that process should be repeated until there is no longer a concern about quality.
Even when most results are concordant upon re-testing, a small number of specimens may have discordant results. For different runs, different kits, or different laboratories, the study protocol should stipulate a hierarchy of outcomes that makes it clear which result to use for each specimen in the prevalence calculation. An example of the decision tree, or hierarchy, may be:
- Use the negative and positive outcomes from the first run, first kit, first laboratory, if the re-test validation for the positive and negative results in the first run meets expected quality
- If there are specimens with equivocal results in the first run and a number of those same specimens yield either negative or positive results in a subsequent, validated run, then substitute the earlier equivocal result may be replaced. Record and later summarize the number of specimens whose equivocal results were replaced.
In this hierarchy, earlier positive and negative results are not replaced, but initial equivocal results can be superseded if positive or negative results are obtained by subsequent re-testing. An important principle is that the policy for designation of the outcome value for each specimen should be stated clearly, adhered to consistently, and documented. It is important to document the number of values that were used from the first run and the number of values that were overturned by testing performed with alternative kits or tested in another laboratory.
9.4.3 Repeatedly equivocal results
If the results for a specimen remain equivocal after subsequent testing, then the protocol will need to provide clear guidance about how to categorize the outcome in prevalence calculations. Three options may be adopted for treatment of equivocal results and the calculation of prevalence.
(1) Include equivocal specimens as seropositive. For some commercial EIAs, it will be biologically probable that specimens with equivocal results are in fact seropositive because the cutoff for a positive result is set at a level of IgG that is higher than the minimum level required for protection. The cut-offs for the kit may be evaluated by testing well-characterized serum panels by an assay that measures neutralizing antibody. For example, results by EIA and plaque neutralization suggested that equivocal results/titres obtained by EIA could be regarded as positive in a study that measured measles susceptibility using the same assay across several countries [12]. The study protocol should indicate that these specimens will be counted as positive. The number of specimens that repeatedly test equivocal should be documented and then reclassified as positive before proceeding with the prevalence calculation.
(2) Include equivocal specimens as seronegative. Alternatively, in some situations the conservative choice will be to consider equivocal results as negative and thereby err on the side of slightly underestimating the population prevalence of immunity. If the study organizers prefer this approach, then document the preference in the study protocol and be sure to summarize the number of specimens that test repeatedly as equivocal. The specimens are then reclassified as negatives and proceed with the prevalence calculation.
(3) Retain a separate category for equivocal results. A third option is to count equivocal results separately and maintain the proportion of equivocal results in the population in a distinct category. This approach allows the reader to conduct a sensitivity analysis by combining the equivocal results with either the positive or negatives. Calculations performed by adding equivocal results to either positives or negatives allows the reader to evaluate the extent to which the equivocal results affect prevalence estimates.