In Table 4, the dimensions of the agreement are calculated as follows: Note that the simple formulas for calculating the sensitivity and specificity of this design described in the appendix are not correct and that such naïve calculations would give biased estimates of sensitivity and specificity. This type of distortion is an example of checking or bias at work. For more information, see Begg (1987), Pepe (2003) or Zhou et al. (2002). Due to COVID-19, there is currently a great deal of interest in the sensitivity and specificity of a diagnostic test. These terms refer to the accuracy of a test for the diagnosis of a disease or condition. To calculate these statistics, the actual condition of the subject must be known, whether the subject has disease or condition. In Table 2, the estimated sensitivity and specificity are calculated as follows: the performance of the new test and the non-reference standard has not changed from Table 6A to 6C, but all agreements have changed simply because the prevalence of the state has changed. It is therefore difficult to generalize the agreement measures in Table 4 to another similar thematic population, unless you have additional information about the status of the state (for example. B Table 6A).
Yes, for example. B, you apply the defined reference standard to a random subset of all subjects or to all subjects for which the new test and comparison method do not match, and to a random sample of subjects if they agree, it is possible to calculate adjusted estimates (and deviations) of sensitivity and specificity. In this case, the FDA recommends re-testing a sufficient number of subjects to estimate sensitivity and specificity with appropriate accuracy. Diagnostic accuracy — the extent of the match between the new test result and the baseline We use the terms “positive percentage agreement” and “negative percentage agreement” with the following warning note. The agreement of a new test with the non-reference standard differs numerically from the non-reference agreement with the new test (contrary to what the term “agreement” implies). Therefore, when using these measures, the FDA recommends that the agreement clearly state the calculations made. Note that a diagnostic test with sensitivity is [1- specificity] has no diagnostic value. That is, if the percentage of subjects with positive results, if the condition is present (sensitivity), is identical to the percentage of subjects with positive results, if the condition is absent (1- Specificity), then the new test result is not influenced by the condition of interest and it has no diagnostic value of interest for this condition.
However, a test where sensitivity and specificity are close to 1 has a good diagnostic ability. Performance criteria should be interpreted in the context of population and study design. Sensitivity and specificity cannot be interpreted on their own; More information is needed. For example, the estimated sensitivity and specificity of the same test may vary from study to study, depending on the sub-themes included in the study and the use of an outdated reference standard compared to a baseline currently accepted by the clinical community. From a purely statistical point of view, the FDA believes that the best approach is to define a baseline and compare the new test with the stated baseline, which refers to subjects representative of the intended use population. We advise you to consult with the FDA before planning a study to ensure that the stated reference standard meets the Agency`s requirements. In this situation, sensitivity and specificity make sense, and you can easily calculate estimates. The appendices have a numerical example. In Table 3, you can calculate different statistical indicators of match. A discussion by Mr.M Shoukri on different types of contractual measures appears in the biostatistics encyclopedia (1998).