Mapping the Colorectal Cancer Screening Proposal: Why Use an Efficiency Frontier

CMS has a current NCD for biomarker CRC screening, using 74% sensitivity and 90% specificity as a benchmark.  This means you pick up about 3/4 of cancers (relative to colonoscopy) and you send about 10 patients per 100 to a false positive based colonoscopy.

Here I expand on a prior blog and show the two new CMS options graphically.

We can show the statistical space on a probability chart.  The vertical axis is specificity (and also shows "FP per 100").   The horizontal axis is the inverse of sensitivity.  It also shows "cancers missed per 100."   The IDEAL PLACE to be is the far upper left corner.

Since the required conditions are expressed as ≥, the look like an x,y point but define a rectangular solution space.  Any given clinical trial will represent a point with a cloud for SD (such as 90% spec +-2, 85% spec +- 3).

Here is the current solution space, ≥ 74 SENS, ≥ 90 SPEC, using green:

CMS proposes to use two new standards as options, or bins.  

• Bin A:    SENS ≥90, SPEC≥87
• Bin B:    SENS ≥ 79, SPEC ≥ 90

Here's Bin A in yellow:

Comparing the green space and the yellow space, CMS proposes to let tests slip downward in SENS a little bit (horizontal line downward, miss a few cancers) if you make a big improvement in SPEC (vertical line much leftward, send lots fewer patients to FP colonoscopy). 

And here's bin B in blue:

CMS may have felt the yellow space alone was too restrictive and the option of the blue space should be given, a space a bit more like current coverage in green.  Some currently covered tests are in the blue space but not in the yellow space.

###

Here's new proposals A and B superimposed.  You also see a pale green dashed line for the current standard (which extended further to the right, to 74% specificity).

The argument for an efficiency frontier can be shown on this last graph.   Admittedly, the void space caused between the two bin options is not too big, since one option uses 90% specificity and the other the slightly different 87% specificity.   Basically, CMS is willing to go a little lower on specificity below 90% (thus sending a few extra patients to FP colonoscopy), as long as the same has at least 90% SENS, much more stringent than the old 74% sens [green dash line].  

Still, if you look at the closeness to the ideal point, if CMS happily allows a test with performance "A" and one with performance "B," it should be reasonable to allow a test with performance "C."

In a prior blog, Chat GPT explained with an efficiency frontier equation.

##

Walking the Chart

Of course, any two-dimensional chart is only a simplified view of the policy problem. What the graph really shows is the tradeoff between false negatives and false positives. As one moves away from the ideal point of 100% sensitivity and 100% specificity in the upper left corner, improvements in one dimension typically come at the expense of the other. In practice, moving leftward on the chart—missing fewer cancers—often means moving downward as well, sending more patients to colonoscopy because of false positives. CMS is essentially deciding how much of each type of error society is willing to tolerate.

Bigger Debate Not 2-D - The Unscreened

But the policy debate is not simply colonoscopy versus biomarker testing. A very large population receives no screening at all—not colonoscopy and often not even stool-based testing. For that group, a test that detects 70–80% of cancers may still represent a major improvement over a 100% accurate colonoscopy that is never performed. In other words, the practical public-health question is not only how a biomarker test compares with colonoscopy, but also how it performs relative to no screening whatsoever. At the same time, CMS must worry about the reverse migration effect: Does CMS worry, if biomarker tests become too attractive, some patients who would otherwise have undergone colonoscopy (which also detects and removes precancers) might switch to a biomarker test that detects cancers but does not prevent them.

The Gnarly Policy World of FIT

Finally, CMS says it is not reopening the longstanding coverage decision for FIT. That is understandable—FIT includes multiple brands, many clinical studies, and a surprisingly wide range of reported performance values. Some of those data points would land comfortably within the new chart, while others would sit well to the right. The regulatory and evidentiary story behind FIT is itself quite interesting, and I discuss it in more detail in a separate blog post. See my FIT blog.