Tuesday, September 13, 2016

LCD Case Study: Deep Dive on Veracyte PERCEPTA LCD

On September 8, 2016, the MolDX program published a draft LCD for coverage of the Veracyte PERCEPTA test, which provides a molecular risk stratification of patients who have a suspicious lung nodule and an ambiguous bronchoscopy study (here).   A deep dive analysis of the LCD follows.


The body of the LCD is 1700 words long, with six tables and 13 footnotes.   The LCD does not refer to one of the specific reasons for coverage: more Medicare patients will be tallied with abnormal lung imaging due to the adoption of a National Coverage Determination for low dose CT screening in February 2015.  (The NCD tracks to USPSTF guidelines, and requires center accreditation.)


The LCD provides coverage for low to intermediate risk patients only.  Stratification guidelines refer to both Gould and McWilliams "AND" are summarized in a table as low risk, <10mm and <10 pack-years; intermediate risk 10-30mm nodules and 10-60 pack-years; high risk nodule > 30mm and >60 pack-years.  

While the Percepta test RNA material must be archived at the time of bronchoscopy, molecular testing is covered for those cases which are both low to intermediate clinical risk AND nondiagnostic post the bronchoscopy.

(Actually ambiguous cytology (cells) are obviously "non diagnostic."   I couldn't tell from the LCD how a post bronchoscopy finding of benign cytology is handled.  Are benign findings "actionably" negative, or are they potentially non diagnostic since tumor may be present but not reached or found?  Are some benign cytology cases diagnostically benign, and others non diagnostic because of concerns about target accuracy at the time of bronchoscopy?)


The LCD requires that physicians are "certified in the Percepta Certification and Training Registry," that suggested CT scans will be performed at 3-6, 9-12, and 18-24 months."

There is a requirement "clinical management is consistent with the post test diagnostic strategy [clinical action table] in >80% of patients."  This post test strategy is providing in a table showing that the expected clinical action (against 80% will be benchmarked) will be low risk test paired to CT surveillance and high risk test paired to biopsy.  The actions of the clinician will occur after the test is completed, reported, and paid by Noridian, so compliance with the clinical action table over a period of weeks does not seem to be required for Medicare payment of Percepta.

A registry is required to be reported to Medicare every six months for 36 months.  Afterward, the LCD requires that the registry data "will be published in a peer reviewed journal with an impact factor > 4.5."


The LCD has 12 scientific citations, plus a 13th citation regarding lung cancer epidemiology.   One of the lead citations is the main NEJM publication of the National Lung Screening Trial Research Team (2011).

Of the remaining 11 publications, seven (Gould 2007, 2013; McWilliams 2013; Tanner 2015; Rivera 2013; Ost 2003; Wiener 2013) are about risk scoring and clinical management.  

Four are publications on the PERCEPTA test.   These four publications are:

Whitney 2015 BMC Med Genomics: Derivation of classifier.
Silvestri 2015 NEJM:  Genomic classifier performance.
Vachani 2016 Chest:  Clinical Utility (case series; test not used clinically; modeling).
Ferguson 2016 BMC Pulmonary Med: Impact on clinical decision-making.

Not cited in the bibliography of the LCD is a fifth publication, specific to Analytical Validity of PERCEPTA, although data from this study appear in one of the LCD tables: 
Hu 2016 BMC Cancer: Analytical performance of classifier. 
Veracyte has also published a cost effectiveness study in abstract:
Feller-Kopman 2016:  Cost effectiveness (abstract).

I very briefly summarize the four publications:

In Whitney et al., specimens were collected from 299 patients (223 cancer positive, 76 cancer free).  17 of 232 relevant genes were chosen.   The algorithm had AUC .78 and .81 in test and validation cohorts, similar performance across a range of patient variables, and NPV of 94%.

In Silvestri et al., reporting from 639 patients (298 AEGIS-1, 341 AEGIS-2).  AUC was .74-.78.  Note that the test has higher sensitivity (88%) than specificity (47%).  In intermediate risk patients, the NPV was 91% after a non diagnostic examination.

In  Vachani et al., 222 patients had low and intermediate pretest probability and a nondiagnostic bronchoscopy, with 188 patients with follow up data.  77 underwent 99 invasive procedures.  Procedures "could have been avoided in 50% (21 of 42) patients..."

In Ferguson, 202 physicians provided case evaluations on 36 intermediate risk patient profiles from AEGIS trials, with and without PERCEPTA data.   Invasive procedure recommendations were reduced from 57% to 18%.   In positive patients, recommendations rose.



Below I include the online abstracts of each publication.

Four of the five major publications are open access.


Whitney DH et al. (2015)  Derivation of a bronchial genomic classifer for lung cancer in a prospective study of patients undergoing diagnostic bronchoscopy.  BMC Medical Genomics 8:18.  [Open Access]

The gene expression profile of cytologically-normal bronchial airway epithelial cells has previously been shown to be altered in patients with lung cancer. Although bronchoscopy is often used for the diagnosis of lung cancer, its sensitivity is imperfect, especially for small and peripheral suspicious lesions. In this study, we derived a gene expression classifier from airway epithelial cells that detects the presence of cancer in current and former smokers undergoing bronchoscopy for suspect lung cancer and evaluated its sensitivity to detect lung cancer among patients from an independent cohort.

We collected bronchial epithelial cells (BECs) from the mainstem bronchus of 299 current or former smokers (223 cancer-positive and 76 cancer-free subjects) undergoing bronchoscopy for suspected lung cancer in a prospective, multi-center study. RNA from these samples was run on gene expression microarrays for training a gene-expression classifier. A logistic regression model was built to predict cancer status, and the finalized classifier was validated in an independent cohort from a previous study.

We found 232 genes whose expression levels in the bronchial airway are associated with lung cancer. We then built a classifier based on the combination of 17 cancer genes, gene expression predictors of smoking status, smoking history, and gender, plus patient age. This classifier had a ROC curve AUC of 0.78 (95% CI, 0.70-0.86) in patients whose bronchoscopy did not lead to a diagnosis of lung cancer (n = 134). In the validation cohort, the classifier had a similar AUC of 0.81 (95% CI, 0.73-0.88) in this same subgroup (n = 118). The classifier performed similarly across a range of mass sizes, cancer histologies and stages. The negative predictive value was 94% (95% CI, 83-99%) in subjects with a non-diagnostic bronchoscopy.

We developed a gene expression classifier measured in bronchial airway epithelial cells that is able to detect lung cancer in current and former smokers who have undergone bronchoscopy for suspicion of lung cancer. Due to the high NPV of the classifier, it could potentially inform clinical decisions regarding the need for further invasive testing in patients whose bronchoscopy is non diagnostic.



Sylvestri GA et al. (2015)  A bronchial genomic classifier for the diagnostic evaluation of lung cancer.  NEJM 373:243-251.  

Bronchoscopy is frequently nondiagnostic in patients with pulmonary lesions suspected to be lung cancer. This often results in additional invasive testing, although many lesions are benign. We sought to validate a bronchial-airway gene-expression classifier that could improve the diagnostic performance of bronchoscopy.

Current or former smokers undergoing bronchoscopy for suspected lung cancer were enrolled at 28 centers in two multicenter prospective studies (AEGIS-1 and AEGIS-2). A gene-expression classifier was measured in epithelial cells collected from the normal-appearing mainstem bronchus to assess the probability of lung cancer.

A total of 639 patients in AEGIS-1 (298 patients) and AEGIS-2 (341 patients) met the criteria for inclusion. A total of 43% of bronchoscopic examinations were nondiagnostic for lung cancer, and invasive procedures were performed after bronchoscopy in 35% of patients with benign lesions. In AEGIS-1, the classifier had an area under the receiver-operating-characteristic curve (AUC) of 0.78 (95% confidence interval [CI], 0.73 to 0.83), a sensitivity of 88% (95% CI, 83 to 92), and a specificity of 47% (95% CI, 37 to 58). In AEGIS-2, the classifier had an AUC of 0.74 (95% CI, 0.68 to 0.80), a sensitivity of 89% (95% CI, 84 to 92), and a specificity of 47% (95% CI, 36 to 59). The combination of the classifier plus bronchoscopy had a sensitivity of 96% (95% CI, 93 to 98) in AEGIS-1 and 98% (95% CI, 96 to 99) in AEGIS-2, independent of lesion size and location. In 101 patients with an intermediate pretest probability of cancer, the negative predictive value of the classifier was 91% (95% CI, 75 to 98) among patients with a nondiagnostic bronchoscopic examination.

The gene-expression classifier improved the diagnostic performance of bronchoscopy for the detection of lung cancer. In intermediate-risk patients with a nondiagnostic bronchoscopic examination, a negative classifier score provides support for a more conservative diagnostic approach.



Vachani A et al. (2016)  Clinical utility of a bronchial genomic classifer in patients with suspected lung cancer.  Chest 150:210-218.  [Open Access]

Bronchoscopy is often the initial diagnostic procedure performed in patients with pulmonary lesions suggestive of lung cancer. A bronchial genomic classifier was previously validated to identify patients at low risk for lung cancer after an inconclusive bronchoscopy. In this study, we evaluated the potential of the classifier to reduce invasive procedure utilization in patients with suspected lung cancer.

In two multicenter trials of patients undergoing bronchoscopy for suspected lung cancer, the classifier was measured in normal-appearing bronchial epithelial cells from a mainstem bronchus. Among patients with low and intermediate pretest probability of cancer (n = 222), subsequent invasive procedures after an inconclusive bronchoscopy were identified. Estimates of the ability of the classifier to reduce unnecessary procedures were calculated.

Of the 222 patients, 188 (85%) had an inconclusive bronchoscopy and follow-up procedure data available for analysis. Seventy-seven (41%) patients underwent an additional 99 invasive procedures, which included surgical lung biopsy in 40 (52%) patients. Benign and malignant diseases were ultimately diagnosed in 62 (81%) and 15 (19%) patients, respectively. Among those undergoing surgical biopsy, 20 (50%) were performed in patients with benign disease. If the classifier had been used to guide decision making, procedures could have been avoided in 50% (21 of 42) of patients undergoing further invasive testing. Further, among 35 patients with an inconclusive index bronchoscopy who were diagnosed with lung cancer, the sensitivity of the classifier was 89%, with 4 (11%) patients having a false-negative classifier result.

Invasive procedures after an inconclusive bronchoscopy occur frequently, and most are performed in patients ultimately diagnosed with benign disease. Using the genomic classifier as an adjunct to bronchoscopy may reduce the frequency and associated morbidity of these invasive procedures.



Ferguson JS et al. (2016)  Impact of a bronchial genomic classifier on clinical decision making in patients undergoing diagnostic evaluation for lung cancer.  BMC Pulmonary Med 16:66.   [Open Access]

Bronchoscopy is frequently used for the evaluation of suspicious pulmonary lesions found on computed tomography, but its sensitivity for detecting lung cancer is limited. Recently, a bronchial genomic classifier was validated to improve the sensitivity of bronchoscopy for lung cancer detection, demonstrating a high sensitivity and negative predictive value among patients at intermediate risk (10–60 %) for lung cancer with an inconclusive bronchoscopy. Our objective for this study was to determine if a negative genomic classifier result that down-classifies a patient from intermediate risk to low risk (<10 %) for lung cancer would reduce the rate that physicians recommend more invasive testing among patients with an inconclusive bronchoscopy.

We conducted a randomized, prospective, decision impact survey study assessing pulmonologist recommendations in patients undergoing workup for lung cancer who had an inconclusive bronchoscopy. Cases with an intermediate pretest risk for lung cancer were selected from the AEGIS trials and presented in a randomized fashion to pulmonologists either with or without the patient’s bronchial genomic classifier result to determine how the classifier results impacted physician decisions.

Two hundred two physicians provided 1523 case evaluations on 36 patients. Invasive procedure recommendations were reduced from 57 % without the classifier result to 18 % with a negative (low risk) classifier result (p < 0.001). Invasive procedure recommendations increased from 50 to 65 % with a positive (intermediate risk) classifier result (p < 0.001). When stratifying by ultimate disease diagnosis, there was an overall reduction in invasive procedure recommendations in patients with benign disease when classifier results were reported (54 to 41 %, p < 0.001). For patients ultimately diagnosed with malignant disease, there was an overall increase in invasive procedure recommendations when the classifier results were reported (50 to 64 %, p = 0.003).

Our findings suggest that a negative (low risk) bronchial genomic classifier result reduces invasive procedure recommendations following an inconclusive bronchoscopy and that the classifier overall reduces invasive procedure recommendations among patients ultimately diagnosed with benign disease. These results support the potential clinical utility of the classifier to improve management of patients undergoing bronchoscopy for suspect lung cancer by reducing additional invasive procedures in the setting of benign disease.



Hu Z et al. (2016)  Analytical performance of a bronchial genomic classifier.  BMC Cancer 16:161.

The current standard practice of lung lesion diagnosis often leads to inconclusive results, requiring additional diagnostic follow up procedures that are invasive and often unnecessary due to the high benign rate in such lesions (Chest 143:e78S-e92, 2013). The Percepta bronchial genomic classifier was developed and clinically validated to provide more accurate classification of lung nodules and lesions that are inconclusive by bronchoscopy, using bronchial brushing specimens (N Engl J Med 373:243–51, 2015, BMC Med Genomics 8:18, 2015). The analytical performance of the Percepta test is reported here.

Analytical performance studies were designed to characterize the stability of RNA in bronchial brushing specimens during collection and shipment; analytical sensitivity defined as input RNA mass; analytical specificity (i.e. potentially interfering substances) as tested on blood and genomic DNA; and assay performance studies including intra-run, inter-run, and inter-laboratory reproducibility.

RNA content within bronchial brushing specimens preserved in RNAprotect is stable for up to 20 days at 4 °C with no changes in RNA yield or integrity. Analytical sensitivity studies demonstrated tolerance to variation in RNA input (157 ng to 243 ng). Analytical specificity studies utilizing cancer positive and cancer negative samples mixed with either blood (up to 10 % input mass) or genomic DNA (up to 10 % input mass) demonstrated no assay interference. The test is reproducible from RNA extraction through to Percepta test result, including variation across operators, runs, reagent lots, and laboratories (standard deviation of 0.26 for scores on > 6 unit scale).

Analytical sensitivity, analytical specificity and robustness of the Percepta test were successfully verified, supporting its suitability for clinical use.



Feller-Kopman DJ et al. (2016)  Cost effectiveness of the Percepta bronchial genomic classifier for the diagnostic evaluation of lung cancer: An analytical framework based on AEGIS studies.  ATS.

Percepta is a bronchial genomic classifier recently prospectively clinically validated to improve diagnostic accuracy for suspect lung cancer. When bronchoscopy is inconclusive for lung cancer, the classifier genomically identifies patients at low probability who may be considered for monitoring instead of an invasive procedure. The AEGIS-1 and AEGIS-2 studies have demonstrated an overall improvement in diagnostic sensitivity of classifier use in conjunction with bronchoscopy versus bronchoscopy alone. Our objective was to develop a health-economic assessment framework to model the clinical and economic value proposition of the use of the bronchial genomic classifier in the diagnostic workup of suspected lung cancer.

We developed a decision-analytic Markov modeling framework comparing bronchoscopy only to
bronchoscopy plus bronchial genomic classifier strategy. The model was developed from the third-party payer perspective, considering both U.S. Medicare and private payer perspectives. Test performance of initial diagnostic strategies was based on the recently published AEGIS results (Silvestri et al, NEJM, 2015). Diagnostic accuracy of non-invasive versus invasive follow-up, as well as associated adverse event rates were derived from a review of the published literature. Procedure costs were based on Medicare claims data analysis and 2015 inpatient and outpatient reimbursement amounts. Because the commercial price of the classifier test is not yet fully established, we used sensitivity analysis to explore the effect of changes in this parameter. All costs and outcomes were discounted at 3% p.a., in line with  health-economic guidelines.

In an intermediate-risk patient cohort (pre-test probability of cancer of 10-60%), bronchoscopy plus the classifier was found to be associated with improved diagnostic accuracy compared to bronchoscopy alone (negative predictive value of 91% and sensitivity of 93% vs. 41%). If patients with negative classifier results were to be followed with CT surveillance, these patients undergo an associated reduction in procedure-related adverse events, disutility and costs. The bronchial genomic classifier strategy was found either dominant or cost-effective across the considered test cost range, when considering commonly accepted thresholds of willingness-to-pay.

Bronchial genomic classifier-augmented bronchoscopy is a new diagnostic approach promising to improve diagnostic performance for the detection of lung cancer and reduce unnecessary invasive procedures, at a favorable health-economic profile ranging from overall cost-saving to cost-effective. Confirmatory analyses are needed once bronchial genomic classifier costs are fully established.