Thursday, April 14, 2016

Modern Views of Diagnostic Testing and Barr's "TTP" Model of the 1980s

Sometimes you hear that today, there is a sudden new interest in outcomes of diagnostic test use, and not just the test result as an analytic variable. That's how it was with payers, you know, five years ago.

This always strikes me as nonsensical, like when someone asserts, "evidence based medicine first became the goal in the 1990s."

Take for example Judith Barr's model of "Total Test Process" which begins with a clinical question and ends 11 steps later with the documentation of the impact on clinical care.  Well - yes!

Below is the key figure from a 1994 review, open access online, by Schumacher & Barr.  The themes under "Total Test Process" originate from Barr's foundational work as early as 1981 (Amer J Med Tech 47:619).  The full 1994 paper is just five pages long, is available free online, and is well worth reading.  Paragraph after paragraph I was circling an idea that was contemporary and relevant and that matches exactly to a payer/new test/evaluation problem we face every day.

1994 summary article, here; from Clin Chem 1998 44:370-4.

Barr and Schumacher was were both at the College of Pharmacy at Northeastern University where they served as Associate Dean and Dean, respectively.  Their interest in the value of therapeutic drug modeling led to authoring several decades of progressive work on diagnostic tests, like this one.  They also published important early on decision modeling analysis for medical diagnostic tests (here).  (I had the chance to correspond with Dr. Barr, who is now professor emeritus.)


In a 2014 paper, Felix Frueh and I expressed a similar underlying idea but in a different diagram, as follows:

We used paired bar charts to emphasize that there had to be a gain in clinical validity (against some comparator), a change in management (from what it had been), and some impact on clinical outcome (against some previous status quo.)   We posited six key questions for any diagnostic test (whether molecular or imaging).  (1) What is the indication/population?  (2) What is the gain in clinical validity or gain in information, against what you had before?  (3) What could the doctor do, he wouldn't have done before [and how often]?  (4) What would be the patient benefit [clinical utility], against what would otherwise have happened?  (5) What do doctors actually do, in the real world?  Finally, use the net clinical impact after questions 1-5, people will ask (6) what is the economic impact (however defined; is it a "reasonable" value?)   We posited that if any of these six questions had no answers, or very vague answers, things would not go well for test adoption.

As noted in the legend, assessors will consider two kinds of uncertainty, both statistical uncertainty (we observed 6 SD ± 2 extra life months) and conceptual or pragmatic uncertainy (will this obtain outside a trial?).  Wise promotors of test adoption will address both kinds of uncertainty, and explicitly.   Frueh & Quinn, here.