We often hear the first step in learning to use AI, is using it as a big powerful kind of Google.
I find AI can often work as a "big powerful PubMed," going beyond the use of keywords in Pubmed and using instructions that AI can interpret and understand. Also, since the domain is PubMed to start with, the risk of (hallucinations) should go from tiny to zero.
Here's an example from my work this afternoon:
##
Prompt (Chat GPT 6)
Use PubMed etc. to find four or five major comprehensive reviews or how-to articles about external control groups (ECG). If there is an FDA white paper or guide on this topic add that too. Please provide each in bibliographic references including hot link and PMID. They don't all have to be open access, but, at least 2 should be.
##
[30 seconds]
[Response]
These five papers make a strong reading set: broad methodological reviews, a practical step-by-step article, and an FDA-authored oncology review. Three have open-access full text in PubMed Central, linked below. The literature commonly uses external control arms (ECA) as well as external control groups (ECG).
Seeger JD, Davis KJ, Iannacone MR, et al. Methods for external control groups for single arm trials or long-term uncontrolled extensions to randomized clinical trials. Pharmacoepidemiology and Drug Safety. 2020;29(11):1382–1392. doi: 10.1002/pds.5141.
PMID: 32964514 · Open-access full text
Best broad methodological starting point. Covers selection of suitable external data, differences between trial participants and external populations, threats to valid comparisons, and design and analytical approaches to mitigate bias. Directly addresses both single-arm trials and long-term extensions. PubMedBurcu M, Dreyer NA, Franklin JM, et al. Real-world evidence to support regulatory decision-making for medicines: Considerations for external control arms. Pharmacoepidemiology and Drug Safety. 2020;29(10):1228–1235. doi: 10.1002/pds.4975.
PMID: 32162381 · Open-access full text
Best bridge between methods and regulatory use. Explains how the regulatory question should determine the data source, study design, and analysis, with attention to data fitness, confounding, and credible evidence generation. PubMedSchmidli H, Häring DA, Thomas M, Cassidy A, Weber S, Bretz F. Beyond randomized clinical trials: Use of external controls. Clinical Pharmacology & Therapeutics. 2020;107(4):806–816. doi: 10.1002/cpt.1723.
PMID: 31725899 · Publisher/DOI link
Best statistical overview of two distinct uses: constructing an external comparator for a single-arm trial and supplementing a randomized trial with external evidence. Includes worked case studies involving propensity-score matching and Bayesian borrowing. PubMedLambert J, Lengliné E, Porcher R, Thiébaut R, Zohar S, Chevret S. Enriching single-arm clinical trials with external controls: Possibilities and pitfalls. Blood Advances. 2023;7(19):5680–5690. doi: 10.1182/bloodadvances.2022009167.
PMID: 36534147 · Open-access full text
Best explicit “how-to” article. Organizes the work into three steps: define the treatment effect and population of interest, select appropriate external controls, and choose the corresponding statistical approach. Uses a multiple-myeloma example but offers broadly applicable guidance for conducting or critically appraising these studies. PubMedMishra-Kalyani PS, Amiri Kordestani L, Rivera DR, et al. External control arms in oncology: Current use and future directions. Annals of Oncology. 2022;33(4):376–383. doi: 10.1016/j.annonc.2021.12.015.
PMID: 35026413 · Publisher/full-text link — PubMed labels this a free article.
Best FDA-perspective review. Written by FDA investigators, it combines an introduction to external controls, study-design considerations, and case studies from marketing applications submitted to FDA. Especially useful for understanding how methodological issues affect regulatory interpretation. PubMed
- FDA guidance to add
U.S. Food and Drug Administration. Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products: Draft Guidance for Industry. February 2023. CDER, CBER, and Oncology Center of Excellence.
FDA guidance page · Free PDF · PMID: Not applicable—FDA guidance document.
This is the directly relevant FDA document, and FDA currently still labels it draft, not final. It addresses population comparability, treatment differences, alignment of “time zero,” outcome measurement, missing data, analysis, and FDA access to supporting records. Its scope is patient-level external controls drawn from other trials or real-world data; it does not cover external-data supplementation of an existing randomized control arm. FDA
###
Write a 250 word summary or recent priorities, emphases, and/or newer ideas, based on your reading of the above. Use headers or subheaders for clarity.
###
From Convenient Comparators to Planned Studies
Across these reviews and FDA’s draft guidance, the central emphasis is that an external control group should be designed as part of the study’s evidentiary strategy. Credibility depends on specifying the clinical question, eligibility criteria, outcomes, and analysis before examining comparative results. Finding an available database is only the beginning. pmc.ncbi.nlm.nih.gov
Comparability Extends Beyond Patient Matching
Matching age, diagnosis, and disease severity is insufficient. Investigators must also align treatment eras, background care, outcome definitions, assessment schedules, and the start of follow-up (“time zero”). Differences in these features can create apparent treatment benefits even when baseline characteristics look balanced. Missing data and inconsistent measurement deserve particular attention. pmc.ncbi.nlm.nih.gov
Define the Effect Before Choosing the Method
A prominent methodological emphasis is defining the estimand: precisely which treatment effect, in which population, the study seeks to estimate. That choice should guide comparator selection and statistical analysis. Propensity-score methods can address measured differences; they cannot automatically eliminate unmeasured confounding. Sensitivity analyses therefore help assess how robust conclusions are to remaining uncertainty. pubmed.ncbi.nlm.nih.gov
External Evidence Has Multiple Roles
External data can supply the entire comparator for a single-arm trial or supplement an existing randomized control arm. Bayesian borrowing offers an approach to the latter, making assumptions about compatibility between historical and current evidence especially consequential. These applications require different methodological and regulatory considerations. pubmed.ncbi.nlm.nih.gov
Regulatory Credibility Requires Transparency
The overarching priority is demonstrable fitness for purpose: appropriate data, prespecified methods, explicit limitations, and accessible supporting records. Early FDA discussion helps establish whether the proposed comparison can support the intended regulatory decision. www.fda.gov
##
##
Is there overlap between the field of expertise of 'external control arms' and the rising field called "target trial emulation?" Up to 150 words (if needed).
##
Yes—substantial overlap in both expertise and methods. Both draw on causal inference, epidemiology, biostatistics, and careful management of confounding and selection bias.
The distinction is that an external control arm describes where the comparator comes from, while target trial emulation describes a framework for designing the comparison: specify the hypothetical randomized trial, then align eligibility, treatment strategies, “time zero,” follow-up, outcomes, and analysis with that protocol.
TTE can therefore guide construction of an external control arm for a single-arm trial. It can also compare treatments entirely within observational data, without an actual clinical trial. External-control studies add challenges because trial and routine-care data may be collected differently. frontiersin.org
A directly relevant paper is Arnold et al. (2024), “Application of the target trial emulation framework to external comparator studies” (PMID 40979394). (Open text.) Neither approach, by itself, eliminates unmeasured confounding. 7pp
##
Note to self - see also PMID 40827506.
