Tuesday, February 7, 2017

The FDA"s Regulation for Expanded Carrier Screening

Last summer, there was a publication in JAMA by Haque et al., the largest study ever undertaken of recessive carrier genes distributed over a population of over 300,000 individuals (here).

After a well-known intervention at 23andMe, the FDA released self-implementing regulations for autosomal recessive test systems.  The regulations are unusual and innovative, dependent in large part on the laboratory's webposting of accuracy and validity data.

Most FDA product category regulations (21 CFR 800ff.) are only a few words long, at most a couple sentences.  For example, Agendia Mammaprint and Nanostring Prosigna breast cancer prognostic tests are cleared under 866.6040:
A gene expression profiling test system for breast cancer prognosis is a device that measures the ribonucleic acid (RNA) expression level of multiple genes and combines this information to yield a signature (pattern or classifier or index) to aid in prognosis of previously diagnosed breast cancer.
In contrast to that type of product classification, the regulation for expanded carrier screening is 3000 words long and runs six pages when clipped into a single-spaced word document.

The regulation (866.5940) is copied below, after the break.   The original Federal Register publication (80 FR 65626ff, October 27, 2015) is here.

The regulation requires hyperlinked reference to credible sources of validity information "such as GeneReviews" that is not otherwise reviewed by FDA before presentation to physician and patient.  This is similar to new guidance on biopharma communications with payers on economic evidence, which must be based on "competent and reliable scientific evidence" (CARSE) not otherwise reviewed by FDA (here).

Subpart F--Immunological Test Systems
Sec. 866.5940 Autosomal recessive carrier screening gene mutation detection system.

(a) Identification. Autosomal recessive carrier screening gene mutation detection system is a qualitative in vitro molecular diagnostic system used for genotyping of clinically relevant variants in genomic DNA isolated from human specimens intended for prescription use or over-the-counter use. The device is intended for autosomal recessive disease carrier screening in adults of reproductive age. The device is not intended for copy number variation, cytogenetic, or biochemical testing.
(b) Classification. Class II (special controls). Autosomal recessive carrier screening gene mutation detection system must comply with the following special controls:
(1) If the device is offered over-the-counter, the device manufacturer must provide information to a potential purchaser or actual test report recipient about how to obtain access to a board-certified clinical molecular geneticist or equivalent to assist in pre- and post-test counseling.
(2) The device must use a collection device that is FDA cleared, approved, or classified as 510(k) exempt, with an indication for in vitro diagnostic use in DNA testing.
(3) The device's labeling must include a prominent hyperlink to the manufacturer's public Web site where the manufacturer shall make the information identified in this section publicly available. The manufacturer's home page, as well as the primary part of the manufacturer's Web site that discusses the device, must provide a prominently placed hyperlink to the Web page containing this information and must allow unrestricted viewing access. If the device can be purchased from the Web site or testing using the device can be ordered from the Web site, the same information must be found on the Web page for ordering the device or provided in a prominently placed and publicly accessible hyperlink on the Web page for ordering the device. Any changes to the device that could significantly affect safety or effectiveness would require new data or information in support of such changes, which would also have to be posted on the manufacturer's Web site. The information must include:
(i) A detailed device description including:
(A) Gene (or list of the genes if more than one) and variants the test detects (using standardized nomenclature, Human Genome Organization (HUGO) nomenclature, and coordinates).
(B) Scientifically established clinical validity of each variant detected and reported by the test, which must be well-established in peer-reviewed journal articles, authoritative summaries of the literature such as Genetics Home Reference (http://ghr.nlm.nih.gov/ ), GeneReviews (http://www.ncbi.nlm.nih.gov/books/NBK1116/ ), or similar summaries of valid scientific evidence, and/or professional society recommendations, including:
(1 ) Genotype-phenotype information for the reported mutations.
(2 ) Relevant American College of Medical Genetics (ACMG) or American Congress of Obstetricians and Gynecologists (ACOG) guideline recommending testing of the specific gene(s) and variants the test detects and recommended populations, if available. If not available, a statement stating that professional guidelines currently do not recommend testing for this specific gene(s) and variants.
(3 ) Table of expected prevalence of carrier status in major ethnic and racial populations and the general population.
(C) The specimen type (e.g., saliva, whole blood), matrix, and volume.
(D) Assay steps and technology used.
(E) Specification of required ancillary reagents, instrumentation, and equipment.
(F) Specification of the specimen collection, processing, storage, and preparation methods.
(G) Specification of risk mitigation elements and description of all additional procedures, methods, and practices incorporated into the directions for use that mitigate risks associated with testing.
(H) Information pertaining to the probability of test failure (e.g., failed quality control) based on data from clinical samples, description of scenarios in which a test can fail (i.e., low sample volume, low DNA concentration, etc.), how customers will be notified, and followup actions to be taken.
(I) Specification of the criteria for test result interpretation and reporting.
(ii) Information that demonstrates the performance characteristics of the device, including:
(A) Accuracy (method comparison) of study results for each claimed specimen type.
(1 ) Accuracy of the device shall be evaluated with fresh clinical specimens collected and processed in a manner consistent with the device's instructions for use. If this is impractical, fresh clinical samples may be substituted or supplemented with archived clinical samples. Archived samples shall have been collected previously in accordance with the device's instructions for use, stored appropriately, and randomly selected. In some instances, use of contrived samples or human cell line samples may also be appropriate; the contrived or human cell line samples shall mimic clinical specimens as much as is feasible and provide an unbiased evaluation of the device's accuracy.
(2 ) Accuracy must be evaluated as compared to bidirectional sequencing or other methods identified as appropriate by FDA. Performance criteria for both the comparator method and device must be predefined and appropriate to the test's intended use. Detailed appropriate study protocols must be provided.
(3 ) Information provided shall include the number and type of specimens, broken down by clinically relevant variants, that were compared to bidirectional sequencing or other methods identified as appropriate by FDA. The accuracy, defined as positive percent agreement (PPA) and negative percent agreement (NPA), must be measured; accuracy point estimates must be greater than 99 percent (both per reported variant and overall) and uncertainty of the point estimate must be presented using the 95 percent confidence interval. Clinical specimens must include both homozygous wild type and heterozygous genotypes. The number of clinical specimens for each variant reported that must be included in the accuracy study must be based on the variant prevalence. Common variants (greater than 0.1 percent allele frequency in ethnically relevant population) must have at least 20 unique heterozygous clinical specimens tested. Rare variants (less than or equal to 0.1 percent allele frequency in ethnically relevant population) shall have at least three unique mutant heterozygous specimens tested. Any no calls (i.e., absence of a result) or invalid calls (e.g., failed quality control) in the study must be included in accuracy study results and reported separately. Variants that have a point estimate for PPA or NPA of less than 99 percent (incorrect test results as compared to bidirectional sequencing or other methods identified as appropriate by FDA) must not be incorporated into test claims and reports. Accuracy measures generated from clinical specimens versus contrived samples or cell lines must be presented separately. Results must be summarized and presented in tabular format, by sample and by genotype. Point estimate of PPA should be calculated as the number of positive results divided by the number of specimens known to harbor variants (mutations) without "no calls" or invalid calls. The point estimate of NPA should be calculated as the number of negative results divided by the number of wild type specimens tested without "no calls" or invalid calls, for each variant that is being reported. Point estimates should be calculated along with 95 percent two-sided confidence intervals.
(4 ) Information shall be reported on the clinical positive predictive value (PPV) and negative predictive value (NPV) for carrier status (and where possible, for each variant) in each population. Specifically, to calculate PPV and NPV, estimate test coverage (TC) and the percent of persons with variant(s) included in the device among all carriers: PPV = (PPA * TC * [pi])/(PPA * TC * [pi] + (1 - NPA) * (1 - [pi])) and NPV = (NPA * (1 - [pi]))/(NPA *(1 - [pi]) + (1 - PPA*TC) * [pi]) where PPA and NPA described either in paragraph (b)(3)(ii)(A)(4 )(i ) or in paragraph (b)(3)(ii)(A)(4 )(ii ) of this section and [pi] is prevalence of carriers in the population (pre-test risk to be a carrier for the disease).
(i ) For the point estimates of PPA and NPA less than 100 percent, use the calculated estimates in the PPV and NPV calculations.
(ii ) Point estimates of 100 percent may have high uncertainty. If these variants are measured using highly multiplexed technology, calculate the random error rate for the overall device and incorporate that rate in the estimation of the PPA and NPA as calculated previously. Then use these calculated estimates in the PPV and NPV calculations. This type of accuracy study is helpful in determining that there is no systematic error in such devices.
(B) Precision (reproducibility): Precision data must be generated using multiple instruments and multiple operators, on multiple non-consecutive days, and using multiple reagent lots. The sample panel must include specimens with claimed sample type (e.g. saliva samples) representing different genotypes (i.e., wild type, heterozygous). Performance criteria must be predefined. A detailed study protocol must be created in advance of the study and then followed. The "failed quality control" rate must be indicated. It must be clearly documented whether results were generated from clinical specimens, contrived samples, or cell lines. The study results shall state, in a tabular format, the variants tested in the study and the number of replicates for each variant, and what testing conditions were studied (i.e.,number of runs, days, instruments, reagent lots, operators, specimens/type, etc). The study must include all nucleic acid extraction steps from the claimed specimen type or matrix, unless a separate extraction study for the claimed sample type is performed. If the device is to be used at more than one laboratory, different laboratories must be included in the precision study (and reproducibility must be evaluated). The percentage of "no calls" or invalid calls, if any, in the study must be provided as a part of the precision (reproducibility) study results.
(C) Analytical specificity data: Data must be generated evaluating the effect on test performance of potential endogenous and exogenous interfering substances relevant to the specimen type, evaluation of cross-reactivity of known cross-reactive alleles and pseudogenes, and assessment of cross-contamination.
(D) Analytical sensitivity data: Data must be generated demonstrating the minimum amount of DNA that will enable the test to perform accurately in 95 percent of runs.
(E) Device stability data: The manufacturer must establish upper and lower limits of input nucleic acid and sample stability that will achieve the claimed accuracy and reproducibility. Data supporting such claims must be described.
(F) Specimen type and matrix comparison data: Specimen type and matrix comparison data must be generated if more than one specimen type or anticoagulant can be tested with the device, including failure rates for the different specimen types.
(iii) If the device is offered over-the-counter, including cases in which the test results are provided direct-to-consumer, the manufacturer must conduct a study that assesses user comprehension of the device's labeling and test process and provide a concise summary of the results of the study. The following items must be included in the user study:
(A) The test manufacturer must perform pre- and post-test user comprehension studies to assess user ability to understand the possible results of a carrier test and their clinical meaning. The comprehension test questions must directly evaluate the material being presented to the user in the test reports.
(B) The test manufacturer must provide a carrier testing education module to potential and actual test report recipients. The module must define terms that are used in the test reports and explain the significance of carrier status.
(C) The user study must meet the following criteria:
(1 ) The study participants must be comprised of a statistically justified and demographically diverse population (determined using methods such as quota-based sampling) that is representative of the intended user population. Furthermore, the users must be comprised of a diverse range of age and educational levels that have no prior experience with the test or its manufacturer. These factors shall be well-defined in the inclusion and exclusion criteria.
(2 ) All sources of bias (e.g., non-responders) must be predefined and accounted for in the study results with regard to both responders and non-responders.
(3 ) The testing must follow a format where users have limited time to complete the studies (such as an onsite survey format and a one-time visit with a cap on the maximum amount of time that a participant has to complete the tests).
(4 ) Users must be randomly assigned to study arms. Test reports given to users must: Define the condition being tested and related symptoms; explain the intended use and limitations of the test; explain the relevant ethnicities regarding the variant tested; explain carrier status and relevance to the user's ethnicity; and provide links to additional information pertaining to situations where the user is concerned about their test results or would like followup information as indicated in test labeling. The study shall assess participants' ability to understand the following comprehension concepts: The test's limitations, purpose, and results.
(5 ) Study participants must be untrained, naive to the test subject of the study, and be provided only the materials that will be available to them when the test is marketed.
(6 ) The user comprehension study must meet the predefined primary endpoint criteria, including a minimum of a 90 percent or greater overall comprehension rate (i.e. selection of the correct answer) for each comprehension concept to demonstrate that the education module and test reports are adequate for over-the-counter use.
(D) A summary of the user comprehension study must be provided and include the following:
(1 ) Results regarding reports that are provided for each gene/variant/ethnicity tested.
(2 ) Statistical methods used to analyze all data sets.
(3 ) Completion rate, non-responder rate, and reasons for non-response/data exclusion, as well as a summary table of comprehension rates regarding comprehension concepts (purpose of test, test results, test limitations, ethnicity relevance for the test results, etc.) for each study report.
(4) Your 21 CFR 809.10 compliant labeling and any test report generated must include the following warning and limitation statements, as applicable:
(i) A warning that reads "The test is intended only for autosomal recessive carrier screening in adults of reproductive age."
(ii) A statement accurately disclosing the genetic coverage of the test in lay terms, including, as applicable, information on variants not queried by the test, and the proportion of incident disease that is not related to the gene(s) tested. For example, where applicable, the statement would have to include a warning that the test does not or may not detect all genetic variants related to the genetic disease, and that the absence of a variant tested does not rule out the presence of other genetic variants that may be disease-related. Or, where applicable, the statement would have to include a warning that the basis for the disease for which the genetic carrier status is being tested is unknown or believed to be non-heritable in a substantial number of people who have the disease, and that a negative test result cannot rule out the possibility that any offspring may be affected with the disease. The statement would have to include any other warnings needed to accurately convey to consumers the degree to which the test is informative for carrier status.
(iii) For prescription use tests, the following warnings that read:
(A) "The results of this test are intended to be interpreted by a board-certified clinical molecular geneticist or equivalent and should be used in conjunction with other available laboratory and clinical information."
(B) "This device is not intended for disease diagnosis, prenatal testing of fetuses, risk assessment, prognosis or pre-symptomatic testing, susceptibility testing, or newborn screening."
(iv) For over-the-counter tests, a statement that reads "This test is not intended to diagnose a disease, or tell you anything about your risk for developing a disease in the future. On its own, this test is also not intended to tell you anything about the health of your fetus, or your newborn child's risk of developing a particular disease later on in life."
(v) For over-the-counter tests, the following warnings that read:
(A) "This test is not a substitute for visits to a healthcare provider. It is recommended that you consult with a healthcare provider if you have any questions or concerns about your results."
(B) "The test does not diagnose any health conditions. Results should be used along with other clinical information for any medical purposes."
(C) "The laboratory may not be able to process your sample. The probability that the laboratory cannot process your saliva sample can be up to [actual probability percentage]."
(D) "Your ethnicity may affect how your genetic health results are interpreted."
(vi) For a positive result in an over-the-counter test when the positive predictive value for a specific population is less than 50 percent and more than 5 percent, a warning that reads "The positive result you obtained may falsely identify you as a carrier. Consider genetic counseling and followup testing."
(vii) For a positive result in an over-the-counter test when the positive predictive value for a specific population is less than 5 percent, a warning that reads "The positive result you obtained is very likely to be incorrect due to the rarity of this variant. Consider genetic counseling and followup testing."
(5) The testing done to comply with paragraph (b)(3) of this section must show the device meets or exceeds each of the following performance specifications:
(i) The accuracy must be shown to be equal to or greater than 99 percent for both PPA and NPA. Variants that have a point estimate for PPA or NPA of less than 99 percent (incorrect test results as compared to bidirectional sequencing or other methods identified as appropriate by FDA) must not be incorporated into test claims and reports.
(ii) Precision (reproducibility) performance must meet or exceed 99 percent for both positive and negative results.
(iii) The user comprehension study must obtain values of 90 percent or greater user comprehension for each comprehension concept.
(6) The distribution of this device, excluding the collection device described in paragraph (b)(2) of this section, shall be limited to the manufacturer, the manufacturer's subsidiaries, and laboratories regulated under the Clinical Laboratory Improvement Amendments.