On Biostatistics and Clinical Trials

Regulatory Biostatistics: A Comprehensive Analysis of the FDA Guidance on Bayesian Methodology in Drug and Biological Product Development

2026-03-30T21:26:00.003-04:00

The release of the January 2026 FDA Draft Guidance for Industry, titled "Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products," marks a definitive transition in the regulatory philosophy governing the evidence required for therapeutic approvals.1 While Bayesian methods have resided on the periphery of drug development for decades—often relegated to early-phase dose-finding or utilized in medical device trials where a 2010 guidance provided a formal pathway—the 2026 document signals a symbolic legitimacy for these methods within the Center for Drug Evaluation and Research (CDER) and the Center for Biologics Evaluation and Research (CBER).1 This shift addresses two of the most persistent challenges in modern pharmaceutical research: the rising cost of development and the logistical hurdles of evaluating treatments for small, niche patient populations.4

The Conceptual Shift from Frequentism to Bayesian Inference

The traditional frequentist paradigm, which has dominated clinical research for nearly a century, treats each trial as an isolated experiment.4 It relies on the p-value—a measure of the probability of observing data at least as extreme as the current results, assuming the null hypothesis of no treatment effect is true—to serve as the arbiter of success.2 In contrast, the Bayesian framework treats medical knowledge as a cumulative process, formally integrating existing information with new study data to update the probability of a hypothesis.4

The Mathematical Mechanism of Knowledge Updating

The core of the Bayesian approach is the application of Bayes’ Theorem, which provides a mathematical formula for updating a prior distribution with a likelihood function to produce a posterior distribution.2 The prior distribution encapsulates information available before the current trial, which may include data from previous clinical phases, non-clinical models, or systematic reviews.2 The likelihood function represents the evidence provided by the current trial data, and the resulting posterior distribution expresses the updated state of knowledge regarding the treatment effect.2

This mechanism allows for a more direct interpretation of trial results than the frequentist model. Instead of stating that the null hypothesis is rejected at a specific significance level, a Bayesian analysis might conclude that there is a 95% posterior probability that the treatment effect exceeds a certain threshold of clinical benefit.4 This intuitive framing is particularly valuable for clinicians and decision-makers who are fundamentally interested in the probability that a drug will be effective for their patients.7

Core Definitions and Terminologies within the Guidance

The FDA guidance carefully defines several technical components essential for the implementation of Bayesian designs. These components ensure that the transition from frequentist to Bayesian methods is grounded in statistical rigor and transparency.

Component	Definition and Regulatory Context
Prior Distribution	A probability distribution capturing pre-study information. Can range from non-informative (providing no "borrowing") to highly informative (using external data). 3
Likelihood Function	The probability of the observed trial data given the parameters of the model. It defines how current evidence is weighted. 2
Posterior Distribution	The final inferential result combining the prior and likelihood. It serves as the basis for primary efficacy and safety conclusions. 2
Bayesian Power	The probability of a trial meeting its success criterion, averaged over the prior distribution. Unlike frequentist power, it accounts for prior uncertainty. 3
Success Criterion	Often expressed as , where the posterior probability of a benefit () must exceed a high threshold (). 5

Strategic Use Cases: When Bayesian Methods are Preferred

The guidance does not suggest that Bayesian methods should replace frequentism in all contexts. Rather, it identifies specific high-value scenarios where the Bayesian approach offers clear advantages in terms of ethics, efficiency, and evidentiary depth.1

Small Populations: Rare Diseases and Pediatric Extrapolation

One of the most profound applications of the Bayesian framework is in settings where patient recruitment is severely limited.9 In rare diseases, which may affect fewer than 5 in 10,000 people, conducting a large-scale randomized controlled trial (RCT) with traditional frequentist power is often impossible.9 Bayesian methods allow researchers to use "informative priors" to borrow information from external sources, such as natural history studies or registries, effectively reducing the number of patients required for the concurrent control arm.9

Similarly, in pediatric drug development, the FDA encourages the use of extrapolation approaches.1 If a drug has established efficacy in adults and the disease process is biologically similar in children, Bayesian hierarchical models can integrate the adult data as a prior for the pediatric trial.2 This minimizes the exposure of children to experimental treatments while still providing a rigorous statistical basis for approval.2

Early-Phase Oncology and Dose-Finding

The transition from toxicity-based dose-finding (like the traditional 3+3 design) to more sophisticated Bayesian models like the Continual Reassessment Method (CRM) has been one of the most successful applications of Bayesian logic in oncology.6 The CRM utilizes a Bayesian model to identify the maximum tolerated dose (MTD) more efficiently by updating the dose-toxicity curve after every patient or cohort.6 This allows for faster escalation to therapeutic doses and prevents the over-enrollment of patients at sub-therapeutic levels.6

Adaptive and Platform Trials

Bayesian methods are inherently suited for adaptive designs because the Bayesian measure of uncertainty is generally unaffected by the presence of interim looks.10 Unlike frequentist group sequential designs, which require strict "multiplicity adjustments" to preserve the Type I error rate when multiple interim analyses are conducted, Bayesian trials can use posterior predictive probabilities to stop for efficacy or futility at any point without traditional penalties.3

In platform trials—where multiple treatment arms are tested against a common control—Bayesian methods allow for the seamless addition or removal of arms.4 This "perpetual" trial design, such as the I-SPY 2 oncology platform, uses Bayesian probability to graduate treatments to Phase 3 only when there is a high likelihood of success, thereby optimizing resource allocation across a portfolio of candidates.4

Clinical Setting	Preferred Method	Rationale for Preference
Large-scale Phase 3 (Common Disease)	Frequentist	Standardized, well-understood by all global regulators, and power is easily achieved through enrollment. 9
Rare Disease / Small N	Bayesian	Ability to leverage informative priors and natural history data to achieve meaningful inference from limited data. 9
Pediatric Trials	Bayesian	Formal mechanism for extrapolation from adult data, reducing the ethical burden of enrolling children. 1
Complex Adaptive Designs	Bayesian	More intuitive stopping rules (predictive probability) and lower "multiplicity penalties" for frequent interim looks. 3
Dose-Finding / Phase 1	Bayesian	More accurate identification of MTD and better patient safety through continuous model updating. 6

Theoretical Nuances and Mathematical Rigor in the Guidance

A significant portion of the guidance is dedicated to the technical requirements for justifying a Bayesian design. The FDA emphasizes that the flexibility of Bayesian methods must be balanced with a high degree of transparency and pre-specification.2

The Prior Distribution: Informative vs. Non-informative

The choice of the prior distribution is arguably the most scrutinized aspect of a Bayesian submission. The guidance categorizes priors based on their level of information. Non-informative or "weakly informative" priors are often used when the sponsor wishes to use the Bayesian framework for its interpretive clarity but does not have (or does not wish to use) external data.3 In these cases, the Bayesian result will closely mirror the frequentist result, and the FDA typically expects traditional control of the Type I error rate.11

Informative priors, however, are used to "borrow" information.3 The guidance stipulates that the source of this information must be relevant and that the process for constructing the prior must be systematic to avoid bias.2 A common challenge is "prior-data conflict," where the results of the current trial significantly differ from the pre-study information.3 In such cases, the guidance suggests the use of "robust" priors or "dynamic borrowing" mechanisms (like hierarchical models) that automatically reduce the weight of the prior if it is found to be discordant with the new data.6

Success Criteria and Benefit-Risk Assessments

The guidance provides a formal Success Criterion Based on Benefit-Risk Assessment or Decision-Theoretic Approaches.1 This allows a sponsor to define success not just as a statistical "win" but as a balance of potential consequences.1 For instance, a decision-theoretic approach might weigh the negative consequence of approving an ineffective drug against the consequence of rejecting a drug that actually works, particularly in high-unmet-need areas.1 This alignment with "real-world" medical decision-making is a hallmark of the Bayesian philosophy.4

Scholarly Perspectives and Industry Reactions

The publication of the draft guidance has sparked a series of high-level discussions in JAMA and on professional forums like DataMethods, reflecting both enthusiasm for modernization and significant concern regarding evidentiary standards.3

The "Embracing" Perspective: Modernizing Clinical Research

J. Jack Lee, Frank Harrell, and colleagues argue in their JAMA Perspective that the Bayesian approach "tackles the real question of interest head-on".7 They assert that by computing the probability of treatment benefit, Bayesian methods provide a direct answer to whether a drug works, rather than the indirect and often misinterpreted p-value.7 This group views the guidance as a critical step toward a more ethical clinical trial ecosystem where data is not "thrown away" but used to optimize decisions on a trial-by-trial basis.3

Key advantages highlighted by this perspective include:

Reduced Patient Exposure: Faster stopping rules for both efficacy and futility ensure that fewer patients are treated with inferior regimens.8
Efficiency: The accumulation of data over time is better suited for a framework that incorporates that data explicitly, potentially reducing the time to market for life-saving therapies.4
Intuition: Posterior probabilities are more easily understood by patients and clinicians than frequentist confidence intervals.4

The Gelman Perspective: Hierarchical Models and Skeptical Priors

Andrew Gelman and colleagues contribute a more methodological view, emphasizing that the "subjectivity" often attributed to Bayesian priors is better framed as "pre-study information".11 They applaud the guidance for recommending simulations and posterior predictive checks to evaluate model fit.7

Gelman introduces a critical nuance regarding "skeptical priors".7 While some might use a skeptical prior centered at a negative value to make success harder to achieve, Gelman recommends a prior centered at zero.7 This assumes a "world in which most effects are small," requiring the current trial data to be robust enough to overcome the initial assumption of neutrality.7

Type of Prior (Gelman)	Characteristic	Regulatory Application
Non-informative	Provides minimal weight to pre-study information.	Functionally similar to frequentism; requires Type I error control. 11
Skeptical (at zero)	Assumes the treatment effect is likely zero or very small.	Used to protect against "false positives" by requiring strong data to shift the posterior. 7
Optimistic	Assumes a positive treatment effect based on early phase data.	Highly scrutinized; requires extreme justification to avoid bias. 1
Hierarchical	Allows the data to determine the degree of borrowing across groups.	Ideal for subgroup analysis or multicenter trials with potential heterogeneity. 7

Critical Critiques: Protecting Scientific Integrity

Not all voices in the community are unequivocally supportive. Scott Evans, Thomas Fleming, and others raised significant concerns in their JAMA "Reflections" piece regarding the potential for Bayesian methods to lower the "Evidence Bar" and compromise scientific integrity.15

The Debate Over Response-Adaptive Randomization (RAR)

A focal point of the critical discourse is the use of Response-Adaptive Randomization (RAR), a technique often enabled by Bayesian updating where more patients are assigned to the treatment arm that is performing better.15 Evans and Fleming argue that RAR, while "noble in intent," is fraught with practical dangers:

Bias from Temporal Trends: In long-running trials, the patient population or standard of care may change over time. RAR can inadvertently assign more "later-stage" patients to one arm, confounding the treatment effect with a temporal trend.15
Inefficiency and Volatility: RAR can lead to highly variable sample sizes and, in some cases, can actually assign more patients to an inferior arm by chance in the early stages of a trial.15
Erosion of Randomization: They conclude that RAR can eliminate the very benefits that randomization is designed to provide—the balance of unknown confounders across treatment groups.15

Evidentiary Standards and Objectivity

Critics also worry that the "subjectivity" of the prior distribution could be exploited by sponsors to achieve favorable results.15 They emphasize that the authority of trial results depends on the quality of the protocol and the protection of randomization integrity.15 This group advocates for a "treatment policy" approach that focuses on clinically relevant causal effects rather than complex Bayesian model-based inferences that may be difficult to validate.15

Technical Implementation and the Role of Simulation

The FDA guidance makes it clear that the evaluation of Bayesian designs relies heavily on statistical simulations rather than closed-form mathematical proofs.6 This is because the "operating characteristics" of a Bayesian design—such as its power and Type I error rate—depend on the complex interaction between the chosen prior, the trial design, and the possible true treatment effects.3

Operating Characteristics: Bayesian vs. Frequentist Lens

There is an ongoing technical debate regarding which operating characteristics are most relevant. Frequentists insist on simulating the Type I error rate (), but Bayesian practitioners argue that this focuses on "long-run" performance over many hypothetical trials.3 Instead, they suggest that the "correctness of the current decision" and the "expected bias" are more meaningful metrics.3

Operating Characteristic	Metric Definition	Regulatory Perspective
Type I Error Rate ()	Probability of a false positive result.	FDA still emphasizes control, especially for non-informative priors. 3
Bayesian Power	Probability of success averaged over a prior.	Used to determine sample size for Bayesian trials. 3
Prob. of Correct Decision	Positive Predictive Value of the trial result.	Gaining traction as a measure of "success" in decision-theoretic designs. 5
MSE / Bias	Accuracy and precision of the effect estimate.	Critical for ensuring that "borrowing" doesn't lead to overestimation of benefit. 3

Software Efficiency and Computational Hurdles

The transition to Bayesian methodology is also a computational challenge. Traditional software like JAGS can be prohibitively slow for the complex hierarchical models required by modern innovative designs.17 The R package NIMBLE has emerged as a more efficient alternative, sometimes outperforming JAGS by a factor of 10 or more by allowing for customized MCMC procedures.18 The FDA guidance recognizes this need for computational power and expects sponsors to provide extensive documentation on their simulation methods and software code to enable a thorough regulatory review.6

Operational Challenges and the "Education Gap"

Despite the guidance providing a regulatory framework, practitioners on the DataMethods forum highlight several real-world barriers to adoption.

The "First-Learned Paradigm" Bias

A primary hindrance is the lack of Bayesian training in graduate schools.3 Many biostatisticians are more comfortable with frequentism because it is what they first learned and practiced.3 This creates a situation where the biostatistical community might "cling" to traditional methods simply due to familiarity, even when a Bayesian approach would be more appropriate.3

Regulatory Consistency and the EMA Contrast

Another operational concern is the perceived "lag" or difference in philosophy between the FDA and the European Medicines Agency (EMA). While the FDA treats Bayesian inference as a coherent and independent framework, the EMA has been criticized for viewing Bayesian methods through a "fundamentally frequentist lens" that emphasizes error control above all else.3 This regulatory dissonance can be a significant barrier for sponsors looking to conduct global clinical trials.3

Feature	FDA Approach (2026 Guidance)	EMA Approach (Concept Paper)
Philosophy	Bayes as a coherent, independent framework.	Bayes through a frequentist "error control" lens.
Informative Priors	Welcomed in specific settings (Pediatrics, Rare Disease).	Requires "special justification" and high skepticism.
Success Metrics	Posterior probabilities and decision-theory.	Strong focus on maintaining p-value equivalents.
Timeline	Active guidance with demonstration projects (C3TI).	Final reflection paper not expected until June 2028.

Real-World Evidence and the Future of Drug Development

The FDA's shift toward Bayesian methodologies is inextricably linked to the broader movement toward using Real-World Data (RWD) and virtual modeling in drug development.4

Digital Twins and Virtual Patients

The guidance opens the door for using "digital twins"—AI-generated virtual representations of patients—to predict outcomes.4 By using RWD to predict how a patient would respond to a placebo, sponsors can potentially reduce the size of the actual control arm by 25% to 50%.4 For a large Phase 3 trial, such as in Alzheimer’s disease, reducing the control group by several hundred patients could save over $100 million in development costs.4

Proven Regulatory Successes

The guidance highlights several examples of Bayesian success that serve as precedents for the industry 4:

Dulaglutide (Trulicity): Utilized a Bayesian adaptive Phase 2/3 design with bi-weekly randomization updates to select optimal doses.4
REBYOTA (Fecal Microbiota Product): Incorporated Phase 2 data into its Phase 3 primary analysis using a Bayesian hierarchical model when recruitment was difficult.4
Cardiovascular Outcomes (Liraglutide): Used a Bayesian hierarchical model to estimate treatment effects across different geographic regions.6

Actionable Recommendations and Strategic Conclusions

The 2026 FDA guidance is a signal that the biostatistical landscape is maturing. For sponsors and biostatisticians, the following conclusions emerge from the synthesis of the guidance and the surrounding expert discourse.

When to Embrace Bayesian Methods

Bayesian methods should be the default consideration in therapeutic areas where patient recruitment is a bottleneck, such as rare diseases, pediatric assessments, and highly segmented oncology populations.9 The ability to borrow information from adults to children or from natural history to concurrent controls provides a scientifically sound pathway to approval that frequentism simply cannot match in small-sample settings.1 Furthermore, in early-phase development, the use of CRM models for dose-finding is increasingly becoming a regulatory expectation rather than an option, given its superior safety and efficiency profile.6

How to Ensure Regulatory Success

Success in a Bayesian submission requires a move away from the "black box" approach. The FDA’s primary requirement is transparency.2 Sponsors must:

Systematically Justify Priors: The source of external information must be meticulously documented, and the weight given to that information must be pre-specified.2
Simulate Extensively: Operating characteristics must be evaluated under a wide range of assumptions, including "pessimistic" scenarios where the treatment effect is zero or where the prior data is misleading.3
Engage Early via Pilot Programs: Utilizing the FDA’s CID and C3TI programs allows for collaborative design development, reducing the risk of a late-stage regulatory rejection.3

Addressing the Criticism

The criticisms raised by Evans and Fleming should not be viewed as roadblocks but as design constraints.15 If a design uses Response-Adaptive Randomization, the sponsor must provide evidence through simulations that temporal trends will not bias the result.15 If an informative prior is used, the sponsor should conduct sensitivity analyses showing that the trial result would still hold under a more "skeptical" prior or a non-informative prior.3

The 2026 guidance does not "lower the evidence bar"—it shifts the focus of the bar from long-run error control to the precision and correctness of the specific decision at hand.3 By formalizing the incorporation of prior knowledge, the FDA is providing the industry with the tools to solve the "productivity crisis" in drug development, while simultaneously ensuring that the integrity of the randomized clinical trial remains the bedrock of medical evidence.4

References:

FDA guidance on Bayesian clinical trials, accessed March 29, 2026, https://www.fda.gov/media/190505/download
FDA Issues Guidance on Modernizing Statistical Methods for Clinical Trials, accessed March 29, 2026, https://www.bigmoleculewatch.com/2026/02/04/fda-issues-guidance-on-modernizing-statistical-methods-for-clinical-trials/
FDA Draft Guidance: Use of Bayesian Methodology in Clinical Trials ..., accessed March 29, 2026, https://discourse.datamethods.org/t/fda-draft-guidance-use-of-bayesian-methodology-in-clinical-trials-of-drug-and-biological-products/28598
The FDA's Shift Toward Modern Statistical Methodologies • Infiuss ..., accessed March 29, 2026, https://infiuss.com/insights/the-fdas-shift-toward-modern-statistical-methodologies
The FDA Just Rewrote the Rulebook for Clinical Trials: Here Are the 4 Biggest Takeaways from the FDA's New Draft Guidance on Bayesian Methodology - Bayesoft - Bayes + AI Innovation, accessed March 29, 2026, https://www.bayesics.ai/blogs/fda-rewrote-rulebook-clinical-trials
Guidance Recap Podcast | Use of Bayesian Methodology in Clinical Trials of Drug and Biological Products | FDA, accessed March 29, 2026, https://www.fda.gov/drugs/guidances-drugs/guidance-recap-podcast-use-bayesian-methodology-clinical-trials-drug-and-biological-products
Statistical Modeling, Causal Inference, and Social Science, accessed March 29, 2026, https://statmodeling.stat.columbia.edu/
Embracing Bayesian Methods in Clinical Trials: FDA's Long-Awaited Draft Guidance, accessed March 29, 2026, https://jamanetwork.com/journals/jama/fullarticle/2847011
Bayesian vs. Frequentist Approaches in Rare Disease Trials - Phastar, accessed March 29, 2026, https://phastar.com/knowledge-centre/blogs/bayesian-vs-frequentist-approaches-in-rare-disease-trials/
Bayesian Strategies in Rare Diseases - PMC - NIH, accessed March 29, 2026, https://pmc.ncbi.nlm.nih.gov/articles/PMC9789883/
Bayesian Statistics | Statistical Modeling, Causal Inference, and Social Science, accessed March 29, 2026, https://statmodeling.stat.columbia.edu/category/bayesian-statistics/
Decision Analysis | Statistical Modeling, Causal Inference, and Social Science, accessed March 29, 2026, https://statmodeling.stat.columbia.edu/category/decision-theory/
Why and how to do Bayes for clinical trials: Our comments on the recent FDA draft guidance, and reactions to two comments by others, accessed March 29, 2026, https://statmodeling.stat.columbia.edu/2026/03/26/considering-some-anti-bayesian-arguments-in-the-context-of-fda-draft-guidance-for-the-use-of-bayesian-methods-in-clinical-trials/
A Bayesian Analysis of a Cognitive-Behavioral Therapy Intervention for High-Risk People on Probation | Request PDF - ResearchGate, accessed March 29, 2026, https://www.researchgate.net/publication/376350746_A_Bayesian_Analysis_of_a_Cognitive-Behavioral_Therapy_Intervention_for_High-Risk_People_on_Probation
Reflections on FDA Draft Guidance on Bayesian Methods in Trials—Protecting Scientific Integrity and Evidentiary Standards - ResearchGate, accessed March 29, 2026, https://jamanetwork.com/journals/jama/fullarticle/2847013
Reflections on FDA Draft Guidance on Bayesian Methods in Trials—Protecting Scientific Integrity and Evidentiary Standards | Scilit, accessed March 29, 2026, https://www.scilit.com/publications/52545a91d08b0d5cc434dc35ce61505f
Introduction to Bayesian Methods in Ecology and Natural Resources - ResearchGate, accessed March 29, 2026, https://www.researchgate.net/publication/347191453_Introduction_to_Bayesian_Methods_in_Ecology_and_Natural_Resources
Bayesian Analysis of Capture-Recapture Data with Hidden Markov Models: Theory and Case Studies in R and NIMBLE | Request PDF - ResearchGate, accessed March 29, 2026, https://www.researchgate.net/publication/400149333_Bayesian_Analysis_of_Capture-Recapture_Data_with_Hidden_Markov_Models_Theory_and_Case_Studies_in_R_and_NIMBLE

Biostatistical and Ethical Determinants of Sham-Controlled Clinical Trials: A Deep Dive into Gene Therapy for Huntington’s Disease and the Regulatory Landscape -The Theoretical Framework of Sham Controls in Procedural Clinical Trials

2026-03-24T23:30:00.001-04:00

The use of sham controls in clinical investigations represents one of the most complex intersections of biostatistical rigor, surgical ethics, and regulatory policy. In the hierarchy of clinical evidence, the randomized, double-blind, placebo-controlled trial remains the gold standard for establishing the safety and efficacy of therapeutic interventions. However, when the intervention is not a pharmacological agent but a surgical or invasive procedure, the "placebo" must take the form of a sham—a procedure that mimics the active treatment in every possible respect except for the specific therapeutic mechanism under investigation.

The primary biostatistical justification for the sham control is the mitigation of the placebo effect, which is notably pronounced in invasive interventions. Research indicates that the more invasive a procedure, the stronger the patient’s "will to believe" in its efficacy, leading to significant improvements in subjective endpoints that are not attributable to the biological action of the treatment itself. In diseases with high inter-patient variability and subjective outcome measures, such as neurodegenerative disorders, the absence of a concurrent blinded control arm can lead to "self-deception" by both the investigator and the participant.

The ethical debate surrounding sham procedures centers on the principle of non-maleficence. Unlike an inert sugar pill, a sham surgery involves real risks—anesthesia, incisions, and potential infection—without the prospect of direct clinical benefit to the control subject. Consequently, regulatory bodies like the Food and Drug Administration (FDA) and the European Medicines Agency (EMA) must weigh the "altruistic gift" of the research subject against the societal necessity of preventing the widespread adoption of ineffective or harmful procedures.

Historical Context and the Necessity of Blinding in Surgery

The historical evolution of surgical clinical trials demonstrates a recurring pattern where open-label procedures, initially perceived as breakthroughs, were subsequently found to be no more effective than sham controls in rigorous trials. For instance, in the late 20th century, fetal tissue implantation for Parkinson’s disease was heralded as a potential cure based on promising results from uncontrolled observational studies. However, when these treatments were subjected to randomized, sham-controlled trials, the long-term results showed little to no difference between the active and control arms.6 This was attributed to a massive and sustained surgical placebo effect, where patients who believed they had received a cutting-edge brain surgery reported subjective improvements that were not mirrored by objective biological measures.

Similar phenomena have been documented in orthopedic surgery. In a landmark trial of arthroscopic surgery for osteoarthritis of the knee, participants in the sham arm received only skin incisions and the sounds of irrigation (water splashed to mimic the sounds of lavage) without any joint manipulation.3 These patients reported identical levels of pain relief and functional improvement as those who underwent the actual procedure, leading to a major reevaluation of the procedure's clinical value.3 These cases provide the empirical foundation for the FDA’s current insistence on sham controls for invasive therapies where the endpoint is subjective or the "will to believe" is high.

The UniQure AMT-130 Controversy: A Case Study in Regulatory Friction

The development of AMT-130, an investigational AAV5-based gene therapy for Huntington’s disease (HD), has become a focal point for the debate over the feasibility and ethics of sham controls in the gene therapy era. Huntington’s disease is a rare, fatal, autosomal dominant neurodegenerative disorder characterized by motor, cognitive, and psychiatric decline. Because there are currently no disease-modifying therapies for HD, the pressure to bring AMT-130 to market is immense.9

The Core of the Argument: Internal vs. External Controls

The dispute between the sponsor, UniQure, and the FDA centers on whether data from a Phase I/II study compared to an external natural history control is sufficient for a Biologics License Application (BLA). UniQure’s Phase I/II trial reported a 75% slowing of disease progression on the Unified Huntington's Disease Rating Scale (UHDRS) after three years when compared to a propensity-score-matched cohort from the Enroll-HD natural history database. UniQure argued that this "totality of data" demonstrated a clear therapeutic signal and that requiring a new Phase III sham-controlled study would be unethical and would unnecessarily delay access to a life-saving therapy.

The FDA, however, flatly rejected this approach. In a strikingly blunt communication, senior FDA officials characterized the comparison to external controls as "distorted" and "manipulated". The agency’s primary concern was an internal inconsistency in UniQure’s data: a small cohort of U.S. patients who were randomized to a sham control in the same Phase I/II program showed no treatment effect at 12 months when compared to the sham group. Conversely, when those same treated patients were compared to the external natural history control at 12 months, they appeared to show a benefit.5 This discrepancy suggests that the apparent "benefit" was a byproduct of the lack of comparability between the trial participants and the external database—specifically the impact of intense trial monitoring and the placebo effect—rather than the gene therapy itself.

The Technical Challenges of AMT-130 Administration

The administration of AMT-130 is uniquely invasive, which fuels the ethical debate. The therapy requires a neurosurgical procedure where the product is delivered directly into the striatum (caudate and putamen) through small holes (burr holes) in the skull using MRI-guided convection-enhanced delivery. UniQure leaders argued that subjecting patients to such an invasive procedure for a placebo arm crossed an ethical threshold, as the control subjects would be exposed to the risks of intracranial surgery with zero possibility of benefit.

In response, the FDA clarified that they were not requesting a "partial sham burr hole" or a full intracranial injection of an inert substance.5 Instead, the agency requested a control arm designed to maintain the blind while minimizing risk: patients would be anesthetized and given "one to three nicks in the scalp" to simulate the surgical incisions.5 The FDA argued that this 30-minute procedure under anesthesia was a minimized risk that was necessary to ensure the validity of the study results, especially given the high heterogeneity of Huntington’s disease and the susceptibility of its rating scales to placebo and practice effects.

Stakeholder	Primary Position	Key Argument
UniQure (Sponsor)	Requesting approval based on external controls.	External data shows 75% slowing of decline; sham surgery is unethical for intracranial delivery.5
FDA (Regulatory Body)	Requiring a randomized, sham-controlled Phase III trial.	Phase I/II sham data showed no effect; external comparison is "distorted"; HD heterogeneity requires blinding.5
HD Community	Urgent need for disease-modifying therapy.	Balancing the need for scientific certainty with the urgency of a terminal, untreated condition.9

Biostatistical Challenges in Huntington’s Disease Trials

The FDA’s demand for a sham control is rooted in the specific biostatistical profile of Huntington’s disease. HD progression is typically measured using the UHDRS, which includes the Total Motor Score (TMS), Total Functional Capacity (TFC), and cognitive assessments like the Symbol Digit Modalities Test (SDMT).

Placebo and Practice Effects in the UHDRS

Analyses of natural history data (such as the Enroll-HD study) compared to the placebo arms of large Phase III trials (such as GENERATION HD1) have revealed significant confounding factors that make external controls unreliable :

Motor Score Volatility: The TMS often shows significant improvement immediately after baseline in trial settings, a phenomenon likely driven by the placebo effect and the "will to believe". This improvement can persist for over a year (up to week 69 in some studies) before converging back to natural history predictions. This means that any therapy designed to slow progression would appear "effective" at one year if compared to a natural history cohort that did not experience this trial-related boost.
Cognitive Practice Effects: Cognitive scores like the SDMT and Stroop Word Reading (SWR) often show sustained improvement in trial settings because participants are tested more frequently than in natural history studies. In the GENERATION HD1 trial, cognitive scores showed sustained improvement outside the 95% confidence interval of the natural history model for up to 101 weeks. This "practice effect" can easily be mistaken for a therapeutic benefit if the control group is an external natural history cohort that was only tested annually.
Heterogeneity and CAG Matching: HD is highly variable in its onset and progression rate, which is influenced by the length of the CAG repeat in the HTT gene and other genetic modifiers. While propensity score matching can attempt to balance these factors, it cannot account for the "unmeasured confounding" of being enrolled in a high-intensity clinical trial versus an observational study.

UHDRS Endpoint	Susceptibility to Placebo Effect	Persistence of Effect	Comparison to Natural History
Total Functional Capacity (TFC)	Low	Negligible	Aligns closely with predicted progression.
Total Motor Score (TMS)	Moderate to High	Up to 69 weeks	Significant improvement post-baseline in trials.
Symbol Digit Modalities Test (SDMT)	High (Practice Effect)	Up to 101 weeks	Sustained improvement over natural history baseline.
Stroop Word Reading (SWR)	High (Practice Effect)	Up to 101 weeks	Sustained improvement over natural history baseline.

The FDA's stance is that without a concurrent, blinded control group, it is impossible to distinguish between the practice effect, the placebo effect, and the actual biological impact of AMT-130. This is particularly critical for AMT-130 because the sponsor's primary claim of efficacy (75% slowing) was based on comparison to an external cohort that did not undergo the rigors of a clinical trial environment.

Regulatory Precedents: Zolgensma and Luxturna

To understand the FDA’s "threshold" for requiring a sham control, it is essential to compare the AMT-130 case with other approved gene therapies that utilized different control strategies. The agency's flexibility is not arbitrary; it follows a specific rubric of disease severity, predictability, and effect size.

Zolgensma (Onasemnogene Abeparvovec) and the "Self-Evident" Threshold

Zolgensma was approved for spinal muscular atrophy (SMA) Type 1 based on single-arm studies (START and STR1VE) using natural history controls. The FDA allowed this approach because the disease met the "threshold" where a sham control was no longer ethical or methodologically necessary:

Highly Predictable Natural History: Untreated SMA Type 1 is universally fatal or requires permanent ventilation by age two, and children never achieve the motor milestone of sitting without support.
Objective and Verification-Ready Endpoints: The primary endpoints—survival and the achievement of developmental motor milestones like sitting for 10 seconds—are objective and not susceptible to placebo effects or practice effects.
Massive and Temporal Effect Size: The treatment effect was massive and closely associated with the time of administration. In the START trial, all children were alive at the conclusion of the study, and many were walking—outcomes never seen in the history of the disease.

In the case of SMA, the "threshold" for an external control was met because the signal-to-noise ratio was so high that a concurrent control group would have provided little additional information while potentially condemning control infants to death. Huntington’s disease, however, is a slowly progressing adult-onset condition where the goal is marginal slowing of decline rather than a dramatic functional gain, placing it firmly below the "self-evident" threshold.

Luxturna (Voretigene Neparvovec) and the "Delayed Entry" Model

Luxturna, the first gene therapy for an inherited retinal disease (IRD) caused by RPE65 mutations, utilized an open-label, randomized, "delayed entry" control design.17 The control group did not receive a sham surgery; instead, they were followed for one year before crossing over to receive the treatment.

The FDA and EMA accepted the rejection of a sham sub-retinal surgery arm for Luxturna due to specific ethical and technical constraints:

Pediatric Ethics and Vulnerability: The inclusion of children as young as four years old made the risks of an unnecessary sub-retinal injection (including retinal tears, macular disorders, and infection) ethically unacceptable, as children cannot provide full legal consent.
General Anesthesia Risks: The procedure required general anesthesia, which posed a significant risk to pediatric participants without any clinical benefit.
Development of Novel Objective Endpoints: To compensate for the lack of a sham, the study utilized the Multi-Luminance Mobility Test (MLMT), a validated obstacle course that provided a more objective measure of functional vision than standard visual acuity.

Criterion	SMA (Zolgensma)	RPE65-IRD (Luxturna)	Huntington's (AMT-130)
Disease Predictability	Extremely High (Terminal)	High (Progressive Vision Loss)	Moderate (Variable Progression).
Primary Endpoint	Objective (Survival/Sitting)	Semi-Objective (MLMT Score)	Subjective (UHDRS Total).
Control Arm	External (Natural History)	Randomized Delayed Entry	Proposed Sham Surgery.
Patient Population	Infants (<2 years)	Children & Adults	Adults (Early Manifest).

Ethical Thresholds for Sham Surgery in Neurosurgery

The ethical justification for sham surgery is typically evaluated using a multi-step framework designed to protect participants while ensuring scientific validity. For a sham control to be considered ethical, it must meet a specific threshold of necessity and risk-minimization.

The Six-Step Framework for Ethical Sham Procedures

Valuable Question: The research must address a clinically relevant question that cannot be answered through less invasive means.
Methodological Necessity: The sham control must be essential to test the hypothesis and exclude bias. In the case of AMT-130, the FDA argues that the subjective nature of the UHDRS makes a sham methodologically necessary.
Risk Minimization: The risk of the sham procedure itself must be minimized.4 The FDA's proposal of "scalp nicks" instead of burr holes is an example of risk minimization to maintain the blind without the full risk of intracranial penetration.
Acceptable Risk Threshold: The risks of the sham must not exceed a predefined threshold of "acceptable research risk". For adults, this threshold is often higher than for children, provided there is informed consent.
Justification by Knowledge: The potential societal benefit of gaining definitive efficacy data must outweigh the risks to the sham participants. The FDA argues that approving a "failed" or ineffective product based on biased data would be a greater societal harm than the nicks in the scalps of a few dozen volunteers.
Informed Consent: The use of deception and the risks of the sham must be fully disclosed to and authorized by the participants.

The Concept of the "Altruistic Gift"

Ethics approval for a sham intervention often hinges on whether the risks are low enough to justify an "altruistic gift" by volunteer subjects who receive no personal benefit. In neurosurgery, this is complicated by the "therapeutic misconception," where patients may believe they are receiving a cutting-edge treatment even when they have been informed of the possibility of receiving a sham. Studies of Parkinson's patients in sham trials found that many subjects viewed their participation as a way to "contribute to science" even if they were randomized to the control arm, suggesting that informed consent can be achieved if the relationship between the investigator and patient is managed transparently.

However, the "tension between the highest standards of research and the highest standards of ethics" remains. Critics of sham surgery argue that if reasonable people disagree about the risks, the default should be a conservative approach.8 Proponents counter that without these controls, we risk adopting expensive and invasive procedures that are essentially "placebo surgeries," leading to a greater aggregate risk to the patient population.

The Biostatistical Rationale for FDA's Refusal

The FDA's decision to reject UniQure’s external control approach was not just about the type of control, but the specific data generated within the AMT-130 program. The agency highlighted an internal inconsistency that undermined the sponsor's credibility.

The "Smoking Gun" in the Randomized Cohort

UniQure's Phase I/II program included a small group of U.S. patients (n=10) who were randomized to either AMT-130 or a sham control. At the 12-month mark, the FDA pointed out that there was no treatment effect when the treated patients were compared to the actual sham-treated patients. However, when those same treated patients were compared to the external natural history cohort, they appeared to show a clinical benefit.

This creates a logical paradox: if the drug is working, why did it not beat the sham control in the same trial? The FDA’s conclusion was that the "benefit" seen against the external control was an artifact of the trial environment—likely the placebo effect or a difference in patient baseline characteristics that propensity matching failed to capture. The senior FDA official stated, "If both of those statements are true, why do we see a benefit at one year in the external controlled data, but not in the randomized data?". This inconsistency is the primary reason the agency labeled the company's data as "distorted" or "manipulated".

The Role of Heterogeneity and the "Will to Believe"

The FDA official outlined five specific conditions where the agency will only accept randomized, concurrent data :

Heterogeneous Conditions: Huntington’s disease is highly variable in its progression rates, making matched external controls inherently unreliable.
Strong Will to Believe: In a fatal disease with no current cure, both patients and investigators are highly susceptible to confirmation bias.
Invasive/Harmful Therapies: When a treatment involves intracranial surgery, the bar for proof of efficacy must be higher to justify the risks.
Difficult-to-Detect Effect Sizes: AMT-130 aims to slow a slow decline, rather than reverse it, making the signal very difficult to separate from background noise.
High Risk of Self-Deception: Without a blind, investigators may inadvertently score subjective motor tests (TMS) more favorably for patients they know received the gene therapy.

Technical and Operational Aspects of Sham Design in Neurosurgery

Maintaining the blind in a neurosurgical trial requires elaborate procedures to ensure that neither the patient nor the outcome assessor knows the treatment assignment.

Strategies for Deception and Masking

Standardized Pre-Op and Post-Op: All patients, regardless of assignment, undergo the same preparation, including hair removal (shaving), anesthesia induction, and post-operative dressings (e.g., head wraps).
Acoustic and Visual Mimicry: During sham orthopedic surgeries, surgeons have been known to splash water to mimic the sound of joint lavage and ask for surgical instruments to be passed to them. In brain surgery trials, the sound of a drill or the presence of the stereotactic frame can be used to simulate the procedure.
Blinded Outcome Assessors: The most critical component is ensuring that the clinicians who perform the UHDRS evaluations are completely separate from the surgical team. These assessors must remain blinded to the patient's treatment assignment throughout the follow-up period.
Cross-over Design: To address ethical concerns about withholding treatment, many sham trials allow control patients to receive the active treatment (cross-over) after a primary endpoint is reached (e.g., at 12 or 24 months).

Managing the Interventionalist’s Deception

A unique ethical challenge is that the "interventionalist" (the surgeon) cannot be blinded. They must perform a "fake" procedure, which requires them to maintain a clinical demeanor and identical concern for the patient's welfare regardless of whether the procedure is active or sham. Researchers have described this as a form of "temporary amnesia" or "good research acting" required to produce valid data.

Financial and Strategic Implications for UniQure

The regulatory roadblock has profound implications for UniQure’s corporate strategy and financial health. The company had planned to file for accelerated approval in early 2026 based on the Phase I/II data. The FDA’s demand for a Phase III sham-controlled trial effectively adds several years and hundreds of millions of dollars to the development timeline.

Capital Constraints and Revenue Targets

UniQure’s financial agreements are often tied to regulatory milestones. For instance, an amendment to their loan agreement with Hercules Capital in September 2025 provided for an additional $100 million in term loans, but $50 million of that was contingent on the "approval of a Biologics License Application". Furthermore, amortization of these loans could be delayed if the company met revenue targets related to AMT-130 prior to 2028—a timeline that is now highly unlikely. The delay also allows competitors, such as Takeda, Voyager Therapeutics, or Novartis, to potentially close the gap in the Huntington’s and ALS gene therapy markets.

The Debate Over "Regulatory Flexibility"

UniQure CEO Matt Kapusta argued that the "totality and durability" of their data warranted the application of the FDA’s "stated commitment to regulatory flexibility". This policy is intended to allow for alternative evidence in rare diseases where traditional RCTs are not feasible. However, the FDA's response indicates that "flexibility" does not mean a "lower bar" for evidence, especially when the existing data is contradictory. The agency's position is that if a sham control can be done safely, it must be done to ensure the product actually works.

Synthesis and Conclusion: The Future of Procedural Gene Therapy

The case of UniQure’s AMT-130 marks a significant shift in the regulatory landscape for gene therapies. While early successes like Zolgensma and Luxturna established a precedent for external or delayed-entry controls, those decisions were grounded in objective endpoints and extreme disease predictability. Huntington’s disease, with its complex neurobiology and subjective rating scales, presents a different set of challenges that, in the eyes of the FDA, necessitate the highest level of biostatistical rigor.

Nuanced Conclusions on the "Ethical Threshold"

The "threshold" beyond which a sham control is no longer ethical is not a fixed point, but a dynamic calculation based on three primary factors:

The Objectivity of the Signal: If a treatment produces a "binary" result (e.g., survival vs. death), the signal is clear enough to bypass a sham. If the result is a "percentage slowing" of a decline, the noise is too high to proceed without a blind.
The Vulnerability of the Population: Regulatory bodies are far more protective of children and those who cannot consent, as seen in the rejection of sham sub-retinal surgery for Luxturna.
The Internal Consistency of Preliminary Data: If a sponsor's own internal randomized data fails to show a benefit, any subsequent attempt to use external controls will be viewed with extreme skepticism by regulators.

Actionable Strategic Recommendations

For developers of invasive neurosurgical therapies, the UniQure controversy offers several critical lessons:

Prioritize Randomized Data Early: Even a small randomized sham cohort (as UniQure used) is more influential than a large external cohort. If the small randomized cohort fails, the external data will not save the program.
Design Minimal-Risk Shams: The "scalp nicks" model proposed by the FDA provides a blueprint for maintaining a blind in neurosurgery without the full risks of intracranial penetration.
Proactive Placebo Modeling: Use existing natural history databases (like Enroll-HD) to model practice and placebo effects before starting the trial, ensuring the study is powered to beat these background effects.
Transparency in Regulatory Communication: Attempting to characterize a regulatory rejection as a "pathway modification" in press releases can lead to blunt and damaging public refutations by the FDA, which can harm investor trust more than the trial delay itself.

The intersection of biostatistics and ethics in sham-controlled trials is ultimately about protecting the integrity of the medical literature. As the FDA official noted, the goal is to avoid "fooling yourself". For patients with Huntington’s disease, the only thing worse than no treatment is the false hope of a "failed" treatment that was approved based on distorted data. By insisting on a sham control, the FDA is arguably fulfilling its most fundamental ethical obligation: ensuring that the benefits of an invasive, permanent therapy are real, durable, and statistically sound.

References:

Recommended methodologies for clinical investigations of high-risk medical devices—Conclusions from the European Union CORE–MD Project - PMC, accessed March 24, 2026, https://pmc.ncbi.nlm.nih.gov/articles/PMC12624797/
Sham Procedure versus Usual Care as the Control in Clinical Trials of Devices: Which is Better? - ResearchGate, accessed March 24, 2026, https://www.researchgate.net/publication/5965535_Sham_Procedure_versus_Usual_Care_as_the_Control_in_Clinical_Trials_of_Devices_Which_is_Better
Sham procedures and the ethics of clinical trials - PMC, accessed March 24, 2026, https://pmc.ncbi.nlm.nih.gov/articles/PMC1079669/
Ethical framework for the use of sham procedures in clinical trials, accessed March 24, 2026, https://pubmed.ncbi.nlm.nih.gov/12626957/
Senior FDA official fires back at uniQure amid controversy, accessed March 24, 2026, https://www.fiercebiotech.com/biotech/senior-fda-official-refutes-uniqure-claims-amid-controversy-over-huntingtons-gene-therapy
Sham surgery can be ethical and safe | Journal of Neurology ..., accessed March 24, 2026, https://jnnp.bmj.com/content/74/2/202
uniQure and FDA Clash Over Regulatory Decision for Huntington's Disease Therapy: Report | PharmExec, accessed March 24, 2026, https://www.pharmexec.com/view/uniqure-fda-clash-regulatory-decision-huntington-disease-therapy-report
Sham Surgery and Reasonable Risks, accessed March 24, 2026, https://www.cmu.edu/dietrich/philosophy/docs/london/London--ShamSurgeryReasonableRisk.pdf
uniQure AMT-130 Updates | Huntington Society of Canada, accessed March 24, 2026, https://www.huntingtonsociety.ca/uniqure-amt-130-update/
Modeling Disease Progression and Placebo Response in ..., accessed March 24, 2026, https://pmc.ncbi.nlm.nih.gov/articles/PMC12057022/
uniQure Provides Regulatory Update on AMT-130 for Huntington's ..., accessed March 24, 2026, https://uniqure.gcs-web.com/news-releases/news-release-details/uniqure-provides-regulatory-update-amt-130-huntingtons-disease-2
Placebo effect characteristics observed in a single, international, longitudinal study in Huntington's disease - PubMed, accessed March 24, 2026, https://pubmed.ncbi.nlm.nih.gov/22162184/
FDA CBER OTAT Town Hall: Clinical Development of Gene Therapy ..., accessed March 24, 2026, https://www.fda.gov/media/167612/download

From "Two-Trial Dogma" to the Single Pivotal Standard: The Evolution of FDA Evidence Requirements

2026-02-22T22:00:00.003-05:00

The "two-trial" rule was born from the 1962 Kefauver-Harris Amendment to the Federal Food, Drug, and Cosmetic Act, which mandated that manufacturers prove a drug was not just safe, but also effective. This established the "substantial evidence" standard, which the FDA historically interpreted as requiring at least two adequate and well-controlled clinical trials. This "two-trial dogma" served as a statistical insurance policy: in a world where biologic understanding was more limited, requiring a developer to be "lucky twice" reduced the probability of a false-positive result from 250 in 10,000 to just 6 in 10,000.

However, as of 2026, the regulatory landscape has reached a historic turning point. Here is how the "substantial evidence" requirement evolved from a rigid duplication rule into a flexible, precision-based standard.

1. The 1998 Foundation: Establishing Statutory Flexibility

The first major shift toward modern flexibility arrived with the 1998 Guidance: Providing Clinical Evidence of Effectiveness for Human Drug and Biological Products. Following the FDA Modernization Act (FDAMA) of 1997, the agency gained formal statutory authority to grant marketing authorization based on a single adequate and well-controlled study combined with "confirmatory evidence".

While this allowed for disease-by-disease flexibility—particularly in oncology and rare diseases, where single trials began to support the majority of approvals—manufacturers remained confused about exactly when a single trial would be accepted. For most "Main Street" drugs or drugs for common diseases, the two-trial expectation remained the functional default.

2. The 2023 Expansion: Defining "Confirmatory Evidence"

In September 2023, the FDA released updated draft guidance "Demonstrating Substantial Evidence of Effectiveness With One Adequate and Well-Controlled Clinical Investigation and Confirmatory Evidence" to clarify what constitutes "confirmatory evidence" when only one pivotal trial is conducted. This guidance acknowledged that modern drug development relies on both statistical and biologic inferences. Under this framework, a single trial could be bolstered by:

Clinical Evidence from a Related Indication
Mechanistic or Pharmacodynamic Evidence
Evidence from Relevant Animal Model
Evidence from Other Members of the Same Pharmacological Class
Natural History Evidence
Real-World Data/Evidence (RWD/RWE)
Evidence from Expanded Access Use of an Investigational Drug

3. The 2026 Paradigm Shift: The New Default

Last week, in February 2026, the FDA officially ended the "two-trial dogma." In a landmark article "One Pivotal Trial, the New Default Option for FDA Approval - Ending the Two-Trial Dogma" in the New England Journal of Medicine, FDA officials announced that a one-trial requirement is now the agency's new default standard for drug approval. We are waiting for the formal FDA guidance to provide the details about this paradigm shift.

Why the shift?

Precision and Biology: Modern drug discovery is increasingly precise. The FDA now considers biochemical changes and biomarkers that tell a "complete biologic story," making overreliance on a second trial unnecessary when the "mechanistic science is sound".
Economic Relief: A single pivotal study can cost between $30 million and $150 million and take years to complete. By moving to a one-trial default, the FDA aims to lower capital costs and remove a primary justification for high drug prices.
Quality Over Quantity: Officials argue that two trials can provide "false assurance" if their designs are deficient (e.g., substandard control arms or dubious endpoints). The agency will now focus its energy on ensuring the single required trial is "robust and sound".

The Guardrails: When Two Trials Are Still Required

The FDA will not abandon the two-trial standard entirely. Additional studies may still be required if:

An intervention has a nebulous or nonspecific mechanism of action.
The trial affects only a labile or short-term surrogate outcome.
The primary trial has underlying limitations or deficiencies.

Conclusion

We have moved from an era of Replication (the 1962-1998 standard) to Precision (the 2026 default). By formally changing the "default option," the FDA expects to spur a surge in biomedical innovation and speed life-saving drugs to the patients.

Additional Reading:

raps.org Experts react to FDA’s shift to single pivotal trials for most drugs
fda.gov (2016) Promoting Safety and Effective Drugs for 100 years
Sasinowski et al (2012) Quantum of Effectiveness EVidence in FDA's Approval of Orphan Drugs
Sasinowski et al (2015) Quantum of Effectiveness Evidence in FDA's Approval of Orphan Drugs: Update, July 2010 to June 2014

The Statistical Magic Trick: How Trials Share Results While Staying Blind

2026-02-16T23:00:00.003-05:00

In the world of clinical research, "breaking the blind" is typically a cardinal sin. Yet, high-stakes Phase 3 trials like ORIGIN 3 (atacicept), PROTECT (sparsentan), and ATTRIBUTE-CM (acoramidis) have all successfully navigated the complex path of publishing interim results in the New England Journal of Medicine while keeping their long-term studies scientifically intact.

Lafayette et al (2026) A Phase 3 Trial of Atacicept in Patients with IgA Nephropathy
Perkovic et al (2026) Sibeprenlimab in IgA Nephropathy — Interim Analysis of a Phase 3 Trial
Gillmore, et al (2024)Efficacy and Safety of Acoramidis in Transthyretin Amyloid Cardiomyopathy and its attached protocol and statistical analysis plan "

How do they pull off this statistical magic trick? It comes down to a rigorous architectural separation of data and people including statistical analysis team. Here is a look at the techniques used to maintain the "blind" during interim disclosures.

1. The "Firewall" Strategy: Independent Reporting Teams

The most critical technique used across all three studies is the creation of a "Firewall."

While a trial is ongoing, the people running the study (Sponsor clinical teams, investigators at hospitals, and the patients) must remain blinded. To perform an interim analysis, the Sponsor appoints an Independent Reporting Team (IRT) or an Independent Statistical Center.

How it works: This group has no contact with the study sites. They receive the raw "unblinded" data, perform the calculations for the primary endpoint (like the 36-week proteinuria reduction in PROTECT or ORIGIN 3), and prepare the manuscript for publication.
The Result: The people actually treating the patients remain in the dark, ensuring their medical decisions aren't influenced by knowing who is on the "winning" drug.

2. Safeguarding Against "Functional Unblinding"

In some trials, the drug’s effect is so obvious it could accidentally reveal the treatment.

In ORIGIN 3: Atacicept significantly lowers serum IgA and IgG levels. If a doctor saw these lab results, they would immediately know the patient was on the active drug. To prevent this, these specific lab values are suppressed. The results are sent to the Independent Reporting Team but are hidden from the investigators and the Sponsor’s site monitors.
In PROTECT: This study compared two active drugs (sparsentan vs. irbesartan). To ensure the difference in pill appearance didn't tip anyone off, they used a Double-Dummy design. Every patient took two sets of pills—one active and one placebo—so the physical routine remained identical for everyone.

3. Aggregate vs. Individual Disclosure

A common misconception is that "publishing the results" means everyone knows who got what. In reality, the NEJM publications for these trials only disclose aggregate data (group averages), not the individual patient level data.

In ATTRIBUTE-CM: When the Part A results (12-month 6-minute walk distance) were disclosed, the public learned how the group performed. However, the individual treatment assignments for each patient remained locked in the secure database.
The Benefit: Even if an investigator reads the NEJM article and sees that acoramidis is effective, they still do not know if the specific patient sitting in their office is receiving acoramidis or the placebo.

4. Prespecified Alpha Spending and "The Gatekeeper"

To maintain the statistical integrity of the final results (like the 104-week kidney function in ORIGIN 3 or the 30-month clinical outcomes in ATTRIBUTE-CM), the Statistical Analysis Plan (SAP) dictates exactly how much "statistical credit" is used during the interim look.

The Independent Data Monitoring Committee (iDMC) acts as the gatekeeper. They review the unblinded data behind closed doors and only allow the trial to proceed if the interim disclosure doesn't compromise the "power" of the final analysis.

Why go through all this trouble?

The goal is Accelerated Approval. By using these techniques, sponsors can show the FDA (and the medical community) that a drug works on a "surrogate marker" (like proteinuria) at an interim stage. This allows life-saving drugs to reach patients years earlier, while the "blinded" portion of the trial continues to gather the long-term data needed for full, traditional approval.

By combining physical dummies, suppressed lab data, and strict "firewalls" between statistical teams, researchers prove that you can indeed share the news of a trial's success without ruining the science that supports it.

Some Extra Words on ATTRIBUTE-CM Study

ATTRIBUTE-CM study is a phase 3, double-blind trial, 632 patients with transthyretin amyloid cardiomyopathy were randomly assigned in a 2:1 ratio to receive acoramidis hydrochloride at a dose of 800 mg twice daily or matching placebo for 30 months. The study contained two parts: Part A with primary endpoint of change from baseline to Month 12 of treatment in distance walked during the 6MWT, Part B with primary endpoint of a hierarchical combination of All-Cause mortality and CV-related hospitalization over a 30month period.

There were two readouts for the study and the Part A readouts were based on an interim analyses by the independent DMC.

In December 2021, BridgeBio Pharma experienced a major setback when its Phase 3 ATTRibute-CM trial for acoramidis (a treatment for transthyretin amyloid cardiomyopathy - ATTR-CM) failed to meet its primary endpoint of improving the 6-minute walk distance (6MWD) at Month 12 (Part A primary efficacy endpoint). In the initial 12-month data, patients taking acoramidis did not show a statistically significant improvement in their 6MWD compared to those on a placebo.

Despite the failure of the 6MWD endpoint at 12 months, the study continued because the independent data monitoring committee noted encouraging trends in other areas. By July 2023, BridgeBio reported positive top-line results from the full study (Month 30), where acoramidis demonstrated a highly statistically significant improvement in a hierarchical analysis that included mortality, hospitalization, and 6MWD (Part B primary efficacy endpoint).

Following the successful long-term data (Part B), which showed a 25% reduction in all-cause mortality and 50% reduction in cardiovascular hospitalization frequency (known as Attruby), the drug was approved by the FDA, with 3,751 prescriptions filled as of August 2025.

The study included an embedded Part A readouts that required the unblinding for interim analysis. The sponsor specified the following for maintaining the blinding for the overall study while the Part A results were analyzed and disclosed.

Maximal Tolerated Dose (MTD) to Recommended Phase 2 Dose (RP2D) - a shift in early oncology trial designs

2026-01-18T13:00:00.002-05:00

As the field of oncology moves from systemic cytotoxic chemotherapies to targeted agents and immunotherapies, the paradigm for dose selection is undergoing a historic shift. For decades, the Maximum Tolerated Dose (MTD) was the "gold standard" for early-phase trials, but today’s clinical trialists and statisticians are increasingly prioritizing the Recommended Phase 2 Dose (RP2D) as a more robust and patient-centric metric.

This evolution is spearheaded by the FDA’s Project Optimus, which emphasizes "dose optimization" rather than simply finding the highest dose a patient can survive.

From "More is Better" to "The Optimal Balance"

The traditional MTD-centric approach was built on the assumption that a drug's efficacy increases linearly with its toxicity—a rule that often held true for classical chemotherapy. However, for modern targeted therapies, the Optimal Biologic Dose (OBD)—the dose that achieves maximum target saturation—often occurs well below the MTD.

Feature	Maximum Tolerated Dose (MTD)	Recommended Phase 2 Dose (RP2D)
Focus	Toxicity-driven; finding the safety ceiling.	Value-driven; finding the therapeutic "sweet spot".
Observation	Short-term (Cycle 1) Dose-Limiting Toxicities (DLTs).	Long-term tolerability, PK/PD, and cumulative safety.
Assumption	Efficacy increases with dose ("More is Better").	Efficacy may plateau while toxicity continues to rise.
Clinical Utility	A safety guardrail to prevent overdosing.	A strategic decision for registrational success.

Why RP2D is Preferred over MTD

For the modern statistician, the RP2D represents a "totality of evidence" that the MTD simply cannot provide:

Sustainability vs. Intensity: MTD focuses on what a patient can tolerate for 21 days. In contrast, RP2D considers the long-term tolerability necessary for chronic treatment, preventing premature discontinuations that can derail a trial's efficacy results.
The Sotorasib Lesson: FDA reviews, such as those for sotorasib, have highlighted the "dosing conundrum" where initial MTD-based doses led to excessive toxicity, eventually requiring post-market studies to find a more optimal, lower dose.
Target Saturation: Modern agents often reach a Pharmacokinetic (PK) plateau where increasing the dose adds no therapeutic benefit but significantly increases the rate of low-grade, chronic toxicities.
Dose-Response Nuance: As discussed in previous explorations of Determining the Dose in Clinical Trials, while the MTD is a safety limit identified through escalation, the RP2D is a comprehensive recommendation for further evaluation that aims to expose as few patients as possible to intolerable doses.

The Statistical Shift: Beyond 3+3

To find a true RP2D, statisticians are moving away from the rigid "3+3" rule-based designs to more flexible, model-informed approaches. These include:

Bayesian Optimal Interval (BOIN) designs that allow for a more nuanced exploration of the therapeutic window.
Randomized Dose-Ranging Studies: Encouraged by Project Optimus, these trials evaluate multiple doses early to compare safety and efficacy side-by-side.
Dose Expansion Cohorts: Used to refine the RP2D by gathering deeper data on preliminary efficacy and late-onset toxicities in specific patient subgroups.

Conclusion

The shift from MTD to RP2D is more than a regulatory requirement; it is a clinical necessity. By identifying an optimized RP2D early, sponsors can avoid the "safety pitfalls" of MTD, improve patient quality of life, and build a stronger evidence chain for final approval. In the era of precision medicine, finding the right dose for the right patient is just as important as finding the right drug.

Excessive number of clinical trial protocol amendments due to complex trial design

2026-01-04T22:08:00.001-05:00

In a previous blog post "Protocol amendment in clinical trials", I discussed the impact of protocol amendments on the clinical trial performance and cost and the reasons for driving the protocol amendments. Protocol amendments are unavoidable, but we can try to think about the study design and execution proactively to minimize the number of protocol amendments. Sometimes, the excessive number of protocol amendment are driven by Complex Innovative Trial Design or CID in short (for example, adaptive design, basket/umbrella/platform trial design, expansion cohort design, Bayesian design...).

We noticed an extreme case of a clinical trial with the study protocol amended 50 times. This refers to Study P001 (also known as KEYNOTE-001, NCT01295827) by Merck, which was a large, multi-cohort Phase 1 trial with numerous expansion cohorts that supported the initial accelerated approval of pembrolizumab. in Statistical Review and Evaluation, BLA 125514, FDA Center for Drug Evaluation and Research, August 2014, The FDA reviewer noted this high number of amendments while discussing the complexity of the trial design. KEYNOTE-001 was a massive "seamless" adaptive trial that evolved from a traditional Phase 1 dose-escalation study into a large study with multiple expansion cohorts (Part A, A1, A2, B, C, D, etc.) covering different tumor types (Melanoma, NSCLC) and dosing regimens. The "50 times" figure likely includes all global and country-specific amendments up to the time of the BLA submission in February 2014.

The high number of protocol amendments for KEYNOTE-001 was a direct result of its innovative, "seamless" adaptive study design. Initially launched as a standard Phase 1 dose-escalation trial, the study evolved into a massive, multi-cohort trial that eventually enrolled 1,235 patients.

The 50 amendments occurred primarily due to the following reasons:

Addition of Expansion Cohorts: As early data showed promising results, the protocol was repeatedly amended to add new expansion cohorts for specific tumor types, most notably melanoma and non-small cell lung cancer (NSCLC).
Sample Size Increases: Striking patient responses led investigators to increase sample sizes within existing cohorts to better evaluate efficacy endpoints like overall response rate (ORR).
Adaptive Dosing Changes: The protocol was amended to change dosing regimens based on emerging safety and efficacy data. For example, Amendment 7 changed dosing from every two weeks (Q2W) to every three weeks (Q3W), and Amendment 10 shifted all participants to a fixed dose of 200 mg.
Biomarker Integration: Amendments were used to add co-primary endpoints related to PD-L1 expression after researchers observed its correlation with drug efficacy. This included the validation of a companion diagnostic assay.
Regulatory Speed: This "seamless" approach allowed Merck to skip traditional Phase 2 and 3 steps for certain indications, leading to the first-ever FDA approval of an anti-PD-1 therapy.

While efficient, the FDA's statistical reviewers noted that such frequent changes (averaging more than one amendment per month during the most active phases) created significant operational and analytical complexity for the trial. The main challenges in analyzing the KEYNOTE-001 trial data, as noted in the FDA's statistical and medical reviews, stemmed from the extreme complexity of a "seamless" design that was modified more than 50 times.

The primary analytical hurdles included:

Statistical Integrity and Type I Error Risk: The frequent addition of new cohorts and subgroups—often based on emerging data—increased the number of statistical comparisons. This raised concerns about "multiplicity," where the probability of finding a significant result by chance (Type I error) increases with every new hypothesis tested.
Operational and Data Management Complexity: Maintaining data quality was difficult when different sites were often operating under different versions of the protocol simultaneously. The FDA noted that this led to potential adherence issues and made it difficult to isolate single cohorts for clean, standalone submissions.
Shifting Dosing and Regimens: The trial transitioned from weight-based dosing (2 mg/kg or 10 mg/kg) to a fixed dose (200 mg) and changed the frequency of administration (every 2 weeks to every 3 weeks) mid-study. This required complex "pooled analyses" to prove that efficacy and safety were consistent across these varying schedules.
Biomarker Selection and Validation: The protocol was amended to include a PD-L1 companion diagnostic while the study was already underway. This created a challenge in defining "training" vs. "validation" sets within the same trial population to establish the diagnostic's cutoff levels without introducing bias.
Lack of a Control Arm: Because the trial was essentially a massive Phase 1 expansion, it lacked a randomized control arm for several indications. This forced reviewers to rely on cross-trial comparisons and historical data, which are inherently more prone to bias than randomized controlled trials (RCTs).
Patient Selection Bias: The "adaptive" nature allowed for rapid accrual in specific successful cohorts, which, while beneficial for speed, made it difficult to ensure the final patient population was representative of the broader real-world population.

Although the excessive number of protocol amendments, the results from the KEYNOTE-001 resulted in the FDA approval of pembrolizumab in the treatment of multiple tumor types. KEYNOTE-001 study was also the basis for the NEJM article "Seamless Oncology-Drug Development" by Prowell, Theoret, and Pazdur.

One-way versus two-way tipping point analysis for robustness assessment of the missing data

2026-01-01T11:00:00.001-05:00

Tipping point analysis (TPA) is a key sensitivity analysis mandated by regulatory agencies like the FDA to assess the robustness of clinical trial results to untestable assumptions about missing data. Specifically, it explores how much the assumption about the missing not at random (MNAR) mechanism would have to change to overturn the study's primary conclusion (e.g., a statistically significant treatment effect becoming non-significant). See a previous blog post "Tipping point analysis - multiple imputation for stress test under missing not at random (MNAR)"

One-Way Tipping Point Analysis for Robustness Assessment

A one-way tipping point analysis is a sensitivity method used to evaluate the robustness of a study’s primary findings by systematically altering the missing data assumption for only one treatment group at a time—most commonly the active treatment arm. While the missing outcomes in the control group are typically handled under a standard Missing at Random (MAR) or Jump to Reference assumption, the missing outcomes in the active arm are subjected to a varying "shift parameter" (δ). This parameter progressively penalizes the imputed values (e.g., making them increasingly worse) until the statistically significant treatment effect disappears, or "tips." By identifying this specific value, researchers can present a clear, one-dimensional threshold to clinical experts and regulators, who then judge whether such a drastic deviation from the observed data is clinically plausible or an unlikely extreme.

Two-Way Tipping Point Analysis for Robustness Assessment

A two-way TPA is an advanced method to assess robustness by independently varying the missing data assumptions for both treatment groups (e.g., the active treatment arm and the control/reference arm).

Missing Data Assumptions (MAR vs. MNAR)

The two-way TPA is used to assess the robustness of the primary analysis, which is typically conducted under the assumption of Missing at Random (MAR).

Missing at Random (MAR): Assumes that the probability of data being missing depends only on the observed data (e.g., a patient with a worse baseline condition is more likely to drop out, and we have observed the baseline data).
Missing Not at Random (MNAR): Assumes that the probability of data being missing depends on the unobserved missing outcome data itself (e.g., a patient drops out because their unobserved outcome has worsened more than what is predicted by their observed data).

Robustness Assessment

The two-way TPA evaluates robustness to plausible MNAR scenarios. This is done by imputing the missing outcomes (often starting with an MAR method like Multiple Imputation) and then applying a systematic, independent "shift parameter" (or δ) to the imputed values in each arm.

Process: The shift parameters (δ $\delta_{Active}$ and δ $\delta_{Control}$ ) are varied systematically across a two-dimensional grid, typically in a direction that reduces the observed treatment effect.
Tipping Point: The δ $\delta_{Active}$ and δ $\delta_{Control}$ values at which the primary conclusion (e.g., statistical significance) is "tipped" or overturned define the tipping point.
Robustness: The larger and/or more clinically implausible the combination of shift parameters required to overturn the conclusion, the more robust the original result is considered to be under different MNAR assumptions.

Two-Way Tipping Point Result Tables

The results of a two-way TPA are typically presented as a grid or heat map table where:

One axis represents the shift parameter applied to the missing outcomes in the Active Treatment arm (δ $\delta_{Active}$ ).
The other axis represents the shift parameter applied to the missing outcomes in the Control/Reference arm (δ $\delta_{Control}$ ).
The cells of the table contain the resulting p-value or estimated treatment difference for that specific combination of assumptions.

The goal is to find the boundary of the grid where the result crosses the significance threshold (e.g., $p \ge 0.05$ or the lower bound of the confidence interval crosses the null value).

Comparison: One-Way vs. Two-Way Tipping Point Analysis

The choice between one-way and two-way TPA is a trade-off between simplicity and comprehensiveness.

Feature	One-Way Tipping Point Analysis	Two-Way Tipping Point Analysis
Missingness Assumption	The shift parameter (δ) is only applied to one arm, usually the active treatment group, while the missing data in the control arm are imputed based on the MAR assumption (e.g., Jump to Reference).	Independent shift parameters (δ $\delta_{Active}$ and δ $\delta_{Control}$ ) are applied to both arms simultaneously.
Sensitivity Explored	Explores MNAR scenarios where dropouts in one arm have systematically worse/better outcomes than assumed by MAR, relative to the other arm's MAR assumption.	Explores a two-dimensional space of MNAR scenarios, allowing dropouts in both arms to vary independently.
Complexity	Simpler to calculate and interpret (one dimension).	More computationally intensive and complex to interpret (two-dimensional grid).
Plausibility	Often viewed as less comprehensive, as it does not model the possibility of simultaneous, independent MNAR mechanisms in both arms.	Considered more comprehensive as it allows for a wider range of clinically plausible and implausible MNAR scenarios.
Result Presentation	A line plot or simple table with a single 'tipping point' value.	A grid/matrix table or heat map showing the boundary of non-significance.

In essence, the two-way TPA is generally preferred by regulatory agencies for its superior ability to assess robustness because it explores a more realistic and exhaustive range of asymmetric MNAR mechanisms.

FDA guidance "Sponsor Responsibilities - Safety Reporting Requirements and Safety Assessment for IND and Bioavailability/Bioequivalence Studies"

2025-12-29T22:30:00.001-05:00

Earlier this month, FDA issued its guidance "Sponsor Responsibilities - Safety Reporting Requirements and Safety Assessment for IND and Bioavailability/Bioequivalence Studies". As an clinical trialist, the updated FDA guidance (or the 2025 guidance) represents a major step forward, primarily by refining the focus on safety assessment and introducing key operational elements.

The 2025 guidance is not a complete rewrite of the 2012 version ("Safety Reporting Requirements for INDs and BA/BE Studies"), but rather a merger of the 2012 guidance content with the principles from the 2015 draft guidance on safety assessment.

Here is a comparison highlighting the key new elements the sponsor must now consider:

Key New Elements in the 2025 Guidance

The most significant change is a shift from focusing solely on individual case safety reports (ICSRs) to a greater emphasis on proactive, systematic safety assessment and the analysis of aggregate data.

New Concept	Description and Implication for Trialists	Relevant Section in New Guidance
Focus on Sponsor Responsibilities Only	The new guidance is strictly limited to Sponsor Responsibilities for safety reporting. All recommendations for Investigator Responsibilities found in the 2012 guidance have been moved to a separate document, reflecting a clear split in regulatory oversight.	Section I, II (Preamble)
Aggregate Data Assessment	This is the central update. The guidance expands significantly on the requirement to perform regular, proactive aggregate analyses of all accumulating safety data. The goal is to identify new or increased risks that would trigger expedited reporting, rather than relying only on individual case reports.	Section III (Definitions) and Section IV (Aggregate Analyses)
Mandatory Safety Surveillance Plan (SSP)	The guidance introduces the term Safety Surveillance Plan (SSP) as a systematic and organized approach to safety monitoring. The plan should include: 1) Clearly defined roles and responsibilities; 2) A plan for the regular review and evaluation of Serious Adverse Events (SAEs); and 3) The process for performing aggregate safety reviews.	Section IV.C (Safety Surveillance Plan)
Sole Sponsor Causality Determination	The guidance emphasizes that the final responsibility for determining whether an event meets the criteria for expedited reporting (i.e., a "Suspected Adverse Reaction," or SUSAR) lies solely with the sponsor. While the sponsor should consider the investigator's opinion, the sponsor is imputed with the ultimate responsibility for the causality judgment for regulatory submission purposes.	Section III.B (Suspected Adverse Reaction)
Flexibility in Safety Review	The new guidance offers greater flexibility by allowing sponsors to choose which individual, group, or entity (e.g., Safety Monitoring Committee, Data Monitoring Committee) is responsible for reviewing, analyzing, and making decisions regarding IND safety reporting.	Section IV.C.1 (Features and Composition of the Entity)

This shift aims to reduce the "noise" of over-reporting uninformative individual adverse events, which was a concern under the old paradigm. Instead, the focus is placed on the sponsor's expert medical review and comprehensive analysis of the overall safety data package.

Here is a side-by-side comparison table summarizing the main discussion points and key changes between the 2012 and 2025 FDA guidance documents on safety reporting.

Safety Reporting Guidance: 2012 vs. 2025 Comparison

Discussion Point	2012 Final Guidance: Safety Reporting Requirements for INDs and BA/BE Studies	2025 Final Guidance: Sponsor Responsibilities — Safety Reporting Requirements and Safety Assessment for IND and BA/BE Studies
Primary Scope and Focus	Focused on procedural requirements for expedited reporting of individual Serious Adverse Events (SAEs).	Mandatory emphasis on safety assessment and aggregate data analysis to identify new, significant risks. Merges content with principles from the 2015 draft guidance on safety assessment.
Division of Responsibilities	Contained recommendations for both Sponsor and Investigator safety reporting responsibilities.	Exclusively focuses on Sponsor responsibilities. Investigator reporting recommendations are placed in a separate, concurrently issued guidance document.
Safety Surveillance/Planning	Implicit in the sponsor's duties, but lacked a formalized planning requirement.	Introduces the new term "Safety Surveillance Plan (SSP)" to describe a required systematic and organized approach.
Plan Components (SSP)	Did not specify formal plan components.	Requires the plan to include clearly defined roles and responsibilities, a process for regular review of SAEs, and a process for aggregate safety reviews.
Requirement for Review	Focused primarily on individual case review to determine if the reporting criteria (Serious, Unexpected, Suspected Adverse Reaction - SUSAR) were met.	Explicitly requires sponsors to review and evaluate all accumulating safety data at regular intervals (aggregate review) to update the overall safety profile.
Decision-Making Body	Lacked specific recommendations for the structure of the internal safety review process.	Offers greater flexibility by allowing the sponsor to choose the individual, group, or entity (e.g., Safety Assessment Committee) responsible for safety reporting and decision-making.
Source of Safety Data	Focused mainly on reports from the clinical trial itself.	Emphasizes that sponsors must review information from any source (e.g., animal studies, scientific literature, foreign reports, and commercial experience) to identify new significant risks to trial participants.
Expedited Reporting Rationale	The concern was the overreporting of uninformative individual Adverse Events (AEs), which hindered the IRB's ability to focus on true risks.	Seeks to reduce overreporting by clarifying that the decision for a 7- or 15-day expedited report must be based on the sponsor's professional judgment of causality (i.e., a reasonable possibility).

Summary of the Shift

The 2025 guidance strongly emphasizes a shift in the regulatory burden from volume-based individual reporting (the 2012 paradigm) to quality-based, comprehensive safety analysis by the sponsor. The overall goal is to enhance patient protection by focusing the FDA, IRBs, and investigators on truly meaningful safety signals derived from cumulative data, rather than individual case reports.

Handling "Median Not Reached": A Guide to Analyzing and Presenting Low Event Rate Survival Data

2025-12-01T07:00:00.001-05:00

In the era of highly effective therapies for may diseases, clinical researchers are increasingly encountering a "good" problem in the time to event analyses: the Kaplan-Meier survival curves are flattening out well above the 50% mark. While this represents a triumph for patient outcomes, it creates a headache for statistical reporting. When the event rate is low (below 50%), the Median Time to Event (e.g., Median Overall Survival) and its 95% Confidence Interval (CI) cannot be estimated (often reported as "NE" (not estimable), "NR" (not reached), or "NC" (not calculable)).

So, how do we robustly describe the efficacy of a treatment when the standard metric fails? This post outlines the best-practice alternatives for summarizing, analyzing, and visualizing survival data in low event settings.

1. The Limitation of the Median

The median survival time is simply the time point at which the survival probability drops to 0.50. If the Kaplan-Meier curve plateaus at 70% or 80% because fewer than half the patients experienced the event, the median is mathematically undefined. Reporting it merely as "Not Reached" (NR) is accurate but clinically uninformative—it tells us what the survival is not, but not what it is.

To provide a complete picture, we must pivot to alternative metrics that describe different parts of the survival distribution.

2. Primary Summary Measures

A. Landmark Survival Probabilities

When we cannot answer "When will half the patients die?", we should ask, "What proportion of patients are event-free at time t ?"

Landmark analysis reports the Kaplan-Meier survival probability (with 95% CIs) at clinically relevant, fixed timepoints (e.g., 24 weeks, 12 months, 24 months, 5 years).

Best Practice: Pre-specify these timepoints in the Statistical Analysis Plan (SAP) to avoid data dredging.
Example Reporting: "Event free rate was 93% at week 24 in the treatment group", "The 3-year recurrence-free survival rate was 88.4% (95% CI: 85.1–91.0) in the treatment arm compared to 82.1% (95% CI: 78.4–85.2) in the placebo arm."

B. Lower-Percentile Survival Times (10th and 25th)

Just because the 50th percentile (median) is missing doesn't mean all percentiles are.

25th Percentile: The time at which 25% of patients have experienced the event (or survival drops to 75%).
10th Percentile: The time at which 10% of patients have experienced the event (or survival drops to 90%).

These metrics characterize the "early failures" or the worst-performing subset of the cohort. They are particularly useful for showing that a treatment delays early progression even if the long-term survival is high.

Metric	Treatment Group	Control Group
Median (50th)	NR (95% CI: NR, NR)	NR (95% CI: 36.7, NR)
25th Percentile	18.4 months (14.2, 22.1)	12.1 months (9.8, 14.5)
10th Percentile	5.4 months (4.1, 6.8)	3.2 months (2.8, 3.9)

Note: In the table above, while the median is NR for both, the 25th percentile clearly demonstrates a 6-month delay in progression for the treatment group.

3. Robust Analytical Alternatives

A. The "Reverse Kaplan-Meier" Method for Follow-Up

In low event trials, it is critical to prove that the "NR" result is due to drug efficacy, not just because patients left the study early. The Reverse Kaplan-Meier method is the gold standard for calculating median follow-up.

How it works: You reverse the censoring indicator (Event = Censored; Censored = Event) and run a standard Kaplan-Meier analysis. The resulting median is the median potential follow-up time.
Why use it: Unlike the "median time on study," it is not biased by early deaths or events, providing a true measure of how long the trial centers monitored the patients.

B. Restricted Mean Survival Time (RMST)

RMST is rapidly becoming the preferred alternative to the Hazard Ratio (HR) in low event trials, especially when the Proportional Hazards assumption is violated (e.g., crossing curves).

Definition: RMST is the "area under the survival curve" up to a specific time point ( $\tau$ ). It represents the average survival time a patient lives during that window.
Reporting: You can report the Difference in RMST (Treatment minus Control) or the Ratio.
Interpretation: "Over the 5-year follow-up period, patients on the new therapy lived, on average, 4.2 months longer than those on the control (RMST difference = 4.2 months, p=0.003)."

4. Visualization Best Practices

A. The Kaplan-Meier Plot: Handling the Y-Axis

In trials with very high survival (e.g., >90%), the survival curves may be squeezed into the top 10% of the graph, making it hard to see separation.

Line Break (Axis Break): It is acceptable to "break" the y-axis to focus on the relevant range (e.g., from 80% to 100%), provided this is clearly marked.
Inverted Plot (Failure Plot): Alternatively, plot the Cumulative Incidence of Events (1 - Survival) on a y-axis ranging from 0% to 20%. This often visualizes the difference in event rates more clearly than a survival curve stuck at the top of the chart.

B. The "Number at Risk" Table

Always include a table below the x-axis aligned with the tick marks. In low event trials, this table reveals whether the "flat tail" of the curve is based on hundreds of patients or just a few who haven't been followed long enough.

5. Optional Exploratory Methods

If pre-specified in the protocol, Parametric Modeling can be used to estimate the median survival even if it hasn't been reached observed data.

Weibull Distribution: By fitting a Weibull model to the observed data, you can extrapolate the curve to predict when the median would be reached, assuming the risk profile remains constant.
Caution: This is a prediction, not an observation. It should be labeled clearly as "Estimated Median (Parametric)" and treated as exploratory evidence.

Summary Checklist for Reporting Low Event Data

State clearly that the median is NE/NR.
Report Landmark Rates (e.g., 3-year survival) with CIs.
Report Lower Percentiles (25th, 10th) to show early separation.
Use RMST to quantify the average time gained.
Calculate Follow-up using the Reverse Kaplan-Meier method.
Adjust Plots (zoom/break y-axis) to make differences visible, but keep the full context clear.

Note: AI-assisted writing for this blog article.

When external controls collide with the FDA: two recent cases and what they teach us

2025-11-09T22:00:00.001-05:00

When trials don’t use a traditional control — What the FDA guidance says

In the usual drug-development pathway, regulators expect a trial where patients are randomly assigned either to receive the investigational drug or a comparator (placebo or active treatment). This randomized-controlled-trial (RCT) design is the “gold standard” for showing that a drug works (i.e., that the benefit is due to the drug not to other factors). For NDA/BLA approvals, FDA issued two guidance documents:

However, there are settings—rare diseases, rapidly progressive conditions, or severe unmet need—where recruiting a traditional randomized control arm is difficult or ethically questionable. In those settings, sponsors may propose to compare patients treated under an investigational protocol against a group of patients drawn from an outside dataset (e.g., prior trials, patient registries, real-world data) who did not receive the investigational drug. Such a comparison group is often called an external control (sometimes “synthetic control” or “historical control”).

In this design:

The treatment arm is prospective, treated under a defined protocol.
The control arm comes from a separate dataset (regardless of when or how collected) and did not receive the investigational product under the same protocol.
Because the groups were not randomized together, there are extra risks of bias and confounding. The key question is: can the FDA rely on such evidence to reach its statutory standard of “substantial evidence of effectiveness”?
In February 2023 the FDA issued a draft guidance for industry titled “Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products”.

Here are some of the most important take-aways from that guidance:

The guidance explicitly recognizes that externally-controlled trials may, in some circumstances, serve as an adequate and well-controlled investigation to support approval of a drug or biologic.
But the guidance is cautious: it states that in many cases “the likelihood of credibly demonstrating the effectiveness of a drug of interest with an external control is low.”
The guidance puts heavy emphasis on using patient-level data (not just summary published numbers) for the external control.
It emphasizes key threats: unmeasured confounding, bias, differences in assessment or follow-up, intercurrent events, immortal time bias, and temporal changes in standard of care or diagnostics.
It recommends early and frequent communication between sponsor and FDA if an external control design is under consideration.
It doesn’t endorse a single statistical adjustment method (propensity scores, Bayesian modelling, etc.). Instead, it says the analytic plan must be prespecified, transparent, and justified in the context of the specific trial.
The guidance makes clear this is not the default approach—it remains a specialized tool rather than the standard route.

In short: the FDA is signaling openness to external controls in selected settings—but the bar remains high.

How this ties to the two recent cases

Here’s how the guidance connects to the two real-world situations.

Case 1: uniQure and AMT-130 in Huntington’s disease

uniQure reported a treated cohort using AMT-130 (a one-time gene therapy) and compared outcomes to a natural-history/external cohort.
The agency essentially pushed back: although the signals were strong, the design raised concerns (especially given the lack of randomization, external control issues) and the company reported that pre-BLA discussions did not confirm a clean path.
Under the FDA external-control guidance, the concerns would include: are the treated and external groups sufficiently similar (baseline disease features, prior therapies, timing)? Was the index date (“time zero”) aligned? Are the outcome assessments the same, and is the follow-up and endpoint definition comparable? Are missing data or unmeasured confounders plausibly influencing the result? The guidance states that if any of these are weak, the design may not credibly distinguish drug effect from other influences.
The uniQure case illustrates: even with promising effect size, the agency may conclude that uncertainties tied to external control reduce confidence in the evidence.

Case 2: Biohaven Pharmaceuticals and Vyglxia (troriluzole) in Spinocerebellar ataxia

Biohaven’s NDA included a large externally-controlled dataset (real-world evidence for the control arm) reporting disease-slowing.
The FDA issued a Complete Response Letter (CRL) citing concerns with the external-control-based evidence (e.g., bias, data‐quality, missing/unmeasured information). According to Biohaven, the agency rejected its drug, called troriluzole, due to issues that can be "inherent to real-world evidence and external control studies, including potential bias, design flaws, law of pre-specification and unmeasured confounding factors."
According to the guidance, when the anticipated treatment effect size is modest, an externally controlled trial is less likely to be acceptable unless the design is very strong.
Also the guidance emphasizes that outcome ascertainment, timing, and data source differences between arms are key threats. The Biohaven case appears to reflect those issues.
This case reinforces that external controls are not a way to bypass rigorous design—they still demand high‐quality data, pre-specification, and detailed justification.

What this means for patients, clinicians and developers

For patients & clinicians: When you see a newly approved drug whose pivotal evidence is based on an external control rather than a randomized trial, ask: how comparable were the groups? How solid was the external data (patient-level, good follow-up, similar measurement)? Has a confirmatory trial been required or planned?
For developers: If you plan to rely on an external control, start very early: identify and curate the external dataset, lock the eligibility and analytic plan, engage FDA in frequent meetings, anticipate the agency will probe data quality, comparability, missing data and bias. Treat the external control design as a rigorous undertaking—not a shortcut.
For the field of rare diseases and high-unmet-need areas (for example your interest area of pulmonary arterial hypertension): External controls offer a promising option when randomization is infeasible—but you must make the case strongly. The FDA’s guidance gives you the checklist: patient-level data, alignment of arms, clear index date, consistent endpoint measurement, robust analytics. If you can’t satisfy those, a randomized or concurrent control may still be needed.

Final thoughts

The FDA’s external‐control guidance is a signal that regulators recognize the realities of these challenging settings—but it does not mean external controls are easy, or will automatically yield approval. The recent uniQure and Biohaven cases are a clear reminder: you may have impressive effect size or compelling unmet need—but the agency will still closely scrutinize design, data source, comparability, missing data, and bias.

FDA Commissioner's National Priority Voucher (CNPV) program versus Priority Review Voucher (PRV)

2025-11-06T23:00:00.002-05:00

FDA created a new voucher program called "Commissioner's National Priority Voucher (CNPV) Pilot Program" to accelerate Drug Review for Companies Supporting U.S. National Interests. The CNPV pilot program offers an unprecedented opportunity to reduce drug and biological product application or efficacy supplement (ES) review times from 10-12 months to just 1-2 months. Announced in June 2025, this innovative program uses a collaborative tumor board style review process to accelerate approvals for companies aligned with critical U.S. national health priorities.

On October 16, 2025, FDA Awards First-Ever National Priority Vouchers to Nine Sponsors. The following 9 products were selected:

Pergoveris for infertility
Teplizumab for Type I diabetes
Cytisinicline for nicotine vaping addiction
DB-OTO for deafness
Cenegermin-bkbj for blindness
RMC-6236 for pancreatic cancer
Bitopertin for porphyria
Ketamine for domestic manufacturing of a critical drug for general anesthesia
Augmentin XR for domestic manufacturing of a common antibiotic

On November 06, 2025, FDA Awards Second Batch of National Priority Vouchers. The following 6 products were selected following external applications and internal nominations from FDA review divisions:

Zongertinib for HER2 lung cancer
Bedaquiline for drug-resistant tuberculosis in young children
Dostarlimab for rectal cancer
Casgevy for sickle cell disease
Orforglipron for obesity and related health conditions
Wegovy for obesity and related health conditions

The new voucher program was criticized in this NEJM article “Flaws in the FDA’s New Priority Voucher Program” by Carpenter, Hwang, and Kesselheim. The authors argue that while the program seeks to promote innovation and address public health needs, it has significant shortcomings. They note that regulatory review already represents a small portion of total drug-development time and that past voucher programs have shown little evidence of spurring innovation while straining FDA resources. The paper warns that the CNPV program, created without congressional authorization and with vague eligibility criteria, risks politicizing FDA decisions, fostering conflicts of interest, and undermining public trust. Moreover, the compressed review timelines could compromise drug safety and overburden staff. The authors suggest that if implemented, the program should focus on generic drugs where expedited review could have tangible benefits, and should incorporate strong transparency, conflict-of-interest safeguards, and legislative oversight to maintain the integrity of the FDA’s regulatory process.

In a previous post, the FDA's Priority Review Voucher (PRV) programs were discussed. Here’s a side-by-side comparison table summarizing the key similarities and differences between the Commissioner’s National Priority Voucher (CNPV) and the Priority Review Voucher (PRV) programs such as the Rare Pediatric Disease PRV:

Feature	Commissioner’s National Priority Voucher (CNPV)	Priority Review Voucher (PRV) – e.g., Rare Pediatric Disease
Origin / Authorization	Created by the FDA Commissioner in 2025 as a pilot program without explicit congressional authorization.	Created by Congress through legislative action (e.g., the 2012 FDA Safety and Innovation Act for rare pediatric diseases).
Purpose	To accelerate review for drugs aligned with U.S. national health priorities, including unmet needs, national security, innovation, and affordability.	To incentivize development of drugs for neglected or rare conditions (e.g., tropical or rare pediatric diseases).
Review Time Benefit	Shortens FDA review time for selected drugs to 1–2 months, much faster than any existing program.	Converts a standard 10-month review to a priority 6-month review for another drug application.
Eligibility Criteria	Broad and somewhat opaque—drugs must align with “national priorities,” such as public health crises or domestic manufacturing.	Clearly defined by statute—applies to drugs for designated rare pediatric diseases (or other legislatively defined conditions).
Voucher Transferability	Nontransferable (can only be used by the same sponsor; valid if company ownership changes).	Transferable—can be sold or traded to another company, often for hundreds of millions of dollars.
Voucher Expiration	Expires after 2 years if unused.	Does not expire.
Selection and Oversight	Controlled by the FDA Commissioner’s office, raising concerns about political influence and conflicts of interest.	Statutory oversight and criteria limit discretion; implementation and tracking are agency-administered but legislatively mandated.
Scope / Eligible Products	Focused on drugs supporting U.S. national interests (innovation, security, affordability).	Focused on drugs addressing specific rare or neglected diseases.
Impact on FDA Workload	Could significantly burden FDA staff due to extremely short review timelines and unfunded mandates.	Adds workload but within a manageable 6-month priority window, and user fees help fund FDA resources.
Pilot Structure / Limits	Initially limited to five awardees in the first year; long-term limits unclear.	Ongoing statutory program with defined eligibility and reporting mechanisms.
Evidence of Effectiveness	No evidence yet; experts predict limited impact on overall development time and potential risks to review quality.	Empirical studies show limited evidence of incentivizing innovation but measurable market and financial impacts.
Key Concerns	Politicization, conflicts of interest, safety risks from rushed reviews, lack of transparency, and legal vulnerability due to lack of statutory basis.	Limited innovation incentive, windfall profits for some sponsors, and added workload, but more transparent and regulated.
Suggested Improvements	Apply to generic drugs, establish clear criteria, ensure independent review, legislative authorization, and transparency.	Better alignment between voucher incentives and public health impact; continue to monitor postmarket safety.

While both CNPV and PRV programs aim to accelerate access to important therapies, the CNPV is a politically driven, non-statutory pilot emphasizing national priorities, with potentially risky acceleration timelines and limited oversight. In contrast, the PRV system—though imperfect—has a clear legislative framework, defined eligibility, and established oversight mechanisms that maintain greater regulatory transparency and accountability.

Industry-Sponsored N-of-1 Trials in Orphan Drug Development

2025-09-17T19:30:00.001-04:00

N-of-1 clinical trial is often mentioned as one of the approaches to address the challenging issues in orphan drug development. at one point, FDA rolled out more guidance on 'N of 1' gene therapies including the guidance on "Individualized Antisense Oligonucleotide Drug Products for Severely Debilitating or Life-Threatening Diseases". AHRQ had a publication "Design and Implementation of N-of-1 Trials: A User’s Guide". Recent "baby KJ's story" was touted by the FDA as a success story for individualized treatment for a N-of-1 disease condition "FDA Cell & Gene Therapy Roundtable: Putting Every Patient Within Reach of Innovation". Nature magazine had an article "A framework for N-of-1 trials of individualized gene-targeted therapies for genetic diseases"

We would like to check the reality to see how many drugs or biological products were approved by the US FDA based on the evidence from N-of-1 clinical trials.

We identified a few ultra-rare disease cases in the past decade where industry or institutional sponsors conducted single-patient (“N-of-1”) trials of experimental therapies. Notably, no drug or biologic approved in the last 10 years appears to have relied on N-of-1 clinical trial data for its FDA approval. The only FDA approval historically based in part on N-of-1 evidence was danazol for hereditary angioedema in the 1980s. Below we summarize key examples of N-of-1 trials involving drugs/biologics and rare indications:

Drug/ Biologic	Orphan Indication	Sponsor (Company/Institution)	N-of-1 Trial Design & Results	Regulatory Outcome
Milasen (custom ASO)	CLN7 Batten disease (Infantile neuronal ceroid lipofuscinosis)	Boston Children’s Hospital (Timothy Yu’s lab)	Single-patient, customized antisense oligonucleotide. FDA cleared an IND for Mila in late 2017, and she received serial intrathecal doses. Her seizure frequency fell dramatically (from ~30/day to <10/day) and her neurologic decline stabilized during treatment.	Investigational (IND only; no FDA marketing approval)
Custom ASO for A-T (no trade name)	Ataxia-Telangiectasia	Boston Children’s Hospital / A-T Children’s Project (Dr. Yu)	Personalized splice-modulating ASO for one AT patient. FDA granted an IND in 2020 for an N-of-1 trial. The child began receiving the ASO (via intrathecal delivery) later in 2020; detailed clinical outcomes have not yet been published.	Investigational (IND only; ongoing treatment)
CRD‑TMH‑001 (CRISPR-based gene therapy)	Duchenne muscular dystrophy (specific exon 1/promoter mutation)	Cure Rare Disease (nonprofit biotech)	Single-patient CRISPR/Cas9 gene-editing therapy. FDA cleared an IND in August 2022 for a one-person trial. The only enrolled patient (the sponsor’s founder’s brother) was dosed but later died (cause under review). As a result, efficacy remains unknown.	Investigational (IND only; trial suspended)
Danazol	Hereditary angioedema	Original developer (Searle/Pfizer)	Historical example: A multi-crossover N-of-1 trial (9 patients, 47 treatment periods) was reported in 1976. This small trial of danazol demonstrated reversal of HAE symptoms and formed part of the FDA approval record.	Approved (for HAE; orphan indication) https://www.fda.gov/media/87621/download section 5.2.4

Each case is discussed in more detail below.

Milasen for Batten Disease (CLN7)

Drug & Sponsor: Milasen is a one-off antisense oligonucleotide (ASO) developed at Boston Children’s Hospital by Dr. Timothy Yu’s team. It was designed for an 8‑year-old patient (“Mila”) with a unique retrotransposon mutation in the CLN7 gene causing Batten disease.
N-of-1 Design & Results: After rapid sequencing and preclinical work, the FDA granted an IND to trial milasen in late 2017. Mila then received intrathecal milasen injections every 3 months under an N-of-1 protocol. Over ~9 months of treatment, Mila’s seizure frequency fell from ~30/day to under 10/day, and her neurological decline notably stabilized. No significant adverse effects were reported, and her condition improved in several clinical measures. These results were published in the New England Journal of Medicine (2019).
Regulatory Outcome: Milasen remains an investigational therapy given under an IND. It was not submitted for FDA marketing approval (and Mila later died in early 2021). Thus, no FDA approval was sought or granted for milasen.

Customized ASO for Ataxia-Telangiectasia

Drug & Sponsor: A patient-specific splice-switching ASO (unnamed) was developed by Dr. Yu’s group (supported by the A-T Children’s Project) for a young girl with ataxia-telangiectasia (AT). The ASO was tailored to correct her particular ATM gene splicing defect.
N-of-1 Design & Results: The FDA granted IND approval for this one-off ASO treatment in 2020actionforat.org. The affected child began receiving the ASO via intrathecal infusion that year, under a single-patient trial protocol. (To our knowledge, no detailed efficacy data from this N-of-1 trial have yet been published.)
Regulatory Outcome: This custom AT therapy is in the investigational stage (IND only). No FDA marketing approval has been sought or obtained.

CRD‑TMH‑001: CRISPR Therapy for Duchenne Muscular Dystrophy

Drug & Sponsor: CRD‑TMH‑001 is a one-time, custom CRISPR/Cas9 gene-editing therapy developed by Cure Rare Disease (a nonprofit “n=1” biotech) to target a specific exon 1/promoter mutation in the DMD gene. Cure Rare Disease’s founder (motivated by his brother’s illness) sponsored the program.
N-of-1 Design & Results: In mid-2022, the FDA cleared a first-in-human IND for CRD-TMH-001. The trial was explicitly a single-patient study: one eligible DMD patient (age 27, the sponsor’s brother) was treated. The therapy was infused intrathecally as planned. Tragically, the patient died several months later; a causality review is ongoing. To date, no efficacy data (or definitive safety findings) from this N-of-1 trial have been published.
Regulatory Outcome: CRD-TMH-001 remains purely investigational under the IND. No further patients have been treated and no FDA approval has been pursued.

Historical FDA Approval: Danazol for Hereditary Angioedema

Drug & Indication: Danazol is a synthetic steroid used to prevent attacks of hereditary angioedema (HAE), a rare genetic edema disorder.
N-of-1 Trial Evidence: In 1976, a classic FDA review noted that the approval of danazol for HAE was supported in part by an N-of-1 multiple-crossover trial of nine patients (47 treatment periods). In that study, each subject alternated between danazol and placebo in random sequence, demonstrating clear reversal of clinical and biochemical abnormalities on danazol. This case was cited in section 5.2.4 of the FDA's Good Review Practice: Clinical Review of Investigational New Drug Applications
Regulatory Outcome: Danazol was approved for HAE prevention (an orphan indication) and remains an approved drug (marketed as DANOCRINE). This case shows that the FDA has historically accepted single-subject crossover data in rare diseases. However, it is an older example; no similar approvals based on N-of-1 data have emerged in the past decade. The N-of-1 study of Danazol in HAE was published in New England Journal of Medicine in 1976 "Treatment of Hereditary Angioedema with Danazol — Reversal of Clinical and Biochemical Abnormalities"

Summary

In summary, we found only a handful of “N-of-1” clinical trials over the past 10 years involving investigational drugs for very rare diseases. These include two custom antisense oligonucleotides (Mila’s milasen for Batten disease, and an ASO for one AT patient) and one CRISPR gene therapy for Duchenne MD. All were “approved” only under individual INDs and have shown promising signals but remain experimental; none has led to FDA marketing approval. Apart from the historical example of danazol (approved via N-of-1 data in the 1970s), our searches of FDA databases, ClinicalTrials.gov, and published literature did not identify any new orphan drugs approved based on N-of-1 trial evidence. Thus, while N-of-1 trials are an emerging tool for ultra-rare diseases, they have not yet changed the landscape of FDA approvals in the last decade.

Steroid Tapering Design in Action - Failed

2025-09-15T23:00:00.001-04:00

In a previous post, I discussed steroid tapering design clinical trials. In these trials, the primary efficacy endpoint is the reduction in steroid dose. Efficacy is demonstrated if patients receiving the experimental treatment are able to reduce their steroid dose significantly more than those in the placebo group.

Steroids remain highly effective for many conditions, particularly those involving chronic inflammation or an overactive immune system. However, long-term or high-dose steroid use is associated with numerous side effects. Therefore, it is highly desirable to develop therapies that allow for steroid dose reduction—or even complete steroid sparing—while maintaining disease control.

One of these diseases is sarcoidosis or lung sarcoidosis. Sarcoidosis is an inflammatory disease in which the immune system overreacts, causing groups of cells to form clusters of inflamed tissue called "granulomas" in one or more organs of the body. The most affected organ is lung. Sarcoidosis symptoms can be treated using corticosteroids, or prednisone, which turn down the immune system's activity to reduce inflammation.

A sponsor, aTyr Pharma, conducted a phase 3, confirmatory trial to investigate if their experimental drug efzofitimod is effective in treating the pulmonary sarcoidosis. The study "Efficacy and Safety of Intravenous Efzofitimod in Patients With Pulmonary Sarcoidosis" was designed as steroid tapering with the primary efficacy endpoint being "Change from baseline in mean daily oral corticosteroid (OCS) dose at Week 48".

This morning, the sponsor issued a press release announcing Topline Results from Phase 3 EFZO-FIT™ Study of Efzofitimod in Pulmonary Sarcoidosis. Study did not meet primary endpoint in change from baseline in mean daily oral corticosteroid (OCS) dose at week 48, although clinical benefit for efzofitimod observed across multiple study parameters.

The change from baseline in mean daily OCS dose reduced to an average of 2.79 mg for the high dose of efzofitimod versus 3.52 mg for placebo, resulting in an insignificant treatment difference. The sponsor blames the unexpected placebo response.

aTyr flunks phase 3 lung disease trial, deflating stock, but still plans talks with FDA

The steroid tapering design offers important clinical advantages by directly addressing the need to reduce long-term steroid exposure, a major source of morbidity in many chronic inflammatory and immune-mediated diseases. The steroid tapering design and the change from baseline in steroid dose is accepted by the regulatory agencies. It provides a patient-centered and easily quantifiable endpoint that demonstrates whether an investigational treatment can maintain disease control while allowing for a lower steroid dose.

However, this design also has limitations. Variability in tapering schedules, risk of disease flares, and ethical concerns regarding aggressive tapering in placebo groups can complicate interpretation. In addition, results may be less generalizable to steroid-naïve patients, and regulatory acceptance of steroid-sparing endpoints as stand-alone primary outcomes remains uncertain. Careful planning and standardization are therefore essential to balance the scientific value with patient safety and trial credibility.

ICH E21 "Inclusion of Pregnant and Breastfeeding Individuals in Clinical Trials"

2025-07-20T23:00:00.001-04:00

US FDA has just endorsed the draft guideline of ICH E21 "Inclusion of Pregnant and Breastfeeding Individuals in Clinical Trials". E21 Training Material can also be found at ich.org.

The ICH E21 guideline aims to provide recommendations for the appropriate inclusion and retention of pregnant and breastfeeding individuals in clinical trials. This is crucial because historically, these populations have been largely excluded, leading to a significant lack of data on the safety and efficacy of medicinal products during pregnancy and lactation.

For clinical trialists, the guideline emphasizes:

Proactive Planning: Sponsors should consider strategies to generate data on medicinal product use in pregnancy and breastfeeding from early stages of development, including nonclinical studies through post-approval.
Risk-Benefit Evaluation: A thorough assessment of potential risks and benefits for both the pregnant or breastfeeding individual and the fetus/breastfed infant is essential.
Ethical Considerations and Informed Consent: The guideline stresses the importance of obtaining truly informed consent, ensuring clear communication of potential risks and benefits, and avoiding any undue influence or coercion. Efforts should be made to reduce the burden of study procedures on these participants.
Study Design and Implementation: It provides practical guidance for designing clinical trial protocols, including considerations for data collection, safety monitoring, and follow-up for both mother and child. This also includes pharmacokinetic (PK) and dosing considerations, as physiological changes during pregnancy can impact drug pharmacokinetics.
Addressing Unmet Medical Need: The inclusion of pregnant and breastfeeding individuals is particularly important for conditions where there is a high unmet medical need for treatment in pregnancy or while breastfeeding, though the guideline's scope is not limited to these scenarios.
Data Generation: The overarching objective is to facilitate the generation of robust clinical data that allows for evidence-based decision-making on the safe and effective use of medicinal products by these individuals and their healthcare providers.

The ICH E21 guideline is a significant step towards improving the evidence base for prescribing medications during pregnancy and lactation, ultimately aiming for better patient outcomes for these underserved populations.

Just several weeks ago, I wrote a blog post on this same topic "Managing Pregnancies in Clinical Trials: Regulatory Guidance and Best Practices", which offered a comprehensive overview of the historical exclusion of pregnant women from clinical trials, the reasons behind this, and the evolving regulatory landscape advocating for their inclusion. The article emphasized a balanced approach, highlighting FDA's shift towards diversity in trials and the ethical imperative to gather data on drug use in pregnancy to ensure evidence-based care. It delved into practical aspects like informed consent, pre-screening procedures, managing pregnancies discovered during trials (including stopping drug, unblinding, and re-consenting), and the importance of detailed data collection and specialized safety monitoring. The blog post also strongly argued for the ethical benefits of inclusion, asserting that systematic exclusion leads to greater harm by forcing healthcare providers and patients to make decisions without adequate evidence.

The newly endorsed ICH E21 guideline directly aligns with various FDA guidance described in the blog post and significantly reinforces the "balanced approach" and the ethical arguments.

ICH E21 provides a more harmonized, international standard for how this inclusion should be achieved proactively throughout drug development.
ICH E21 offers detailed recommendations on study design, ethical considerations, pharmacokinetic and dosing adjustments, safety monitoring, and data collection specifics for pregnant and breastfeeding individuals, directly addressing the "when and how" question you pose at the end of your article.
ICH E21 reiterates the need for IRBs with maternal-fetal expertise, suggesting consultation with specialists for study design and safety monitoring.
ICH E21 highlights the critical importance of robust informed consent and ongoing safety monitoring for both the mother and the child.
ICH E21 essentially provides the detailed, internationally agreed-upon framework, moving the industry further towards systematic and responsible inclusion rather than historical exclusion.

Rank Analysis of Covariance (RANK ANCOVA) versus Aligned Rank Stratified Wilcoxon Test

2025-07-04T23:00:00.003-04:00

In clinical trials where outcome measures violate the assumptions required for parametric statistical methods such as analysis of covariance (ANCOVA), non-parametric approaches are often employed. The classical two-sample Wilcoxon rank-sum test (also known as the Mann–Whitney U test) can be used in such cases; however, this method does not allow for adjustment of covariates.

Randomized clinical trials often adjust for baseline covariates (FDA 2021 guidance "Adjusting for Covariates in Randomized Clinical Trials for Drugs and Biological Products") to improve precision. When the outcome is non‐normal or ordinal, rank‐based methods offer alternatives to parametric ANCOVA. Two popular approaches are: (1) Rank ANCOVA – an ANCOVA on rank‐transformed data (the “rank‐transformation ANCOVA” of Conover and Iman), and (2) Aligned‐Rank Stratified Wilcoxon – a Wilcoxon rank‐sum test applied to responses “aligned” by covariate effects (e.g. Hodges‐Lehmann or van Elteren style). Below we compare their theory, practical use, pros/cons, and SAS implementations.

Theoretical Properties

Rank ANCOVA (Rank‐Transform ANCOVA):

Replace the outcome Y by its overall ranks and fit a usual linear ANCOVA model (treatment + covariates). Under the null it tests whether the treatment coefficient in the rank‐scale model is zero. In effect, the null is “no difference in the adjusted rank‐distributions” between groups. This approach assumes a linear additive model for covariate effects (on the rank‐scale) with homogeneous slopes across treatments. Conover and Iman showed the rank‐transform inherits the robustness and power properties of rank tests in regression. However it is not fully model‐free: violations of slope homogeneity or covariate–treatment interactions can invalidate the test. The test statistic is simply the usual F (or t) from ANCOVA on ranks. Rank ANCOVA was proposed by Dana Quade (1967) "Rank Analysis of Covariance" in JASA and was popularized after the book by Stokes, Davis, and Koch "Categorical Data Analysis Using SAS" where there was a dedicated chapter to discuss the RANK ANCOVA.

Aligned‐Rank Stratified Wilcoxon:

Agligns outcomes within each randomization stratum by subtracting the stratum’s Hodges–Lehmann shift (a median-based location estimate) and then applies a Wilcoxon rank-sum test to the pooled aligned data. Effectively performs an un-stratified Wilcoxon test on aligned values. Yields a Hodges–Lehmann estimate of the overall median shift (with confidence interval) as the treatment effect.

This method is fundamentally a rank‐sum test controlling for strata or covariates. In a stratified design (or ANCOVA context), one assumes a model Yij=μ+βi+τjY_{ij}=\mu +\beta_i+\tau_j where βi\beta_i are strata/covariate effects and τj\tau_j are treatment effects. The null hypothesis is τ1=τ2=0\tau_1=\tau_2=0 (no treatment shift). To remove (align) the strata effects, one subtracts a location statistic (stratum mean/median/Hodges–Lehmann) from each YijY_{ij}, yielding “aligned” responses. These aligned values (now centered by stratum) are pooled and ranked, and a Wilcoxon test is performed ignoring strata. Equivalently, with a STRATA factor one can perform a Van Elteren (stratified Wilcoxon) test. The resulting null distribution is distribution‐free (asymptotically normal, exact via permutations), requiring only that within each stratum the treated and control distributions differ by a location shift. In summary, Rank ANCOVA assumes a linear rank‐model for covariate adjustment, whereas the Aligned‐Rank Wilcoxon assumes only an additive strata effect (with no specific distribution form) and tests a location‐shift null in a stratified rank framework.

Practical Applications

Outcome types: Both methods suit continuous or ordinal outcomes that violate parametric assumptions. Rank ANCOVA can handle multiple continuous covariates straightforwardly (since one simply adds them to the ANCOVA on ranks). Aligned‐rank tests naturally handle categorical strata (e.g. randomization factors, center, or baseline strata). If the covariate is continuous (e.g. baseline measure), one may either form strata (e.g. quantiles) or perform alignment via regression residuals before ranking.

Robustness: Both are robust to outliers and non‐normality because they use ranks. Conover and Iman noted that rank‐transformed regression is robust and powerful even under heavy‐tailed or skewed data. The aligned‐rank approach is fully distribution‐free under its null hypothesis, and by aligning removes nuisance location effects (e.g. site or block means). In practice, Ye and Lai (2023) found that both a covariate‐adjusted rank‐sum (rank ANCOVA) and an aligned‐rank test yielded narrower confidence intervals and maintained type I error across clinical trials, compared to unadjusted tests.

Small samples: Neither method “magically” solves small‐N issues. Rank ANCOVA relies on large‐sample ANCOVA (normal‐theory) approximations on ranks. In contrast, the aligned‐rank/Wilcoxon method can use exact computations (via PROC NPAR1WAY) if samples are very small. However, note that alignment can behave erratically in very small samples according to some studies. In moderate samples both methods are generally acceptable. Both methods lose some efficiency if there are many ties or if covariate effects are very non‐linear.

Advantages and Disadvantages

Rank ANCOVA

– Advantages: It is conceptually simple (just rank the data and run standard ANCOVA). It fully utilizes continuous covariates in a regression framework, and retains much of the power of ANOVA tests while being robust to non‐normal errors. If the linear model is correct (in the rank‐scale), the test is valid and can easily test interactions. Simulations suggest rank‐based ANCOVA is often powerful when parametric assumptions fail. Adjusting for covariates in this way typically yields more precise estimates (narrower CIs) than unadjusted ranks.

– Disadvantages: Its interpretation is subtle: one is testing effects on the ranks of the outcome, not its mean or median in original units. Thus the “treatment effect” corresponds to a location shift in the ranked distribution. Experts caution that rank‐transform ANCOVA is not a direct test of medians in the original scale, and may give misleading inference if reporting medians. It also implicitly ranks the covariates (if one ranks them too) which can distort relationships. If the homogeneity of slopes assumption is violated, the rank‐ANCOVA test can be invalid. In summary, it is not fully nonparametric – it still relies on the linear model structure (albeit on ranks) and on large‐sample approximations.

Aligned Rank Wilcoxon

– Advantages: This method is fully nonparametric under a location‐shift null, and can produce an easily interpreted Hodges–Lehmann median shift estimate with confidence limits (e.g. via ALIGN=STRATA(HL) in SAS). It inherently accounts for stratification/baseline effects by alignment, so it naturally handles block effects or randomization strata (van Elteren’s approach). It does not require ranking the covariate itself – only the outcome after centering. In practice it has robust Type I error even under heteroscedasticity or skewness, so long as the alignment model is reasonable. SAS’s PROC NPAR1WAY provides this test directly (see below).

– Disadvantages: One must specify the strata or alignment model in advance. If there is only one continuous covariate, one needs to decide how to align (e.g. subtract predicted baseline effect or stratum median). The choice of alignment statistic (median vs mean vs HL) can affect results slightly. If strata are very small, the within‐stratum rankings may be unstable. Unlike rank‐ANCOVA, this approach cannot easily incorporate arbitrary continuous covariates without discretization or pre‐alignment. In very small samples, alignment procedures can suffer from erratic Type I behavior (especially if alignment assumptions are misspecified). Finally, it is less familiar to many practitioners and thus may be harder to explain.

SAS Implementation

Rank ANCOVA: SAS does not have a dedicated “rank‐ANCOVA” proc. A common workaround is to rank the data (using PROC RANK) and then run a standard ANCOVA (PROC GLM or PROC REG) on the ranked outcome. For example:

proc rank data=trial out=ranked ties=mean;

var Y; ranks Y_rank;

run;

proc glm data=ranked;

class Trt;

model Y_rank = Trt Baseline;

run;

This fits a linear model to the ranks. Alternatively, one can implement the Hettmansperger–McKean aligned‐rank procedure by first regressing Y on the covariate (e.g. with PROC REG or PROC ROBUSTREG) and then ranking the residuals to test the treatment effect. In short, one must manually rank or residualize and then use standard SAS procs. (There is no built‐in PROC RANKANCOVA or similar in SAS.)

The book "Categorical Data Analysis Using SAS®, Third Edition" by Stokes, Davis, and Koch contains the sample SAS codes indicating three steps in performing the Rank ANCOVA.
proc rank nplus1 ties=mean out=ranks;
   by center; 
   var before after;
run;
proc reg noprint; 
   by center;
   model after=before;
   output out=residual r=resid;
run;
proc freq;
   tables center*group*resid / noprint cmh2; 
run;

Aligned Rank Stratified Wilcoxon: SAS’s PROC NPAR1WAY supports both stratified Wilcoxon and aligned‐rank tests. For a Van Elteren stratified Wilcoxon, use the STRATA statement without alignment. For example:

proc npar1way data=trial wilcoxon;

class Trt;

strata Center; /* stratification variable */

var Change;

run;

This computes the stratified Wilcoxon (van Elteren) test and provides both one‐sided and two‐sided p-values. To perform an aligned‐rank test, use the ALIGN=STRATA option (with STRATA) in PROC NPAR1WAY. For example:

proc npar1way data=trial wilcoxon align=strata(hl);
class Trt;
strata Center;
var Change;
run;

This subtracts the stratum median (or Hodges–Lehmann shift if (HL) is specified) from each response before ranking, then conducts the Wilcoxon test. The output will include the Hodges‐Lehmann estimate and CI for the location shift. Thus, PROC NPAR1WAY with the STRATA and ALIGN=STRATA options directly implements the aligned‐rank stratified Wilcoxon test. (By default, RANKS=STRATUM is used and weights are by stratum, yielding van Elteren.)

Notice that for both RANK ANCOVA and Aligned Rank Stratified Wilcoxon test, only p-value will be obtained. The treatment difference (so called the difference in medians or location shift in medians) needs to be calculated using Hodges-Lehman estimator.

Side-by-Side Comparison of RANK ANCOVA and Aligned Rank Stratified Wilcoxon

The table below summarizes these points:

	Rank ANCOVA (Rank‐transform ANCOVA)	Aligned‐Rank Stratified Wilcoxon
Null hypothesis	H₀: no treatment effect on the ranked outcome (treatment coeff=0 in rank‐scale model).	H₀: no treatment effect (no location shift) in stratified model (τ1=τ2=0\tau_1=\tau_2=0).
Model / Assumptions	Linear model on ranks: assume covariate effects are additive and linear on the rank‐scale, with equal slopes across groups. (No distributional form assumed beyond this.)	Additive strata model: Y=μ+βi+τjY=\mu+\beta_i+\tau_j. Assume observations are exchangeable within strata after alignment. Does not assume specific distribution shape.
Test statistic	ANCOVA F or t on the ranked outcome (i.e. usual parametric test applied to ranks).	Wilcoxon rank‐sum on aligned data. (Equivalently, stratified Wilcoxon/Van Elteren statistic.)
Distribution	Uses large‐sample normal/chi‐square approximations (from GLM on ranks). No exact test available in SAS for this.	Asymptotic normal (z) or exact (via permutation) available. SAS PROC NPAR1WAY can compute exact Wilcoxon p-values within strata.
Outcome type	Continuous or ordinal outcomes. Can include multiple continuous covariates in model.	Continuous or ordinal outcomes. Requires (or creates) strata: typically categorical covariates (e.g. randomization strata) or aligned by regression.
Robustness / Outliers	Not fully nonparametric. Robust to outliers and non‐normality (rank‐based). However, if covariate–treatment interactions exist or slope equality fails, Type I error can inflate.	Fully nonparametric. Robust to outliers (rank‐based) and handles non‐normal/heteroscedastic data well. Alignment removes nuisance location shifts.
Small sample	Relies on asymptotic ANCOVA on ranks; no built‐in exact test. May be liberal if sample is very small or distribution very discrete.	PROC NPAR1WAY can use exact Wilcoxon (by strata) for small N. Alignment in tiny samples may have less stable Type I.
Advantages	Easy to implement via standard ANCOVA tools. Uses full continuous covariate information. Retains high power under model correctness. Covariate adjustment usually reduces variance (narrower CIs).	Nonparametric (distribution‐free) test. Directly yields Hodges–Lehmann shift estimate and CI. Naturally incorporates stratification/blocks (van Elteren). Valid under mild assumptions.
Disadvantages	Tests on ranks, not raw scale, so interpretation of effect size is not straightforward. Not a test of medians in original units. Can mislead if model assumptions fail. No simple SAS proc – must manually rank or regress.	Must predefine strata or alignment model. Less flexible for multiple continuous covariates (usually one strata factor). Alignment choice (median vs mean) can affect results. In small samples, alignment may behave poorly.
SAS implementation	No single procedure. Typically use PROC RANK to create ranked Y, then PROC GLM (or PROC REG) with covariates on ranks. Alternatively, regress Y on covariate (PROC REG/ROBUSTREG), rank the residuals, and test group difference. (All manual steps.)	Use PROC NPAR1WAY. For stratified Wilcoxon: STRATA statement (no ALIGN) yields Van Elteren test. For aligned‐rank: add ALIGN=STRATA (and optionally (HL) or (MEAN) option) in PROC statement. E.g. proc npar1way data=… wilcoxon align=strata(hl); class Trt; strata Covar; var Y; run;.

Some Studies Where the Rank ANCOVA or Aligned Rank Stratified Wilcoxon Were Used:

Hoeper et al (2023) "Phase 3 Trial of Sotatercept for Treatment of Pulmonary Arterial Hypertension" and details in the attached protocol and SAP - Aligned Rank Stratified Wilcoxon was used.
Statistical Analysis Plan for "A Phase 2b, Dose-Ranging, Randomized, Double-Blind, Placebo Controlled, Multicenter Study of Rodatristat ethyl in Patients with Pulmonary Arterial Hypertension" - Aligned Rank Stratified Wilcoxon was used.
Van Dyck et al (2022) "Lecanemab in Early Alzheimer’s Disease" and details in the attached protocol and SAP - rank analysis of covariance was used for sensitivity analysis to assess the robustness of the primary efficacy analysis using MMRM
NDA 22,535/0045 Pirfenidone in the treatment of IPF - Rank ANCOVA was used
NDA 207533/ O-1 ARISTADA for Treatment of Schizophrenia - Rank ANCOVA was used

ICH E20 "Adaptive Designs For Clinical Trials" and FDA Guidance "Adaptive Designs for Clinical Trials of Drugs and Biologics"

2025-07-01T23:30:00.001-04:00

The International Council for Harmonisation (ICH) has released its highly anticipated draft guideline that harmonizes principles for the design, analysis, and reporting of adaptive clinical trials in new drug development. ICH E20 "Adaptive Designs for Clinical Trials" was endorsed by ICH last week on June 25, 2025. This was 5 1/2 years after FDA's guidance for industry "Adaptive Designs for Clinical Trials of Drugs and Biologics".

Both documents aim to provide guidance on the use of adaptive designs in clinical trials. The ICH E20 guideline is a draft version, and is currently under public consultation. It emphasizes that until a final guideline is agreed upon, the draft should not be understood as confirming full regulatory acceptance or superseding current regional guidance. The FDA guidance, issued in November 2019, represents the FDA's current thinking and provides nonbinding recommendations.

Here's a side-by-side comparison of common topics and key differences:

Side-by-Side Comparison: ICH E20 Draft Guideline vs. FDA Guidance

| 1. Purpose and Scope |

ICH E20: Provides guidance on confirmatory clinical trials with an adaptive design within the context of its overall development program. Defines adaptive design as "a clinical trial design that allows for prospectively planned modifications to one or more aspects of the trial based on interim analysis of accumulating data from participants in the trial." Emphasizes "prospectively planned". Excludes unplanned modifications or changes based on external information, and routine operational monitoring. Focuses on principles for planning, conduct, analysis, and interpretation of trials to confirm efficacy and support benefit-risk assessment. While primarily for confirmatory trials, principles are relevant to all phases. |

FDA: Provides guidance to sponsors on the appropriate use of adaptive designs for clinical trials to provide evidence of effectiveness and safety of a drug or biologic. Describes important principles for designing, conducting, and reporting results. Advises on information to submit for FDA evaluation, including Bayesian adaptive and complex trials relying on simulations. Primary focus is on adaptive designs for trials supporting drug effectiveness and safety, but concepts are useful for early-phase/exploratory trials and post-marketing commitments. Defines adaptive design as "a clinical trial design that allows for prospectively planned modifications to one or more aspects of the design based on accumulating data from subjects in the trial".

| 2. Advantages and Challenges |

ICH E20:

Advantages: Provides flexibility against inaccurate assumptions. Ethical advantages (e.g., early stopping for efficacy reduces exposure to inferior control). Improved efficiency (e.g., increased power for a given expected sample size). Improved understanding of treatment effects and decision-making (e.g., dose selection).
Challenges: Adds complexities and uncertainty to key principles. Logistical difficulties in maintaining confidentiality. Risks to trial integrity. More complex and time-consuming planning and assessment. Conventional analysis methods may lead to increased Type I error and biased estimates. May provide less information about safety. Not beneficial in all settings (e.g., fast enrollment relative to endpoint assessment). Requires clear and compelling justification weighing advantages against uncertainties.

FDA:

Advantages: Adjusts to information not available at trial start. Statistical efficiency (greater power or smaller expected sample size/shorter duration). Ethical considerations (e.g., early stopping for futility reduces patient exposure to ineffective treatment). Improved understanding of drug effects (e.g., adaptive enrichment, adaptive dose selection). Acceptability to stakeholders (e.g., sponsors willing to commit, patients willing to enroll in RAR trials).
Limitations: Requires specific analytical methods to avoid increasing erroneous conclusions and bias. Methods may not be readily available for complex designs, requiring simulations. Efficiency gains in some respects may be offset by losses in others (e.g., increased maximum sample size, longer lead times). Logistical challenges in trial conduct and integrity (e.g., limiting access to interim results, timely high-quality data). Limited utility in certain settings (e.g., minimum safety sample size, long outcome ascertainment period). Adaptive changes may lead to differing results before and after adaptation, challenging interpretability. |

| 3. Key Principles |

ICH E20:

Four key principles for confirmatory trials with adaptive designs: Adequacy Within the Development Program (proper design, conduct, analysis within overall program, limited complexity of adaptations at confirmatory stage).
Adequacy of Trial Planning (pre-specification of adaptations, timing, rules, statistical methods, integrity approaches, documentation).
Limiting the Chances of Erroneous Conclusions (controlling Type I error probability, understanding impact on safety and secondary endpoints, evaluating adequacy for trial objectives).
Reliability of Estimation (reliable and unbiased treatment effect estimates, accurate measures of uncertainty, evaluation of bias and variability, reliability of interim estimates for adaptation decisions).
Maintenance of Trial Integrity (limiting knowledge of unblinded data, use of independent IDMC, confidentiality agreements, minimizing inferred information from adaptations, documentation of processes). |

FDA:

Four key principles for trials providing substantial evidence of effectiveness: Controlling the Chance of Erroneous Conclusions (assessing probability of incorrect conclusions, addressing Type I error inflation due to multiple tests or adaptive features, use of simulations).
Estimating Treatment Effects (reliable estimates for benefit-risk, addressing bias in estimates and confidence intervals, using adjusted methods when available).
Trial Planning (complete pre-specification of design details, timing, type, statistical methods, adaptation algorithm; flexibility for deviations from anticipated algorithm with appropriate methods).
Maintaining Trial Conduct and Integrity (limiting access to comparative interim results, use of independent personnel like DMC, planning to avoid issues from knowledge of accumulating data, documentation of access control). |

| 4. Types of Adaptations |

ICH E20:

Discusses Early Trial Stopping (efficacy or futility, group sequential designs, stopping boundaries, impact on safety/secondary endpoints).
Sample Size Adaptation (based on nuisance parameters or interim treatment effect estimates, blinded vs unblinded, Type I error control, bias, maintaining integrity).
Population Selection (targeted subpopulations, enrollment restriction, combining data, Type I error control, bias, scientific rationale).
Treatment Selection (different doses/drugs, interim selection, combining data, Type I error control, bias).
Adaptation to Participant Allocation (Response-adaptive randomization, covariate-adaptive, challenges with time trends, integrity). |

FDA:

Discusses Group Sequential Designs (interim analyses, stopping for efficacy/futility, Type I error control, binding vs non-binding rules, impact on safety/secondary endpoints, adjusted estimates).
Adaptations to the Sample Size (based on nuisance parameters with/without treatment assignment, or comparative interim results, Type I error control, bias, trial integrity).
Adaptations to the Patient Population (e.g., Adaptive Enrichment) (modifications based on comparative interim results, hypothesis tests in multiple populations, multiplicity, scientific rationale, diagnostic devices).
Adaptations to Treatment Arm Selection (adding/terminating arms, dose-ranging, seamless designs, platform trials, Type I error control).
Adaptations to Patient Allocation (covariate-adaptive and response-adaptive randomization, Type I error, predictability, short-term outcomes).
Adaptations to Endpoint Selection (modification to primary endpoint choice, prespecified rules, multiplicity).
Adaptations to Multiple Design Features (combining features, complexity, simulations often critical).

| 5. Special Topics and Considerations |

ICH E20:

Further Considerations on Data Monitoring (IDMC expertise, access to unblinded data, charter, independent statistical group, sponsor access).
Planning, Conducting, and Reporting Simulation Studies (objectives, designs/analysis options, key operating characteristics, scenarios, implementation details, comprehensive report).
Adaptive Designs Using Bayesian Methods (ICH E9 principles, informing adaptations, borrowing external information, prior distribution, false positive control, simulations, sensitivity analyses).
Adaptive Designs in Time-to-Event Settings (number of events vs participants, follow-up time, independence assumption, Type I error control, longitudinal outcomes).
Adaptive Designs in Exploratory Trials (relevance of principles, more adaptations, reliable decision-making, sponsor role in monitoring, safety).
Operational Considerations (maintaining integrity, informed consent, randomization systems, drug supply, data quality/timeliness). |

FDA:

Simulations in Adaptive Design Planning (critical role, estimating operating characteristics, Type I error probability, nuisance parameters, number of iterations, precision, different random seeds).
Bayesian Adaptive Designs (various uses, frequentist vs Bayesian inference, informative priors, simulations).
Adaptations in Time-to-Event Settings (number of events vs subjects, follow-up, use of surrogate/intermediate outcomes).
Adaptations Based on an Intermediate Endpoint (surrogate/intermediate endpoints, correlation with primary, Type I error inflation, safeguards).
Secondary Endpoints (Type I error inflation, biased estimation, adjustment methods).
Safety Considerations (adaptations on safety endpoints, sufficient safety information, minimum sample size/exposure, risk of escalating doses).
Adaptive Design in Early-Phase Exploratory Trials (usefulness, less rigorous expectations, risk of erroneous conclusions).
Unplanned Design Changes Based on Comparative Interim Results (discouraged without FDA discussion, difficulty in Type I error control).
Design Changes Based on Information From a Source External to the Trial (acceptable, do not affect validity, challenging to ensure solely external). |

| 6. Documentation |

ICH E20:

Prior to conducting: Rationale (clinical and statistical), description of adaptations (aspects, timing, rule, estimands), statistical analysis methods (interim, primary/secondary, sensitivity), implementation details (who does what, committee roles), steps to maintain confidentiality/integrity (information transfer, access control, records), operating characteristics (simulation report if critical). Core elements in protocol, other details in separate documents (e.g., IDMC charter), details of adaptation rule may be restricted.
After completion (marketing application): All prospective plans, information on how design was implemented (actual timing, conduct variations, interim results, adaptation decisions, deviations), compliance with data access/integrity processes, IDMC deliberations records, appropriate reporting of results. |

FDA:

Prior to conducting: Rationale, detailed description of adaptation plan (number/timing of analyses, modifications, rule), roles of bodies (DMC/adaptation committee), prespecification of statistical methods (interim/final, software, code), evaluation/discussion of operating characteristics (Type I error, power, sample size, bias, coverage - analytical or simulation). Detailed simulation report if primary technique (trial description, examples, parameter configurations, iterations, results, code, summary). Comprehensive data access plan (who performs, who has access, control, decisions, dissemination, compliance evaluation, meeting minutes). Can be in protocol or separate documents.
Evaluating and Reporting a Completed Trial: All prospective plans, committee charters, supporting documentation, compliance with planned rule and data access, deliberation records, interim analysis results, appropriate reporting in package insert (design, adjusted estimates/cautions). |

| 7. Regulatory Interactions |

ICH E20:

States that the draft guideline is currently under public consultation and does not yet confirm full regulatory acceptance or supersede current regional guidance. The final guideline will indicate key adaptive design principles and approaches for which discussion at the planning stage is particularly critical. Specific feedback is sought on principles and their impact on industry-regulatory interactions to inform finalization. Sponsors should discuss potential implications of adaptations on trial conduct with regulators at the planning stage. Documentation prior to trial initiation should align with national and regional regulatory requirements.

FDA:

Encourages sponsors to explore design options and discuss them with FDA at regulatory meetings (e.g., End-of-Phase-2, Type C). Increased complexity of adaptive trials may warrant earlier and more extensive interactions. FDA's role shifts from safety focus in early development to more extensive evaluation of design and analysis in later phases to ensure reliable results. SPAs for complex adaptive designs only if extensive prior discussion with FDA. FDA review considers whether design and analysis satisfy key principles (Type I error control, reliable estimation, integrity, prespecification). FDA generally not involved in prospectively planned adaptive decision-making; this is sponsor's responsibility, often via a committee (e.g., DMC). Meeting minutes of closed sessions and comparative interim results generally kept confidential until trial conclusion, except for safety risks. All important documentation should be submitted to FDA during design stage for feedback prior to initiation. |

| 8. Definition of Adaptive Design |

ICH E20:

"a clinical trial design that allows for prospectively planned modifications to one or more aspects of the trial based on interim analysis of accumulating data from participants in the trial." |

FDA:

"a clinical trial design that allows for prospectively planned modifications to one or more aspects of the design based on accumulating data from subjects in the trial."

| 9. Definition of Interim Analysis |

ICH E20:

Not explicitly defined as broadly as FDA. Focuses on "sequential analyses of accumulating unblinded data with anticipated rules for stopping".

FDA:

"any examination of data obtained from subjects in a trial while that trial is ongoing and is not restricted to cases in which there are formal between-group comparisons." Includes baseline, safety, PK/PD, biomarker, or efficacy data. Broader than ICH E9 definition.

| 10. "Prospectively Planned" |

ICH E20:

Means "the potential trial adaptations are pre-specified in the clinical trial protocol prior to initiation of the trial."

FDA:

Means "that the adaptation is planned and details specified before any comparative analyses of accumulating trial data are conducted. In nearly all situations, potential adaptive design modifications should be planned and described in the clinical trial protocol (and in a separate statistical analysis plan) prior to initiation of the trial."

Key Differences and Nuances:

Stage of Guidance: The most significant difference is that ICH E20 is a draft guideline currently under public consultation , while the FDA guidance is a finalized document from November 2019. ICH E20 explicitly states it does not supersede current regional guidance until finalized.
Emphasis on "Confirmatory Trials": Both documents primarily focus on adaptive designs in confirmatory clinical trials. However, ICH E20 consistently uses the phrase "confirmatory clinical trials" throughout its document, defining its scope around them. The FDA guidance states its "primary focus...is on adaptive designs for clinical trials intended to support the effectiveness and safety of drugs," but also acknowledges the usefulness of concepts for early-phase/exploratory trials and post-marketing commitments.
Definition of Interim Analysis: The FDA guidance provides a broader definition of "interim analysis" compared to the ICH E9 definition, stating it's "any examination of data obtained from subjects in a trial while that trial is ongoing and is not restricted to cases in which there are formal between-group comparisons". ICH E20 uses the term more in the context of "sequential analyses of accumulating unblinded data with anticipated rules for stopping".
Handling Unplanned Changes: The FDA guidance explicitly addresses "Unplanned Design Changes Based on Comparative Interim Results," strongly discouraging them without prior FDA discussion due to the difficulty in controlling Type I error. It also discusses "Design Changes Based on Information From a Source External to the Trial," which are generally considered acceptable as they don't affect statistical inference validity, but notes the challenge in ensuring the decision was entirely based on external information. ICH E20 directly states its scope "does not include trials with unplanned modifications to the design" or "design changes based entirely on emerging information from a source external to the trial". While both recognize the issue, the FDA guidance delves more into how these situations might be handled if they arise.
Level of Detail on Specific Adaptations: Both documents cover similar types of adaptations (early stopping, sample size, population, treatment arm, patient allocation). However, ICH E20 includes a dedicated section on "Adaptive Designs in Time-to-Event Settings" and "Adaptive Designs Using Bayesian Methods" as full sections within its "Special Topics and Considerations," suggesting a more structured and perhaps deeper dive into these areas. The FDA also covers these, but they are subsections.
Trial Integrity and Confidentiality: Both documents strongly emphasize maintaining trial integrity and limiting access to unblinded interim results to independent parties (e.g., IDMC). However, ICH E20 suggests that details of the adaptation rule "could be reserved for a specific document rather than the protocol, such as a confidential appendix to the IDMC charter" , and that sponsor personnel, investigators, and participants could be shielded from knowledge of specific adaptive changes. The FDA also mentions that the protocol could outline only the general approach with details reserved for documents like the DMC charter.
Documentation: Both guidances provide extensive lists of documentation requirements before and after conducting an adaptive trial. ICH E20's section 6.1 and 6.2 provide a comprehensive outline of what should be documented, including explicit links between clinical and statistical assumptions and simulation results. The FDA also requires detailed simulation reports and data access plans.
Context of Development Program: ICH E20 places a strong emphasis on the "Adequacy Within the Development Program" , suggesting that increasing the number and complexity of adaptations at the confirmatory stage should generally be limited and not replace a sequence of multiple trials or a proper dose-ranging trial. This principle is woven into the justification of adaptive designs. The FDA also touches upon this by noting that adaptive designs are useful for early-phase exploratory trials.

In summary, while both the ICH E20 draft guidelines and the FDA guidance share a common understanding of adaptive designs and their core principles, the ICH E20 document, being a newer draft, appears to offer a more structured and perhaps slightly more detailed approach to certain advanced topics and their implications within the overall drug development program. The FDA guidance provides a solid foundation of principles that aligns with many aspects of the ICH E20 draft, particularly regarding the need for robust statistical methods, trial integrity, and comprehensive documentation.

Low-Dose Active-Control Trials in Psychedelic and CNS Drug Development

2025-06-26T23:30:00.001-04:00

This week, Compass Pathways plc (Nasdaq: CMPS) announced the successful achievement of the primary endpoint in the ongoing Phase 3 COMP005 trial, the first of two Phase 3 trials evaluating COMP360, a synthetic, proprietary formulation of psilocybin, for treatment-resistant depression (TRD). Psilocybin is a psychedelic drug. The field of psychedelic drug development is experiencing a renaissance, driven by growing interest in their potential to treat mental health disorders including major depression disorder, Post-Traumatic Stress Disorder (PTSD), and advancements in research methodologies. It is interesting to see that the phase 3 study conducted by Compass Pathways plc utilized the clinical trial design with low dose or sub-perceptual dose of the active drug as the control group (versus the traditional placebo as the control group).

Challenges in Clinical Trial Design for Psychedelic Drug Development

According to FDA guidance "Psychedelic Drugs:Considerations forClinical Investigations", the followings are key challenges for psychedelic drug clincial trials.

Functional Unblinding and Expectancy Bias: Psychedelic drugs cause intense perceptual disturbances and alterations in consciousness, making it difficult to maintain blinding in studies. Subjects receiving the active drug often experience "functional unblinding." Additionally, high expectations from participants receiving the active drug, or disappointment (nocebo effect) in those receiving a placebo, can influence outcomes.
Control Conditions: Using a traditional placebo can be problematic for assessing efficacy due to functional unblinding and the potential for a nocebo effect. While inactive controls help contextualize safety, alternatives like subperceptual doses or other psychoactive drugs are being considered, but they also present their own challenges.
Contribution of Psychotherapy: Many psychedelic drug development programs incorporate psychological support or psychotherapy. This adds a variable that complicates the assessment of the drug's effectiveness and presents challenges for future product labeling. The specific contribution of the psychotherapy component to any observed efficacy is not yet characterized and has the potential to increase expectancy and performance biases.
Dose-Response Relationship: The dose-response relationship for most psychedelic drugs is not well understood for both efficacy and safety.
Durability of Treatment Effect and Repeat Dosing: While current programs explore single or intermittent dosing, many conditions being studied are chronic. Characterizing the durability of response, recommended interdose intervals, and the safety and efficacy of repeat dosing are significant challenges.
Safety Monitoring During Sessions: Subjects remain in a vulnerable state for up to 12 hours after receiving active treatment. This necessitates specific safety monitoring measures, including observation by two monitors, with a lead monitor who is a healthcare provider with graduate-level training and clinical experience in psychotherapy and is licensed to practice independently.
Abuse Potential Assessment: Psychedelic drugs act on the central nervous system and produce psychoactive effects, requiring evaluation for abuse potential. This includes monitoring abuse-related adverse events, even if they are hypothesized to be associated with a therapeutic response.

Reagan-Udall Foundation for the FDA organized a workshop "Advancing Psychedelic Clinical Study Design Design", the challenges in psychedelic clinical trial design were extensively discussed:

Blinding & Expectancy: It's difficult to blind studies due to the profound psychoactive effects, leading to "functional unblinding" where participants and therapists often know who received the active drug. High expectations from media coverage can also inflate outcomes
Control Conditions: Traditional placebos can be problematic, leading to unblinding and nocebo effects. Active comparators are an option, but finding one with similar subjective effects without being therapeutic is challenging. Sub-perceptual doses are also being explored, but they might still have psychoactive effects.
Role of Psychotherapy/Facilitation: The therapeutic intervention (preparation, in-session monitoring, integration) significantly influences outcomes, and its contribution needs to be distinguished from the drug's effect
Dose-Response Relationship: Understanding the dose-response relationship for safety and efficacy is crucial, and dose-response trials are considered valuable
Durability of Treatment Effect: A major question is how long the effects last and the need for redosing, especially for chronic conditions

One of the approaches to address the blinding issue or functional unblinding issue is to use the low dose (or sub-perceptual dose, sub-optimal dose) of the active drug (in this example, the psychedelic drug psilocybin).

Low-dose active control is used beyond the psychedelic drug clinical trials. It is also used in other CNS drug clinical trials. Some of the trials with low-dose active control lead to the drug approval by the FDA. Here are examples of Low-Dose Active-Control Trials in CNS Drug Approvals

Lamotrigine XR (Lamictal XR) – Indication: Conversion to monotherapy for partial-onset seizures in patients ≥13 years. Trial: Study LAM30055 (multicenter conversion-to-monotherapy). Sponsor: GlaxoSmithKline. FDA Approval: April 2011. Reason for design: In epilepsy monotherapy trials, giving a true placebo is unethical. The FDA accepted a “pseudoplacebo” control (historical data from low‐dose valproate) to demonstrate lamotrigine’s efficacy This low-dose VPA arm served as an inferior active control so that a full placebo arm could be avoided while still assessing drug effect.
Oxcarbazepine (Trileptal) – Indication: Monotherapy for partial-onset seizures. Trial: Two Phase 3 trials comparing high-dose Trileptal (2400 mg/day) to a subtherapeutic Trileptal dose (300 mg/day). Sponsor: Novartis. FDA Approval: mid-2000s (first-line monotherapy around 2005). Reason: Again, placebo was deemed unethical in active seizure patients. The control arm received only 300 mg (a suboptimal dose) of oxcarbazepine. This “low-dose active” control (essentially a pseudo-placebo) allowed demonstration of efficacy (fewer seizures/dropouts in the high-dose arm) without exposing patients to no treatment.
Pregabalin (Lyrica) – Indication: Monotherapy for partial-onset seizures. Trial: Randomized conversion-to-monotherapy trial of pregabalin 600 mg/day vs pregabalin 150 mg/daypmc.ncbi.nlm.nih.gov (high-dose vs low-dose active control). Sponsor: Pfizer. FDA Approval: ~2017 (supplement to extend indication). Reason: As with other AEDs, full placebo was not used. The trial randomized patients to a high dose or a low “pseudo-placebo” dose (150 mg) of pregabalin. Using a subtherapeutic active dose allowed assessment of efficacy (few patients worsened on 600 mg) while avoiding a true placebo arm. Regulatory briefing notes explain that monotherapy trials in epilepsy often use a “pseudo-placebo” (suboptimal dose of drug) for exactly this ethical reason.

Rationale for Low-Dose Active Controls

These examples share a common rationale: in serious conditions like major depression disorder, PTSD, and epilepsy, withholding effective therapy (i.e. using true placebo) is ethically problematic. Using true placebo results in the functional unblinding. Regulatory authorities have allowed a “low-dose active” or “pseudo-placebo” comparator to ensure trial blinding and assay sensitivity while not denying therapy In practice, patients in the control group receive a sub-therapeutic dose of an anticonvulsant (e.g. low-dose valproate or pregabalin) instead of no drug. This strategy maintains the appearance of treatment, minimizes risk of uncontrolled seizures, and still permits demonstration of dose–response or superiority of the higher dose. As noted in FDA/clinical guidance and literature, this design avoids exposing patients to a completely inert treatment in life‐threatening disorders. In all cases above, the low-dose arm (active control) was chosen for ethical reasons (protecting patients) and to meet regulatory requirements for demonstrating efficacy without a true placebo.

Managing Pregnancies in Clinical Trials: Regulatory Guidance and Best Practices

2025-06-14T22:43:00.001-04:00

Clinical research historically has excluded pregnant women, creating major data gaps. Today, less than 1% of trial participants are pregnant, and most approved drugs lack pregnancy safety data. There are various reasons pregnant women are often excluded from clinical trials:

Scientific and medical concerns: potential fetal harm, unknown pharmacokinetics and dosing, Complex mother-fetus risks, and impact on maternal health.
Religious or cultural considerations: Emphasis on fetal protection, Paternalistic IRB attitudes, Participation and social norms.
Liability and legal concerns: Fear of fetal-harm lawsuits, Insurance and compensation, Regulatory ambiguity, Cascading regulatory costs

Regulators such as US FDA now emphasize a balanced approach – gathering information on drug use in pregnancy by allowing inclusion when appropriate, while protecting fetal safetyf FDA guidance and recent legislation stress that trials should broadly reflect real-world populations, with any exclusions (like pregnancy) justified by safety and scientific rationale. In fact, pregnancy is explicitly identified as a historically underrepresented group in FDA diversity initiatives. This shift in thinking – codified by laws like FDARA and FDORA and by draft FDA guidances – is intended to improve evidence on treating pregnant patients.

Regulatory Context and Diversity Guidance

FDA regulations and guidances set the framework for handling pregnancy in trials. The FDA’s 21 CFR Part 50 Subpart B (pregnant women, fetuses, and neonates) and 21 CFR Part 56 (IRB requirements) require heightened protections for pregnant participants. For example, FDA recommends that if a study offers benefit solely to the fetus, both the pregnant woman and her partner must consent (with narrow exceptions). Crucially, FDA guidance mandates that “each individual providing consent is fully informed regarding the reasonably foreseeable impact of the research on the fetus or neonate”. In practice, this means consent forms and discussions must spell out known pregnancy risks (and uncertainties) based on available animal or prior data. IRBs reviewing these trials must include members experienced with maternal-fetal medicine and ensure extra safeguards (per 21 CFR 56.111(a)(2) and (b)).

At a higher level, FDA’s draft guidance on pregnant women (2018) explicitly endorses “judicious inclusion of pregnant women in clinical trials” to inform safe drug use in pregnancy. Similarly, the 2020 FDA guidance on broadening trial diversity urges sponsors to continuously “broaden eligibility criteria” as safety data accrues, narrowing unnecessary exclusions. These documents reinforce that excluding pregnant women should not be automatic: instead, inclusion should be considered whenever the potential benefits outweigh risks. In summary, the regulatory message is clear: plan trials for diversity (as now required by law with Diversity Action Plans) and, where scientifically justified, include or at least carefully manage pregnant participants.

Informed Consent and Pre-Screening Procedures

Before enrollment, trials must address pregnancy risk. Protocols almost universally exclude pregnant or breastfeeding women, and require women of childbearing potential (WOCBP) to use reliable contraception and have negative pregnancy tests at screeningonbiostatistics.blogspot.com. Per longstanding FDA advice, informed consent must cover pregnancy issues explicitly: if nonclinical reproduction studies are lacking, sponsors must “fully inform” women and advise on contraceptive measuresonbiostatistics.blogspot.com. In practice, consent forms and discussions should include pregnancy-specific language and summarize what is (and isn’t) known about fetal risk.

Contraception and Testing: Investigators should ensure WOCBP agree to use effective birth control (including abstinence) and take a pregnancy test before each dosing period. FDA guidance has long recommended pregnancy tests at screening and counseling on contraceptiononbiostatistics.blogspot.com. Some trials repeat tests periodically to catch early pregnanciesonbiostatistics.blogspot.com.
Risk Disclosure: Consent discussions must describe potential risks to a fetus. The FDA draft guidance stresses that subjects be informed about “reasonably foreseeable impact” of trial participation on the fetus or neonate. Even if evidence is limited, the consent should transparently explain any known animal or human data, and state unknowns. This empowers participants to make decisions with full awareness of pregnancy-related risks.

Managing Pregnancies Discovered During Trials

Despite precautions, some participants will become pregnant during a study. Best practices focus on timely detection, notification, and safety:

Detection: Continue pregnancy testing throughout the study (e.g. at regular visits or follow-ups). One biostatistics review notes that trials “usually perform pregnancy tests periodically” so pregnancies can be caught earlyonbiostatistics.blogspot.com.
Immediate Actions: If a pregnancy is identified, the standard practice is to stop the study drug to avoid further exposureonbiostatistics.blogspot.com. The investigator should promptly notify the sponsor and IRB, and arrange any needed medical follow-up. FDA guidance goes further: it recommends unblinding the subject’s treatment (drug vs placebo) so the woman and her physician can discuss risks based on actual exposure. The patient should then undergo a second informed-consent process that incorporates the new risk-benefit assessment. For example, if the drug may benefit the mother, continuation might be allowed if potential benefits outweigh fetal risks (with the mother’s informed agreement). Otherwise, she should be withdrawn from treatment.
Data and Follow-Up: Regardless of continuation, sponsors must collect detailed follow-up data. FDA guidance explicitly states that “regardless of whether the woman continues in the trial, it is important to collect and report the pregnancy outcome”. In practice, this means recording gestational age at drug exposure, duration of exposure, and all outcomes (live birth, gestational age at delivery, congenital events, etc.). The pregnant participant should receive standard obstetrical care and fetal monitoring alongside the study’s safety assessments. For example, cord blood or neonatal samples might be collected for drug levels if relevant.
Discontinuation and Missing Data: If the participant discontinues the study, subsequent efficacy visits are often ceased; statisticians typically handle missing data using predefined methods. Notably, many sponsors maintain a pregnancy registry or report form for study pregnanciesonbiostatistics.blogspot.com. These registers feed into post-market surveillance (and often meet regulatory reporting requirements). Any adverse fetal outcome (miscarriage, birth defect, etc.) must be reported as a serious adverse event per 21 CFR 312.32/64.
Monitoring: Trials with pregnant subjects or exposures require specialized monitoring. For example, the protocol should specify involvement of obstetric/maternal-fetal experts on the safety monitoring team. Ongoing review of maternal and fetal safety signals is essential – a dedicated data monitoring committee may be warranted. In extreme cases (e.g. clear evidence of harm), the trial’s stopping criteria may trigger halting enrollment.

Reporting and IRB Notification

Pregnancy events are subject to regulatory and ethics oversight. Investigators should report any pregnancy to the IRB as soon as it is identified (often as an “unanticipated problem” in light of initial exclusion criteria). FDA guidance notes that IRBs reviewing such protocols must have the right expertise and must ensure “additional safeguards” are in place for pregnant subjects. For example, IRBs should verify that consent materials address pregnancy, that medical backup (e.g. obstetric care) is arranged, and that procedures (e.g. prohibition of termination inducements) are followed.

Sponsors, in turn, must follow safety-reporting rules. Under FDA IND regulations, any pregnancy with drug exposure is reported to FDA (particularly if it results in a serious fetal or neonatal outcome). The IRB and health authorities should be kept informed according to institutional policies. In practice, many trials use a structured form or registry entry to document trial pregnancies and outcomesonbiostatistics.blogspot.com. This ensures timely communication of safety information to all stakeholders.

Ethical Considerations of Inclusion vs. Exclusion

The exclusion of pregnant women raises ethical issues. Traditionally, fear of fetal harm led to a protective approach, but contemporary ethicists argue that systematic exclusion often causes more harm. As one commentary notes, refusing to study drugs in pregnancy “merely shifts risk to the clinical context” – doctors and patients then must decide on therapies with no evidence, which is “hardly an ethical approach”. High-profile cases (e.g. thalidomide) illustrate the dangers of not studying drugs in pregnancy. In fact, a commissioned analysis found no liability cases from including pregnant women in trials, whereas many lawsuits have arisen from unforeseen drug harms in pregnant patients after approval.

Conversely, including pregnant participants – with proper precautions – yields direct benefits. It allows rigorous collection of safety/efficacy data in a controlled setting, reducing uncertainty. Experts stress it is a humane and scientific responsibility to prioritize pregnant women’s inclusion when possible. Denying them evidence essentially “denies pregnant women—and their healthcare providers—the evidence necessary to make informed decisions”. Modern ethical guidance and FDA policy now urge balancing fetal protection with the pregnant woman’s health needs, rather than default exclusion.

Recommendations and Future Directions

Given this landscape, sponsors and investigators can follow several best practices:

Education and Consent: Develop clear, patient-friendly consent materials that discuss pregnancy risks and emphasize the importance of contraception for WOCBP. Train staff to discuss these issues openly.
Safety Monitoring: Include pregnancy in safety monitoring plans. Engage obstetric/maternal-fetal medicine consultants in trial planning and oversight.
Trial Design: Where feasible, design trials to allow pregnant participants (or planned pregnancy cohorts) for conditions that affect women of childbearing age. FDA’s 2018 draft guidance and recent NIH recommendations encourage such trials for pregnancy-specific or relevant indications.
Community Engagement: Build partnerships with obstetric care providers and clinics. Embedding trial activities in OB settings can greatly improve recruitment and trust among pregnant patients.
Regulatory Planning: Anticipate the need for pregnancy considerations in regulatory submissions. Under the Pregnancy and Lactation Labeling Rule (PLLR), FDA expects clear labeling of pregnancy data (or lack thereof). Sponsors should be prepared to update labels as new pregnancy data emerge.
Continued Advocacy: Engage with regulatory agencies. FDA’s task forces and guidance initiatives (e.g. on diversity action plans) reflect ongoing shifts. Industry input can help shape policies that balance scientific goals with patient safety.

In summary, managing trial pregnancies requires a structured approach: robust informed consent, vigilant monitoring, regulatory reporting, and ethical reflection. With these measures, sponsors can protect participants while generating the much-needed data on drug safety in pregnancy. As one expert put it, including pregnant women in research is no longer a question of if but when and how – given the broad benefits of more equitable, evidence-based care.

References:

FDA guidance (2018) "Pregnant Women: Scientific and Ethical Considerations for Inclusion in Clinical Trials"
FDA guidance (2020) "Enhancing the Diversity of Clinical Trial Populations-Eligibility Criteria, Enrollment Practices, and Trial Designs". This guidance was subsequently removed, but the talking points are still there.
The FDA law blog (2025) "Clinical Research Representation: Pass it On"
Onbiostatistics.blogspot.com (2017) "Exclusion of pregnant women from clinical trial and pregnancy test for women of childbearing potential (WOCBP)"
Life Science Leader (2025) "Pregnant Women: An Underrepresented Group In Clinical Trials"
Applied Clinical Trials (2025) "Pregnant Women: An Underrepresented Group in Clinical Trials"

Composite Strategy for Intercurrent Event and the Use of Trimmed Means in Clinical Trial Data Analyses

2025-06-08T23:30:00.039-04:00

When composite strategy is used to handle the intercurrent event (ICE), specially the terminal event such as death, the occurrence of the ICE is integrated into the endpoint definition, often by assigning a specific value to participants who experience the event.

In ICH E9-R1 "Addendum on Estimands and Sensitivity Analysis in Clinical Trials" training material, about the composite strategy to handle the intercurrent event, trimmed mean is mentioned to be an approach in handling the intercurrent event.

A trimmed mean may also be called truncated mean and is the arithmetic mean of data values after a certain number or proportion of the highest and/or lowest data values have been discarded. The data values to be discarded can be one-sided or two-sided. A trimmed mean can be defined as a robust average computed by discarding a specified fraction of the lowest and highest observations and averaging the remainder. In effect, it “trims” the tails of the data, reducing the influence of outliers. For example, a 50% trimmed mean discards the bottom 25% and top 25% of values, averaging the middle 50% In general, an alpha‑trimmed mean removes the lowest and highest alpha/2 fraction of data (where alpha is expressed as a percentage of the total).

After trimming, the mean of the remaining values is computed by the usual arithmetic formula. In practical use, common trims range from 5–20% per tail (e.g. alpha=10%-40% total) in robust estimation, though some clinical examples have trimmed up to 50%. By removing extreme observations, trimmed means downweight outliers and model the assumption that missing or dropout outcomes are worse than any observed values.

Advantages of Trimmed Means in Handling Outliers and Skewed Data

Trimmed means are recognized as robust estimators of central tendency, demonstrating less sensitivity to deviations from assumed models or distributions, such as the presence of outliers or non-normality, when compared to classical methods like the sample mean. This robustness translates into more stable and reliable results under challenging data conditions.

For asymmetric distributions, where variability is more pronounced on one side, trimmed means can provide a superior estimation of the location of the main body of observations. By removing extreme values, they offer a more robust estimate of the central value and are less influenced by skewed data distributions. Furthermore, the standard error of the trimmed mean is less susceptible to the effects of outliers and asymmetry than that of the traditional mean, which can lead to increased statistical power for tests employing trimmed means.

The advantages of trimmed means extend beyond mere statistical robustness; they enable a more clinically meaningful interpretation of treatment effects, particularly in heterogeneous patient populations or when extreme outcomes (e.g., severe adverse events, rapid disease progression) might otherwise obscure the true effect in the majority of patients. This aligns with the need for statistical methods that accurately reflect real-world clinical practice and patient experience. If extreme values arise from factors not directly related to the treatment's intended effect on the typical patient—such as rare severe adverse events or non-adherence driven by external circumstances—then their removal allows for a clearer assessment of the treatment's impact on the majority. Conversely, if extreme values are an inherent part of the treatment effect, such as severe lack of efficacy leading to patient dropout, the

trimmed mean can define an estimand for the subpopulation that did not experience these extreme negative outcomes. Both scenarios offer a more focused and potentially more interpretable clinical picture.

It is important to acknowledge the inherent trade-off between robustness and efficiency: more robust methods, including trimmed means, may sacrifice some efficiency (precision or variability) compared to optimal methods under ideal statistical assumptions. The selection of a method ultimately depends on the nature of the data and the specific goals of the analysis.

Comparison with Traditional Measures of Central Tendency (Mean, Median)

The choice among the mean, median, and trimmed mean is not merely a statistical decision but reflects a fundamental determination about the target estimand and the specific clinical question being addressed.

● Mean: The traditional arithmetic mean calculates the average of all values in a dataset. It is highly sensitive to extreme values, which can significantly distort the measure of central tendency and lead to a less representative average, especially in the presence of outliers or skewed distributions.

● Median: The median represents the middle value in an ordered dataset and is highly resistant to the influence of extreme values.3 Conceptually, the median can be viewed as an extreme form of a trimmed mean, where all but one or two central observations are effectively removed. While both the trimmed mean and the median reduce the impact of outliers, the median is generally considered more robust in certain contexts due to its reliance solely on rank order.

The trimmed mean strikes a balance between these two traditional measures. It retains more information from the dataset than the median, which discards a significant portion of the data, while simultaneously offering greater robustness to outliers than the arithmetic mean.3 This allows for a nuanced definition of "average effect" that acknowledges the presence of extreme outcomes without being unduly influenced by them (like the raw mean) or implicitly discarding them entirely (like the median). This highlights the importance of defining the estimand before selecting the statistical method, a principle strongly emphasized by ICH E9(R1).

Regulatory Context and Trial Examples

Trimmed means have been discussed in statistical literature and regulatory settings as a way to handle dropout or intercurrent events. Permutt and Li (FDA biostatisticians) first proposed using trimmed means for “symptom trials” with dropout, treating each dropout as a complete (nonnumeric) observation ranked as the worst outcome. Under this approach, all subjects who discontinue before the endpoint are implicitly assigned the worst possible values and then an equal percentage are trimmed from each arm. In effect, trimming favors treatments with fewer dropouts, since “having more completers is a beneficial effect of the drug”.

Notably, trimmed means correspond to a composite estimand in the ICH E9(R1) framework: one can assign intercurrent events (e.g. dropouts) a worst-case value and then summarize the outcome distribution by a median or trimmed mean. For instance, the ICH E9 addendum training materials explicitly cite trimmed means (alongside medians) as summary measures under a composite strategy when dropouts are scored as extreme unfavourable outcomes. A recent FDA review of glaucoma drug Rocklatan (netarsudil/latanoprost) illustrates this: the statistical reviewer computed an “adaptive trimmed mean” IOP reduction by coding patients who withdrew for lack of efficacy or adverse events as worst outcomes. The reviewer noted this analysis aligns with the composite strategy in ICH E9(R1) and can reinforce the main intent-to-treat result.

However, we found no FDA-approved trial in which a trimmed mean was the pre‑specified primary endpoint analysis. In all examples, trimmed means have been used as sensitivity or supportive analyses rather than the main test. For example, in a uterine fibroid drug application (NDA 214846), a trimmed‐mean analysis of menstrual blood loss was reported: the trimmed mean in each arm used the 50% best-performing patients, reflecting a 50% trim. Similarly, an ophthalmology (geographic atrophy) trial review (NDA 217171) applied trimmed‐mean + multiple imputation as a sensitivity: certain dropouts were “excluded (trimmed)” in the reviewer’s analysis. In each case, the trimmed mean analysis “assumes missing data as ‘bad outcomes’” (i.e. MNAR). These examples underscore that trimmed means appear in regulatory documents as robust or conservative analyses (often labeled “completer” or MNAR analyses) rather than as the primary efficacy metric.

Key examples:

Glaucoma (Rocklatan NDA 208259): Reviewer performed an adaptive trimmed mean IOP analysis, excluding patients who withdrew for lack of effect and assigning worst values (citing Permutt & Li as method).
Uterine fibroids (NDA 214846): Primary efficacy (menstrual blood loss) was also examined by trimmed means (50% trim) as sensitivity. The FDA report notes the trimmed‐mean is based on the “top 50% best performers in each arm”.
Geographic atrophy (NDA 217171): A trimmed-mean + MI analysis was done by the reviewer; dropouts due to adverse events or lack of efficacy were “excluded (trimmed)” from one scenario.

These applications align with recent methodological studies showing trimmed means estimate a unique estimand: essentially, the mean outcome in the subpopulation of patients who would have remained on treatment. This estimand privileges treatments that prevent dropout, but it relies on the strong assumption that every dropout truly has an “unfavourable” outcome. As Wang et al. note, “the trimmed mean estimates a unique estimand” and its validity “hinges on the reasonableness of its assumptions: dropout is an equally bad outcome in all patients”. Ocampo et al. similarly emphasize that trimmed means work well when discontinuation is strongly associated with poor outcome, but give biased estimates if data are actually missing at random.

Calculation and Assumptions

Formula: To calculate an alpha‑trimmed mean, one typically sorts the data and discards a proportion alpha/2 from each end. For example, the 50% trimmed mean drops the lowest 25% and highest 25% of values, averaging the middle 50%. In general, if alpha (0–1) is the total fraction trimmed, remove the lowest alpha/2 and highest alpha/2 observations, then compute the mean of the remaining values. (The interquartile mean is the special case alpha=0.5.) Typical choices of alpha are guided by robustness needs: small trims (e.g. alpha=0.1 for 5% each tail) mildly reduce outlier influence, while large trims (up to 0.5) exclude half the data.

Statistical assumptions: Trimmed‐mean analyses assume that any trimmed/missing values are in fact the worst outcomes. In the dropout context, this treats early withdrawal as if the patient’s true result were extremely poor. As Ocampo et al. explain, the trimmed mean approach “sets missing values as the worst observed outcome and then trims away a fraction of the distribution”. Under this MNAR assumption, the trimmed mean can provide an unbiased estimate of the treatment effect (on the subpopulation that remains). However, if outcomes are actually missing at random (MAR) without relation to extreme values, trimmed means will be biased. In simulations, trimmed means were found to fail under MAR (because the assumption of trimming “bad” outcomes is then invalid).

Interpretation: Because the trimmed mean ignores equal fractions from each arm, it essentially compares the upper quantiles of the outcome distribution. Clinically, it reflects the mean of the best-performing subset of patients. This is why Permutt and Li emphasize that their method makes no assumptions beyond randomization: it is a nonparametric “exact” test for efficacy that includes all randomized subjects (by ranking dropouts worst). Regulators caution that the trimmed-mean estimand differs from a standard ITT mean; it answers the question, “What is the mean outcome among patients who would have remained in the trial?”.

SAS Implementation Example

Trimmed means can be computed in SAS using procedures like PROC UNIVARIATE or PROC MEANS. For instance, PROC UNIVARIATE supports a trimmed= option. The snippet below computes a 10% trimmed mean (i.e. removes 10% from each tail) of a variable Y and captures the result via ODS:


/* Example: Compute a 20% trimmed mean (10% each tail) for outcome Y */
ods output TrimmedMeans=TrimmedStats;
proc univariate data=trial_data trimmed=0.10;
   var Y;
run;

proc print data=TrimmedStats noobs; 
   title "Trimmed Mean of Y";
run;

Alternatively, PROC MEANS (SAS 9.4+) also supports trimmed means. For example:

/* Using PROC MEANS with OUTPUT to get trimmed mean */
proc means data=trial_data noprint trimmed=0.10;
   var Y;
   output out=Stats (drop=_TYPE_ _FREQ_) trimmed=Y_trimmed;
run;

proc print data=Stats noobs; 
   title "Trimmed Mean of Y via PROC MEANS";
run;

These code examples illustrate that one can easily obtain trimmed‐mean estimates in SAS by specifying the trimmed percentage (e.g. 0.10 for 10%) and directing the procedure output to a dataset for further use. (In practice, one would adjust trimmed= according to the planned trim proportion.)

Using Google Gemini, a comprehensive report on using trimmed means in clinical trials was generated and can be accessed here.

Baby KJ's Story - a remarkable success of the customized CRISPR gene editing treatment

2025-06-05T17:30:00.024-04:00

This morning, FDA conducted a roundtable discussion on cell and gene therapies. The roundtable invited 23 panel members from the academic and industry and was attended by FDA commissionner Dr Marty Makary and FDA CBER director Dr. Vinay Prasad. At the second half of the roundtable discussion, HHS secretary, RF Kennedy Jr, NIH director, Dr Jay Bhattacharya, and CMS Administrator, Dr Mehmet Oz all gave commentary speeches. The overall tone of the roundtable is very positive and the government agencies are very supportive to the cell and gene therapy (including xenotransplantation) development. The panel suggests various ways to support the cell & gene therapies. Renewing the pediatric priority review voucher program, Reduction of CMC requirements, continued support of flexible trial design and accelerated approval, and more reliance on post approval requirements and real-world evidence (RWE) were highlighted as key tools for the agency to reduce the number of programs caught in the

clinical “valley of death”. See the articles reported by FierceBiotech "'Not a horse and pony show': at FDA cell and gene roundtable, officials look for ways to cut red tape". and by BioSpace "RFK Jr. Joins Deputies in Voicing Support for Regulatory Flexibility for Cell and Gene Therapies".

During the roundtable discussions, the baby KJ's case was mentioned multiple times. It is worth discussing the baby KJ's story.

KJ Muldoon is a 10-month-old infant who became the first person in the world to receive a personalized CRISPR-based gene-editing therapy, marking a significant milestone in the treatment of rare genetic diseases.

Diagnosis and Condition

Shortly after birth, KJ was diagnosed with severe carbamoyl phosphate synthetase 1 (CPS1) deficiency, a rare and life-threatening genetic disorder that impairs the body's ability to eliminate ammonia. This condition can lead to toxic ammonia buildup, causing severe neurological damage or death in infancy.

Development of Personalized Therapy

With conventional treatments offering limited hope, a multidisciplinary team from the Children's Hospital of Philadelphia (CHOP) and Penn Medicine embarked on an unprecedented effort to develop a customized gene-editing therapy tailored specifically to KJ's unique genetic mutations. Within six months, researchers identified two truncating variants in the CPS1 gene and designed a bespoke CRISPR-based therapy using base editing technology. This approach allowed precise correction of the faulty DNA without cutting the genetic code.

Treatment and Outcome

KJ received multiple infusions of the experimental therapy, delivered directly to his liver using lipid nanoparticles carrying the gene-editing components. The treatment aimed to correct the genetic defect responsible for his condition. Following the therapy, KJ showed significant improvement, including better tolerance to dietary protein and reduced dependence on supportive medications. After spending over 300 days in the hospital, he was discharged and returned home with his family.

Significance and Future Implications

KJ's case represents a groundbreaking advancement in personalized medicine and gene-editing therapies. It demonstrates the potential of customized CRISPR treatments to address ultra-rare genetic disorders by rapidly developing individualized therapies. While long-term outcomes and scalability remain areas for further research, this success story offers hope for treating other rare diseases through similar personalized approaches.

For more detailed information, you can watch the FDA roundtable discussion on cell and gene therapies where KJ's case was mentioned: FDA Roundtable Discussion.

Impact of Baby's KJ's Case on Clinical Trial Design of Gene Therapies

Baby KJ's story was extensively discussed in a follow-up podcast "FDA Direct Ep. 7: This Week at the FDA!" by FDA commissioner Dr Makary and CBER director Dr Prasad. Their discussion of Baby KJ's story extended to the clinical trial designs for gene therapy studies and ultra rare diseases.

The FDA's approach to clinical trial designs for gene therapy studies emphasizes flexibility and a nuanced understanding of the specific condition and therapy. Here are the key points:

Flexible Trial Designs: The FDA acknowledges that a "one-size-fits-all" approach is not suitable. They adapt trial designs based on the specific condition, its frequency, severity, and the uniqueness of the therapy.
"N of 1" Trials for Rare Diseases: For extremely rare and dire conditions, a single-patient ("N of 1") trial can be sufficient for approval, especially when there is a plausible mechanism of action and clear biological markers demonstrating effectiveness. An example highlighted is the rapid approval of a custom-tailored gene editing treatment for Baby KJ's rare condition.
Plausible Mechanism Pathway: A strong, scientifically sound mechanism of action can support approval, even without extensive large-scale trials, if it suggests safety and effectiveness through extrapolated data.
Challenges with Slowly Deteriorating Conditions: For conditions with slow and variable deterioration, relying on anecdotal evidence from "N of 1" studies without clear biological correlates is more difficult, as it's harder to distinguish treatment effects from the natural disease progression.
Surrogate Endpoints: The FDA accepts the use of surrogate endpoints, such as tumor shrinkage in cancer, as indicators of a drug's activity.
Industry Desire for Predictability: The industry seeks greater predictability regarding FDA expectations for endpoints, control arms, and when randomization is necessary. Improved communication and transparency are suggested to address this.
Concerns with International Trial Data: There are concerns about relying solely on clinical trial data from certain countries, particularly if the majority of participants are from a single foreign country, for U.S. regulatory decisions.
Re-evaluation of Trial Requirements: The FDA is open to re-evaluating requirements, such as the need for two clinical trials versus one for new drug approvals, to potentially streamline the process.

The overarching theme from recent FDA discussions is a shift towards a more flexible, common-sense, and scientifically driven approach to gene therapy regulation, prioritizing patient needs and scientific plausibility, especially for rare and life-threatening conditions.

References:

Forwarded: "Data Distortions: When Statistics Can Lead Us Astray in Drug Safety"

2025-06-01T12:30:00.001-04:00

The DIA Global Forum's article, "Data Distortions: When Statistics Can Lead Us Astray in Drug Safety," brought attention to a common pitfall: the practice of performing individual hypothesis tests for each adverse event (AE), calculating p-values, and then concluding statistical significance based on an arbitrary threshold like

p < 0.05

. This mirrors a concern I raised earlier in my blog article, "Should hypothesis tests be performed and p-values be provided for safety variables in efficacy evaluation clinical trials?"

It's a widely held statistical consensus that conducting hypothesis tests for individual AEs is unsound, and the p-values derived from them are prone to misinterpretation. Despite arguments that such tests could be performed for "exploratory purposes" with a disclaimer, the inherent risk is that these p-values will inevitably be misused to draw misleading conclusions from the data. Unfortunately, we've seen journal articles, often at the insistence of editors, include p-values for individual AEs, perpetuating this problematic practice.

JSON vs SAS XPT Data Formats in Clinical Trial Data Submissions

2025-05-01T22:06:00.000-04:00

Regulatory study data (e.g. SDTM/SEND tabulations and ADaM analysis datasets) are currently exchanged in SAS XPORT (XPT) format, the legacy transport format used by the FDA. Each dataset is submitted as a separate .xpt file (for example, dm.xpt, adsl.xpt) with an accompanying define.xml to describe metadata. See a previous blog article "Submit the Clinical Trial Datasets to FDA: Using the right .xpt file format" and FDA's "STUDY DATATECHNICAL CONFORMANCE GUIDE Technical Specifications Document".

In April 2025, FDA issued a Federal Register notice "Electronic Study Data Submission; Data Standards; Clinical Data Interchange Standards Consortium Dataset-JavaScript Object Notation;Request for Comments' stating it is exploring CDISC’s Dataset-JSON (v1.1) – a JSON-based schema – as a new exchange standard for study data, with the long-term potential to replace SAS XPT v5. The FDA is requesting public comment on adopting Dataset-JSON for future submissions. This report compares the JSON and XPT formats in the context of clinical data exchange and FDA submissions, covering their overviews, advantages/disadvantages, official regulatory stance, and practical sponsor considerations.

JSON Format (CDISC Dataset-JSON)

JSON (JavaScript Object Notation) is a text-based, human-readable data format widely used in web and health IT. For example, HL7’s FHIR standard commonly uses JSON for healthcare data exchange. CDISC’s Dataset-JSON (v1.1) is a JSON schema specifically designed to represent tabular clinical study data. It is part of the CDISC Operational Data Model (ODM) v2.0 framework and is open-source and machine-readable. By design, each Dataset-JSON dataset can include column values in JSON and can reference a CDISC define.xml document for full metadata, linking data values to variable definitions. This format supports both file-based and API-based exchange of data. In practice, a set of JSON “dataset” files (one per domain) can be packaged with a define.xml or delivered via web services. The format is schema-driven and extensible, meaning it can accommodate richer metadata and longer field names than legacy formats. FDA notes that Dataset-JSON is simple to implement, very stable, and “widely supported” across software platforms. Its use of JSON (Unicode text) makes it easy to parse with standard programming libraries (JavaScript, Python, R, etc.), and it aligns with modern data standards and the FDA’s Data Modernization goals.

SAS XPORT (XPT) Format

The SAS XPORT (XPT) Transport Format v5 is the longstanding standard for FDA study data submission. XPT is a binary file format defined in the 1980s (SAS Technical Report TS-140) that encodes one dataset per file. In FDA submissions, each SDTM or ADaM dataset is delivered as an .xpt file (e.g. dm.xpt for demographics) along with a corresponding define.xml describing its variables. FDA’s guidance and catalogs explicitly support XPT v5: for example, a technical guide lists DM.xpt and ADSL.xpt as required files. The format is natively supported by SAS software (via PROC COPY or LIBNAME XPT) and by some third-party tools, ensuring that sponsors with SAS infrastructures can readily produce and consume it. However, XPT is not human-readable (it is binary) and has inherent limitations: variable names are limited to 8 characters (per the v5 spec) and labels to 40–200 characters, and there is no direct way to embed metadata (hence the separate define.xml). Because XPT v5 is a fixed, legacy format, it cannot represent nested or hierarchical data and requires separate metadata files. Despite these drawbacks, XPT is currently the required FDA exchange format for standardized study data – submissions that do not use FDA-approved formats (listed in the Data Standards Catalog) risk rejection.

Advantages and Disadvantages

JSON advantages: JSON is a modern, widely-used exchange format. Dataset-JSON supports linking to define-XML and can include rich metadata within or alongside the data. It is text-based and open, so it can be parsed by virtually any software (not just SAS), and it naturally integrates with web and API workflows. FDA’s 2022 assessment found that JSON offers “smaller file sizes, additional metadata, and simpler processing” compared to legacy formats. Because it is extensible, JSON removes XPT’s old limitations on field lengths and formats, enabling future evolution of data standards. In the PhUSE pilot, sponsors noted potential for improved efficiency, hardware cost savings, and alignment with digital data ecosystems.
JSON disadvantages: Dataset-JSON is not yet standard for FDA submissions, so adopting it today would require regulatory discussions or waivers. Industry tooling is nascent: sponsors must develop or acquire new processes (for example, SAS can export JSON but may need custom mapping to CDISC JSON schema). The FDA notice explicitly solicits comments on “integration challenges with existing tools and systems,” reflecting concern that current CDMS/SDTM pipelines are geared to XPT. Managing two formats during a transition also adds complexity. Because JSON is text, very large numeric datasets might be bulkier uncompressed (though gzip can mitigate this). Finally, until FDA grants formal acceptance (which would require a new guidance), sponsors using JSON would be taking a risk.
XPT advantages: XPT is a proven, FDA-sanctioned format. All major clinical data tools (especially SAS) can readily produce XPT. Regulatory reviewers and submission systems are already built for it, so sponsors face no surprise validation issues. Using XPT ensures immediate compliance with FDA standards (as affirmed in guidance and the Data Standards Catalog). The process of creating .xpt files is well-understood (e.g. using SAS PROC COPY or EXPORT), and many legacy datasets and analysis programs assume XPT input. XPT’s fixed format and single-table-per-file approach are simple and do not require on-the-fly schema negotiation. Long-term archiving of XPT files is routine (with define.xml), so sponsors have established practices for retainment.
XPT disadvantages: XPT is technologically outdated. Its fixed schema (8-char names, etc.) and binary nature limit flexibility. It cannot easily accommodate new metadata or complex data types. Interoperability outside the SAS world is limited (one must use conversion tools). The format does not support streaming or API-based exchange, only static files. Because define.xml is separate, there is a risk of mismatches between data and metadata if not carefully managed. From an innovation standpoint, XPT is a single-version format (v5) with no path for evolving, so it is not aligned with modern data architectures (e.g. FHIR or big-data standards). Sponsors must also maintain SAS environments or rely on third-party readers, which may be a constraint for non-SAS shops.
It is noted that SAS has a procedure (Proc JSON) to facilitate the conversion of the SAS data sets to JSON format. It will not be an issue when data sets in JSON format are required for submission.

FDA Policy and Future Adoption (Federal Register Context)

According to the recent Federal Register notice, the FDA is not yet changing requirements but is actively evaluating JSON as an option. The notice explains that CDER and CBER have already conducted a pilot (with CDISC and PhUSE) showing that Dataset-JSON “has the potential to serve as a transport file for study data”. Based on a 2022 assessment, the FDA found JSON to be the most promising modern format to potentially replace XPT v5. FDA explicitly states it is considering Dataset-JSON “with the long-term potential to replace SAS XPORT Format (XPT)” for eStudy data. The Agency is requesting comments on the benefits and risks of adopting JSON and on integration challenges with current tools.

Importantly, the notice does not immediately authorize use of JSON in submissions. Until any regulatory change is finalized, sponsors must continue using FDA-supported formats (i.e. XPT v5 files with define.xml) for study data. FDA will consider the public feedback before deciding. The notice indicates that if FDA does adopt Dataset-JSON, it will update its guiding documents (specifically the “Standardized Study Data” guidance implementing Section 745A(a)) to specify JSON as a permitted format. In summary: FDA’s official preference today remains XPT (v5), but a future shift to JSON is on the table pending the rulemaking process and guidance revisions.

Practical Considerations for Sponsors

Regulatory compliance: Sponsors should follow FDA’s current standards. Until JSON is explicitly allowed, electronic study data must use formats in FDA’s Data Standards Catalog (currently XPT v5 for tabulation/analysis data). Any use of JSON for a submission would require prior FDA agreement (e.g. a pilot protocol or waiver). Sponsors should monitor the comment process (comments due June 9, 2025) and watch for any updated guidance.
Data preparation: Most sponsors build SDTM/ADaM in SAS or similar tools. Producing XPT files is straightforward in that environment (PROC COPY, EXPORT, or LIBNAME XPT). Moving to Dataset-JSON would require developing new export routines or converters. SAS 9.4 can write JSON, but additional CDISC JSON schema mapping may be needed. Conversely, new entrants or CDS/non-SAS shops may find JSON easier since many analytics platforms (R, Python, etc.) parse JSON naturally. Either way, sponsors may need to invest in tool upgrades or staff training if and when JSON becomes accepted.
Long-term archiving and interoperability: JSON’s plain-text nature may benefit long-term data access (no proprietary format, easily versioned). On the other hand, XPT has a long track record for archiving and reusability within regulated drug development. Sponsors should plan how to store meta-data (define.xml or embedded JSON schema) for whichever format they use.
Transition planning: FDA’s pilot (reported by CDER/CBER and industry) suggests promising results. Sponsors may consider participating in further testing or industry surveys to shape the outcome. They should factor potential future regulatory changes into their IT roadmaps. For example, new statistical or data warehouse systems could be chosen with JSON capabilities in mind. Stakeholders (data managers, statisticians, IT) should communicate so that any shift in format will be smooth (e.g. ensuring traceability between old and new-format data).
Resource impact: In the near term, maintaining support for XPT remains essential (the FDA is not dropping it yet). In the long term, shifting to JSON may lower costs (e.g. fewer hardware needs, quicker data processing as noted in industry pilots) but will require upfront effort. Sponsors should balance these factors and perhaps begin exploratory work (e.g. trial converting legacy XPT files to JSON) to assess any challenges ahead.

Trends and Upcoming Changes

The Federal Register notice signals a trend toward modernizing study data formats. JSON’s ubiquity in web and healthcare (e.g. FHIR) and its alignment with FDA’s Data Modernization Action Plan are strong drivers. CDISC’s release of Dataset-JSON v1.1 (Dec 2024) and ongoing PhUSE work show industry momentum. If JSON is adopted, expect a multi-year transition: FDA will announce any implementation timeline in future Federal Register updates (similar to how new CDISC versions are phased in). Internationally, regulators (like Health Canada or PMDA) may also follow FDA’s lead. In practice, sponsors should prepare for eventual co-existence of formats: for some time, both XPT and JSON may be permitted (with effective dates).

In summary, the immediate trend is that the FDA is open to modern data standards: it found JSON superior to alternatives (SAS XPT v8 or XML) in 2022. However, any concrete requirement change awaits the rulemaking process. Sponsors should stay informed, consider testing JSON internally, and be ready to meet whichever format the FDA ultimately endorses.

References:

### This article is written with AI assistance ###.

Vanda Pharmaceuticals Sues FDA Over Tradipitant Hearing Delay – A Misplaced Battle?

2025-04-28T23:00:00.001-04:00

This past week, Vanda Pharmaceuticals filed a federal lawsuit accusing the FDA of unlawfully delaying a hearing on the company’s new drug application (NDA) for tradipitant in gastroparesis. According to Fierce Biotech, the dispute stems from a Complete Response Letter (CRL) the FDA issued in September 2024 rejecting tradipitant as a gastroparesis treatment. Vanda contends the agency “generally disregarded” its evidence, and is now blaming FDA bureaucracy and mass layoffs for stalling the hearing. However, an examination of the facts suggests Vanda’s legal aggression is largely misdirected. The FDA’s cautious stance appears justified by the clinical data: the pivotal Phase 3 trial failed to meet its primary endpoint, and the company is left relying on post-hoc analyses to claim efficacy, which is not a substitute for a positive trial result.

Tradipitant and Gastroparesis: The Clinical Background

Gastroparesis is a chronic stomach motility disorder often marked by severe nausea and vomiting. Vanda’s investigational drug, tradipitant (an NK1-receptor antagonist), was tested in diabetic and idiopathic gastroparesis patients. The key Phase 3 trial (ClinicalTrials.gov NCT04028492) randomized 201 adults (about half diabetic, half idiopathic) to tradipitant 85 mg vs. placebo twice daily for 12 weeks. The primary endpoint was the change from baseline in daily nausea severity at 12 weeks. Secondary endpoints included other gastroparesis symptoms and patient-reported outcome measures.

The trial did not meet its primary endpoint. In the full intention-to-treat (ITT) analysis, the difference in nausea reduction between tradipitant and placebo was not statistically significant (P = .741). In fact, the overall results showed no significant improvement on nausea severity or other major symptoms. According to FDA briefing materials and Vanda’s own publications, both the primary and secondary endpoints failed to reach significance. Vanda’s press release and a Healio report acknowledge that “the drug failed to meet statistically significant change in nausea severity at 12 weeks vs. placebo”. These negative findings were the basis for the FDA’s CRL.

Primary endpoint missed: Tradipitant did not significantly outperform placebo on nausea reduction at 12 weeks (P = .741).
Secondary endpoints missed: Other symptom scores likewise showed no statistical benefit over placebo.
Post-hoc analyses only: Vanda performed subgroups and sensitivity analyses (e.g. patients with high blood levels of drug or excluding certain “confounders”) that showed some improvement. However, these were not pre-specified endpoints, and the official trial data remained negative.

Because the registered primary outcome was negative, the trial is conventionally considered “failed.” Vanda’s scientists point out that in post hoc subsets (for example, patients with adequate drug exposure) tradipitant appeared to reduce nausea significantly. But regulatory agencies are very clear: post-hoc or exploratory findings cannot replace a prospectively powered success. The FDA expects sponsors to present all trial data, not just cherry-picked favorable subsets. As the FDA guidance states, submissions must include “all data or information relevant to an evaluation of…effectiveness” and avoid “selecting only those sources that favor a conclusion of effectiveness.” Any conflicting evidence must be explained by a compelling scientific rationale. Vanda’s reliance on post-hoc signal is precisely the sort of selective evidence that regulators warn against.

Regulatory Standards: When One Trial Fails

FDA’s drug approval standard under the U.S. Food, Drug, and Cosmetic Act requires “substantial evidence” of effectiveness, typically meaning at least two adequate and well-controlled trials each convincing on its own. In practice, the FDA may accept a single trial only if it is exceptionally persuasive and is backed by confirmatory evidence. But in this case, tradipitant’s lone Phase 3 trial produced a non-significant primary result. Under these rules, that trial generally does not qualify as adequate evidence on its own.

Key FDA guidelines reinforce this approach:

FDA has long held that substantial evidence usually means two positive trials, or one trial plus “convincing” confirmatory data.
A negative primary outcome is usually regarded as inconclusive: experts advise that a failed primary endpoint generally warrants additional trials (with better design or power) rather than reinterpreting existing data. In fact, Pocock and Stone’s NEJM review “The Primary Outcome Fails – What Next?” (2016) emphasizes that missing a primary endpoint should prompt new studies, not substitution by post-hoc findings.
FDA guidance explicitly warns sponsors to present all data. Trials may be questioned if negative overall results are explained away by excluding unfavorable subsets without strong justification.

Furthermore, the FDA’s own Good Review Practices (GRPs) underscore that reviewers follow documented best practices – focusing on consistency, transparency, and rigor. These internal policies help ensure each application is handled fairly and methodically, not arbitrarily slowed. The agency pointed out that Vanda’s request involves 15,000 pages, including new analyses not in the original NDA, which justifies thorough review under GRPs.

In sum, the regulatory framework suggests that FDA’s request for additional data and its cautious timetable are not arbitrary delays but adherence to standard procedures. A single failed pivotal trial simply does not meet the substantial-evidence bar. According to these criteria, the CRL was scientifically grounded.

When a Primary Outcome Fails

Statisticians and regulators agree: if a trial misses its primary outcome, the conservative path is to consider the study negative (or at best inconclusive) and plan further research. Pocock and Stone’s NEJM review makes this clear. They argue that a non-significant primary result generally means the experiment didn’t prove efficacy; turning to retrospective subgroups or alternative endpoints without a new trial risks false-positive “findings.” In practical terms, a missed endpoint should lead to redesigning studies or confirming any hints of effect in fresh data. Simply reanalyzing the same data to “find” significance is discouraged.

FDA guidance mirrors this stance. Even when one trial shows some favorable signal, the agency demands that confirmatory evidence come from independent sources (for example, a second trial or external data). Confident conclusions require evidence that is not retrospectively cherry-picked. In Vanda’s case, the company essentially trimmed its data after the fact (excluding patients and focusing on those with higher drug exposure) to claim a positive result. By FDA and statistical standards, that strategy is insufficient. It falls short of the clear, prospectively defined success needed to approve a new treatment.

The Lawsuit and Vanda’s Claims

Vanda’s lawsuit is technically about FDA’s timing: the company alleges the FDA is unlawfully delaying a regulatory hearing on its NDA and related disputes. According to Fierce Biotech, FDA told Vanda it would not schedule the hearing until September 2025 because of the case’s complexity, the new material submitted, and even unrelated litigation and staff layoffs. Vanda responded by suing, accusing the FDA of stonewalling it and even blaming the agency’s Commissioner and hiring cuts.

In parallel, Vanda is fighting on multiple fronts: it has also challenged FDA rules on its insomnia drug, on clinical holds for other tradipitant studies, and more. The Fierce article notes that Vanda has pressed FDA to hold a hearing within 120 days of its January 2025 request, as the company believes is its right.

However, all these battles circle back to the core issue: tradipitant’s failed data. The CRL that triggered the hearing request was grounded in the negative phase 3 results. In public statements, Vanda’s CEO complained that FDA “generally disregarded” the evidence — but that “evidence” was largely the same trial data and post-hoc arguments that regulators found unconvincing. The FDA’s position is that the application needed additional study, in line with scientific norms.

A Misplaced Blame

It’s understandable that Vanda is frustrated: developing new drugs is costly, and delays eat into patents and investor patience. But blaming the FDA’s procedures overlooks the science. The data from the major gastroparesis trial simply didn’t demonstrate clear efficacy. In drug development, a failed trial is a standard setback, not usually a basis for litigation. Lawsuits over process cannot overcome the fact that the trial was negative by its own primary analysis.

Regulatory experts would likely say that calling for more trials is the correct outcome. As FDA guidance and statistical authorities emphasize, when a primary endpoint is missed, one doesn’t put lipstick on the dataset to make it pass. Instead, one either finds a new, adequately powered study design or gathers stronger confirmatory evidence. Vanda’s focus on legal maneuvers and after-the-fact data trimming comes across as deflecting from this core reality.

In the meantime, the FDA is merely following its structured review timelines and good practice guidelines. Delays stem from legitimate reasons (volume of material, outside litigation, workforce changes), not from any conspiracy to stall Vanda unfairly.

Ultimately, Vanda’s case serves as a reminder that science should drive decision-making, even amid disputes. The trial evidence—supported by FDA and statistical guidance—points to one conclusion: traditionpitant’s efficacy was not established by the trial as conducted. Vanda may see a silver lining in exploratory data, but regulators are right to hold the company to proven standards. Suing over process, in this instance, appears like a costly distraction from the real task: generating reliable clinical proof for tradipitant.

References:

FDA guidance “Demonstrating Substantial Evidence of Effectiveness With One Adequate and Well-Controlled Clinical Investigation and Confirmatory Evidence” (2023)
FDA guidance “Demonstrating Substantial Evidence of Effectiveness for Human Drug and Biological Products “ (2019)
Carlin et al (2024) The Efficacy of Tradipitant in Patients With Diabetic and Idiopathic Gastroparesis in a Phase 3 Randomized Placebo-Controlled Clinical Trial, Clin. Gastroenterol. Hepatol.
Pocock & Stone (2016) The Primary Outcome Fails - What Next? NEJM
FDA internal “Good Review Practice: OND Review Management of INDs and Marketing Applications for Nonprescription Drug Products “
FDA internal ““Good Review Practice: Statistical Review Template”

Comparison of “Serious Breach” (EMA) vs “Important Protocol Deviation” (FDA)

2025-04-26T23:00:00.002-04:00

International Council for Harmonisation (ICH) E3 Guideline: Structure and Content of Clinical Study Reports Questions & Answers (R1) (2012) provided the definition for protocol deviation and important protocol deviation. The term "protocol violation" should be avoided and replaced with "protocol deviations" :

A protocol deviation is any change, divergence, or departure from the study design or procedures defined in the protocol.
Important protocol deviations are a subset of protocol deviations that may significantly impact the completeness, accuracy, and/or reliability of the study data or that may significantly affect a subject's rights, safety, or well-being. For example, important protocol deviations may include enrolling subjects in violation of key eligibility criteria designed to ensure a specific subject population or failing to collect data necessary to interpret primary endpoints, as this may compromise the scientific value of the trial.

Protocol violation and important protocol deviation are sometimes used interchangeably to refer to a significant departure from protocol requirements. The word “violation” may also have other meanings in a regulatory context. However, in Annex IVa, Subject Disposition of the ICH E3 Guideline, the term protocol violation was intended to mean only a change, divergence, or departure from the study requirements, whether by the subject or investigator, that resulted in a subject’s withdrawal from study participation. (Whether such subjects should be included in the study analysis is a separate question.)

To avoid confusion over terminology, sponsors are encouraged to replace the phrase “protocol violation” in Annex IVa with “protocol deviation”, as shown in the example flowchart below. Sponsors may also choose to use another descriptor, provided that that the information presented is generally consistent with the definition of protocol violation provided above.

In adopting and implementing the ICH E3 guidelines, the European Medicines Agency (EMA) and the U.S. Food and Drug Administration (FDA) differ in certain respects. In particular, the EMA guidelines introduce a new term: “serious breach.” While the concept of a “serious breach” overlaps with that of an “important protocol deviation,” subtle distinctions remain between the two terms.

The EMA’s clinical trial protocol breach guideline (CTR Article 52) uses the term “serious breach” to mean a protocol or regulation violation with significant impact on a trial. In contrast, FDA’s protocol deviations guidance (Draft version, December 2024) adopts ICH definitions and uses “important protocol deviation” for critical departures from the protocol. We compare these terms by definition, thresholds, reporting, examples, and implications.

	Serious Breach (EMA/CTR)	Important Protocol Deviation (FDA/ICH)
Definition	Any deviation of the approved protocol (or EU CTR) likely to affect subject safety, rights, or data reliability to a significant degree. This is a legal term under EU CTR (Art. 52).	A subset of protocol deviations that might significantly affect data completeness, accuracy, or reliability or significantly affect a subject’s rights, safety, or well‑being. (FDA guidance aligns with ICH E3(R1)).
Criteria/Threshold	Threshold is “likely to affect to a significant degree” at least one of: safety, rights, or data integrity. The impact must be substantial (regulatory text: “significant degree”). In practice, breaches that systematically endanger participants or core data are serious.	Threshold is “might significantly affect” key trial aspects. Important deviations are those that could undermine data quality or subject welfare – for example, affecting critical‑to‑quality factors (ICH E8(R1)) such as eligibility, endpoint collection, randomization, or safety monitoring. Deviations not reaching “significant” impact are considered minor.
Reporting Obligations	Mandatory notification to regulators: The sponsor must report via CTIS (EU portal) without undue delay, and within 7 calendar days of becoming aware. The sponsor is responsible for the report (it may delegate CTIS submission). All serious breaches must be entered into the EU database (CTIS). Investigators and CROs must promptly inform the sponsor of any suspected serious breach. Failure to report can trigger GCP inspection findings.	No direct mandatory FDA notification: FDA does not require sponsors to notify FDA of individual protocol deviations. Instead, FDA recommends that investigators promptly report important deviations to the sponsor and IRB, and sponsors document them. Specifically, investigators should report all deviations to the sponsor (highlighting the important ones), and device trials must report emergency deviations to sponsor/IRB within ~5 days. Sponsors should capture important deviations in oversight and include them in regulatory submissions: FDA guidance advises sponsors to discuss important deviations in the Clinical Study Report (NDA/BLA) and list all deviations in the Study Data Tabulation Model (SDTM) (with an indicator of importance).
Examples	EMA guideline Appendix I (non‑exhaustive) includes situations like: dosing a subject with the wrong drug or dose (e.g. incorrect IMP, wrong frequency); giving IMP to a pregnant subject without protocol-required test; systematically failing to administer required therapy. Other examples: temperature excursions unverified by corrective action, severe informed consent lapses, etc. These all have significant safety/data impact.	FDA guidance examples highlight failures that impair safety or data. For instance: skipping key safety assessments (e.g. missing lab tests or not administering study drug per spec); giving a prohibited concomitant treatment; failure to obtain valid informed consent or protect private data; dosing errors (wrong treatment or dose); non‑adherence to randomization; enrolling ineligible subjects or missing primary endpoint data; or unblinding inappropriately. These deviations affect subject protection or critical data (as listed in the FDA document).
Consequences/Implications	Regulatory action: Reporting a serious breach obligates Member States to evaluate it. Some breaches may require corrective actions overseen by regulators; others may trigger inspections or trial suspension. Failure to have a proper breach‐reporting system can itself be a GCP inspection finding. If serious breaches reveal fundamental trial defects (e.g. safety is compromised), the trial approval or application may be withdrawn. In summary, a serious breach can lead to heightened regulatory scrutiny, mandatory CAPA plans, or more severe sanctions.	Data validity risks: The FDA guidance notes that frequent or severe important deviations can compromise a trial’s validity. Reviewers may judge a study “not adequate and well‑controlled” if key deviations occur (e.g. wrong enrollment or missing data). As a result, sponsors are urged to design protocols to minimize these issues (via “quality‑by‑design”). Important deviations do not immediately trigger FDA enforcement, but they must be documented; pervasive deviations could lead FDA to question the reliability of submitted data. Investigators/IRBs also review important deviations for participant safety.

Key differences: A serious breach is a legal CT regulation concept requiring prompt notification to EU authorities; by contrast, an important protocol deviation is a statistical/GCP concept from FDA/ICH guidance, focused on documenting major protocol departures in the trial record. The EMA rule imposes timely reporting to regulators (CTIS) for serious breaches, whereas the FDA guidance places emphasis on internal reporting and documentation (investigator→sponsor/IRB, sponsor→study report) rather than immediate FDA notification.

References:

EMA’s “Guideline for the notification of serious breaches of Regulation (EU) 536/2014 or the clinical trial protocol”
FDA’s Draft Guidance for Industry “Protocol Deviations for Clinical Investigations of Drugs, Biological Products, and Devices".

Classifying Protocol Deviations - New FDA Guidance on Protocol Deviations

2025-04-25T23:00:00.001-04:00

In clinical trials, protocol deviations are generally unintentional departures from the IRB-approved protocol and are commonly not discovered until after they occur (e.g., an investigator’s failure to perform a protocol-required test is discovered by the study monitor during a routine monitoring visit).

Too many protocol deviations could indicate the protocol issue (the protocol is poorly written), the protocol compliance by the investigators, and the overall quality of the study. Protocol deviations are collected and reviewed throughout the study. At the end of the study, the final list of protocol deviations will be converted into the data set (CDISC SDTM DV data set). The protocol deviations will then be listed and summarized for the inclusion in the clinical study report. According to ICH E3 "Structure and Contents of Clinical Study Reports", protocol deviations will be described in the CSR section 10.3.

For NDA/BLA submissions to CDER, FDA, there is a special requirement for submitting the BIMO (bioresearch monitoring) data sets and listings to assist FDA to select the investigational sites for inspection. Protocol deviations are important part of the BIMO data. According to "BIORESEARCH MONITORINGTECHNICAL CONFORMANCE GUIDE", the protocol deviations need to be provided in listings by subject and by clinical site.

7. Protocol Deviations
This by-subject, by-clinical site listing should include all protocol deviations. The listing should include a description of the deviation and identify whether the sponsor considered the deviation to be an important or non-important protocol deviation.

Protocol deviations need to be classified based on the seriousness of the protocol deviation and the impact on the safety and efficacy evaluation of the protocol deviations. Traditionally, we have classified the protocol deviations into minor and major categories. However, other terms are used for categorizing the protocol deviations: critical protocol deviations, significant protocol deviations, ...

FDA's new guidance "Protocol Deviations for Clinical Investigations of Drugs, Biological Products, and Devices" (December 2024) sets this straight. Only two categories will be used for protocol deviations: Important protocol deviations and other protocol deviations.

1. Important Protocol Deviations
As noted above, in this guidance an important protocol deviation is a subset of protocol deviations that might significantly affect the completeness, accuracy, and/or reliability of the study data or that might significantly affect a subject’s rights, safety, or well-being. While other terms such as major, critical, and significant have sometimes been used to classify such protocol deviations, FDA recommends using important to encompass all these terms.
2. All Other Protocol Deviations
All other protocol deviations that do not meet the definition of an important protocol deviation may encompass the commonly used terms minor, noncritical, and non-significant deviations. Examples of all other protocol deviations may include small deviations from protocol-specified visit windows; a signed consent with a page missing a participant’s initial; or failure to perform a study procedure not relevant for safety monitoring or not related to an important study efficacy endpoint (e.g., primary or secondary endpoints).