7.3 Estimand Shift
The adaptation that changes the question
Sections 7.1 and 7.2 examined specific forms of adaptation—sample size re-estimation and population enrichment—and noted, in passing, that each can shift the estimand. This section examines estimand shift directly: the ways in which adaptations that do not explicitly change the primary endpoint nonetheless change what the trial is estimating.
Estimand shift is the most conceptually underappreciated risk in adaptive design. A sponsor who adds patients through SSR, or who enriches toward a biomarker subgroup, understands that something has changed. What is less often recognized is that even when the nominal endpoint, the analysis model, and the enrolled population all remain unchanged, an adaptation can alter the conditional structure of the analysis in a way that changes the estimand implicitly.
The ICH E9(R1) framework requires that the estimand—population, variable, intercurrent event strategy, and population-level summary—be specified before the trial begins and remain stable throughout. When an adaptation changes any of these four attributes, the estimand has shifted, and the primary analysis must address the shift explicitly. When an adaptation changes the conditional structure of the analysis without changing any of the four attributes nominally, the estimand has shifted implicitly, and the shift is harder to detect and harder to correct.
Both forms of shift require design-stage attention—before enrollment, as part of the pre-specification of the adaptive rule—not analysis-stage correction.
Explicit estimand shifts
An explicit estimand shift occurs when the adaptation changes one of the four estimand attributes in a way that is visible in the design.
Population shift is the clearest case. Adaptive enrichment, as examined in Section 7.2, changes the population attribute: the patients enrolled after the enrichment are a different population than those enrolled before. The estimand that was specified for the pre-adaptation phase—a population-level treatment effect in all eligible patients—does not describe the post-adaptation phase’s population. A new estimand, or a combined estimand, must be specified.
Treatment attribute shift occurs in seamless phase II/III designs that adapt the dose based on phase II results. The treatment attribute of the estimand is the specific dose that patients receive. When the dose is selected adaptively—based on phase II efficacy and safety data—the treatment attribute changes at the adaptation point: patients in phase III receive a different dose than some patients in phase II. If phase II patients who received the selected dose are included in the primary analysis—a common efficiency gain in seamless designs—the estimand for those patients is the same as for phase III patients. If phase II patients who received doses that were not selected are included, or if phase II patients are excluded entirely, the estimand for the pooled population must be specified to describe who is included and why.
Intercurrent event strategy shift can occur when the adaptation changes the background therapy or the trial’s context in ways that affect how intercurrent events are expected to occur. An enrichment that shifts the population toward sicker patients may increase the rate of rescue medication use; a SSR that extends the trial duration may increase the rate of treatment discontinuation due to tolerability. When the intercurrent event rate or pattern changes as a consequence of the adaptation, the pre-adaptation intercurrent event strategy may not adequately describe the post-adaptation situation, and the combined analysis must address the change.
Implicit estimand shifts
Implicit estimand shifts are more dangerous because they are harder to recognize. They occur when the adaptation changes the conditional structure of the analysis without visibly changing any estimand attribute.
The most important example is effect-size SSR. When the sample size is adjusted based on the interim treatment effect estimate, the final analysis is conditional on the interim result—in the sense that the patients enrolled in the second stage are a sample selected from the population under the condition that the interim result was in the region that triggered an increase. The final test statistic is not an unconditional estimate of the population treatment effect; it is a conditional estimate, conditioned on the interim result falling in the region that triggered the adaptation.
The combination test methods used in effect-size SSR are designed to produce a valid test—one that controls the type I error—under this conditional structure. But the treatment effect estimate from the combination test may not be a valid estimate of the population treatment effect in the same sense that a fixed-design estimate would be. The conditional maximum likelihood estimate—the estimate obtained by conditioning on the adaptive decision—is approximately unbiased for the population treatment effect when the adaptation threshold is not too aggressive, but the approximation worsens as the threshold tightens.
This is an implicit estimand shift: the nominal estimand has not changed—same population, same endpoint, same intercurrent event strategy, same summary measure—but the analysis is estimating a quantity that is conditioned on the interim result, which is not the same quantity as the unconditional population treatment effect.
Seamless phase II/III designs
Seamless designs—in which phase II and phase III are conducted in a single trial with a pre-specified adaptation at the phase boundary—are the most complex form of adaptive design because they introduce multiple potential estimand shifts simultaneously.
A typical seamless design enrolls patients in phase II across multiple doses, selects one or more doses for continuation based on phase II results, and continues enrollment in phase III at the selected dose or doses. The primary analysis combines phase II and phase III data for the selected dose, with the combination test method controlling the type I error for the adaptive dose selection.
The estimand for the combined analysis must specify who is included from each phase and why. If phase II patients at the selected dose are included, the estimand covers a population that was enrolled before and after the dose selection—which may have different characteristics if the dose selection affected site practices, patient counseling, or eligibility interpretation. If phase II patients at non-selected doses are excluded, the estimand covers only the patients at the selected dose, but the dose selection was based on all phase II data, creating a conditional structure that the combination test must account for.
The dose selection rule is itself a source of estimand shift. If the selected dose is the one with the highest efficacy signal in phase II—regardless of safety—the patients enrolled at that dose in phase III are a population selected under the condition that the efficacy signal was highest at that dose. The treatment effect estimate in phase III may be influenced by selection bias: the dose with the highest phase II signal is likely to have a higher phase II estimate than the true dose-specific effect, because of regression to the mean. The phase III estimate—which conditions on the dose having been selected—may partially correct for this, or may not, depending on the combination method.
Pre-specification of the dose selection rule—including the criteria for selection, the weighting of efficacy and safety signals, and the method for handling ties—is therefore not just a governance requirement. It is a scientific specification of what the primary estimand is, because the dose selection rule determines which patients are in the primary analysis and under what conditions they were enrolled.
The estimand for the adapted trial
Specifying the estimand for an adaptive trial requires working through the four attributes of the estimand for each phase of the trial and for the combined analysis.
Before the adaptation: the estimand is as specified at design. Population, variable, intercurrent event strategy, and population-level summary are all pre-specified and unchanged.
After the adaptation: the adaptation changes one or more attributes. The new values of the changed attributes must be specified as part of the pre-specification of the adaptive rule. If the population changes—because of enrichment—the post-adaptation population attribute is specified as part of the enrichment rule. If the treatment changes—because of dose selection—the post-adaptation treatment attribute is the selected dose, and the dose selection rule pre-specifies which dose or doses will be selected under which circumstances.
For the combined analysis: the combined estimand must cover the patients enrolled in both phases. The combined population, the combined variable, the combined intercurrent event strategy, and the combined summary measure must be specified. When the population changes at the adaptation, the combined estimand requires explicit specification of whether the pre- and post-adaptation patients are analyzed jointly or separately, and how the combination weights the two phases.
This four-part specification—before adaptation, after adaptation, combined—is the estimand framework for adaptive designs. It is more demanding than the estimand framework for fixed designs, and it must be completed before enrollment begins—which means before the adaptive rule can be finalized, because the estimand determines what the adaptive rule is adapting.
Documentation and the audit trail
The estimand shift at an adaptive decision point must be documented in real time, not reconstructed afterward. At the time of the adaptation, the following must be recorded.
What data triggered the adaptation. What the adaptation decision was and the basis for it—including the specific values of the criteria in the pre-specified rule that were met or not met. What the post-adaptation estimand is, based on the pre-specified framework. Who made the adaptation decision and under what governance structure. What information was made available to each party in the adaptation process.
This documentation is the audit trail for the adaptation. It is the evidence that the adaptation was made according to the pre-specified rule rather than in response to information beyond what the rule allowed. Without it, the adaptation cannot be verified as pre-specified, and the claim that the primary analysis is valid under the combination test method cannot be established.
The audit trail for adaptive designs is more demanding than for fixed designs, because the adaptive decisions are made in real time—under time pressure, with governance constraints, by multiple parties with different information access—and the record of those decisions must be complete and contemporaneous. A reconstruction of the adaptation decision based on meeting notes and email records, prepared for regulatory submission, is not an audit trail. It is a narrative, and narratives can be selective.
What this section demands before proceeding
Before Section 7.4’s examination of non-inferiority failure modes in adaptive designs, the estimand shift analysis must be complete.
For each planned adaptation in the trial—SSR, enrichment, dose selection, or other—the estimand shift must be identified: which of the four attributes changes, how it changes, and what the post-adaptation estimand is. The combined estimand must be specified: how the pre- and post-adaptation patients are handled in the primary analysis, with the combination weights and combination method pre-specified. And the documentation requirements for the adaptation—the audit trail—must be specified before enrollment, so that the information to be recorded is known before the adaptation is triggered.
An adaptive design whose estimand shifts have not been identified and addressed is a design that will answer a different question than it was designed to answer—and the difference may not be recognized until the regulatory review asks what population the primary result describes.
References: ICH E9(R1) Addendum on Estimands and Sensitivity Analysis in Clinical Trials (2019); Bauer et al., “Adaptive Designs for Confirmatory Clinical Trials,” Stat Med 2016; Wassmer and Brannath, Group Sequential and Confirmatory Adaptive Designs in Clinical Trials (2016); FDA Guidance for Industry, Adaptive Designs for Clinical Trials of Drugs and Biologics (2019).