8.4 Design Review and the Independent Challenge
Why independent review is a governance requirement
The design decisions documented in Chapters 1 through 7 are made by people who are close to the trial—the clinical scientists who understand the disease, the statisticians who understand the analysis, the clinical operations team who understand the logistics. These are the right people to make the decisions. They are also people who have invested in the design, who believe in the treatment, and who are subject to the optimism biases and the organizational pressures that Chapter 3 identified as sources of effect size overestimation and Chapter 4 identified as sources of governance failure.
Independent review is not a challenge to these people’s competence. It is a structural protection against the specific vulnerabilities that competent, motivated people exhibit when they are close to their own work. The same human tendencies that make an expert design team effective—deep familiarity with the science, strong commitment to the development hypothesis, pattern recognition that integrates experience into judgment—make them poor reviewers of their own design. They know what the design is supposed to do; they are less able to see what it actually does when their assumptions are wrong.
An independent reviewer brings the complementary virtue: distance. They do not know what the design is supposed to do; they know only what it says. This distance allows them to ask questions that the design team stopped asking because they seemed obvious: Does the estimand actually correspond to the endpoint definition? Does the power calculation use the adjusted alpha after interim analyses? Does the hierarchy order reflect clinical importance or statistical convenience? Is the DSMB charter specific enough to govern the decisions the DSMB will be asked to make?
These questions may have good answers. The independent reviewer’s job is not to find problems but to verify that the answers are defensible—that the design as documented can withstand the scrutiny it will face from regulatory reviewers, from DSMB members who did not participate in the design, and from clinical audiences who will interpret the trial’s result.
What independent review must examine
The independent review of a clinical trial design is not a single event. It is a staged process that mirrors the document lock sequence of Section 8.1: a review of the protocol before finalization, a review of the SAP before finalization, a review of the DSMB and SSR charters before the first interim, and a review of the decision log before the database lock.
At each stage, the reviewer examines the document against the design principles established in the prior chapters.
Protocol review. The reviewer examines whether the estimand is specified with sufficient precision to be operationalized in the SAP without additional decisions. Whether the endpoint definition is specific enough to be applied consistently across sites and assessors. Whether the intercurrent event strategy is consistent with the estimand’s scientific intent. Whether the co-primary decision rule, if present, specifies the conjunctive or disjunctive criterion clearly. Whether the interim analysis plan specifies the spending function. Whether the pre-specified subgroup list correctly labels each analysis as confirmatory or exploratory.
SAP review. The reviewer examines whether the hierarchical testing order reflects clinical importance rather than statistical convenience. Whether the primary analysis model is fully specified—no deferred decisions about covariate selection or missing data handling. Whether the sensitivity analyses address the assumptions most likely to be challenged. Whether the adaptive rule details—the SSR formula, the enrichment threshold, the combination test method—are specific enough to be implemented without discretion.
Charter review. The reviewer examines whether the DSMB charter specifies the decision hierarchy for scenarios when multiple stopping signals are present simultaneously. Whether the SSR charter specifies the data to be used, the formula, and the communication protocol with sufficient precision. Whether both charters specify the documentation requirements that will enable verification of implementation.
Decision log review. The reviewer examines whether the decision log is complete—whether every major protocol amendment, every DSMB recommendation, every regulatory interaction is recorded. Whether the entries were made within the specified time windows—whether the timestamps are consistent with contemporaneous recording rather than retrospective reconstruction. Whether the entries for DSMB interactions record the recommendation and the sponsor’s response, without revealing the arm-specific data that drove the recommendation.
Who should conduct independent review
The independence requirement has two dimensions: organizational and technical.
Organizational independence means the reviewer has no reporting relationship to the people who made the design decisions, no financial stake in the trial’s outcome, and no involvement in the development of the documents being reviewed. A statistician from a different division of the same company, reviewing a design produced by the clinical development team, is organizationally independent of the specific design team but not independent of the company’s interests. A statistician from an academic institution with no current consulting relationship to the sponsor is more independent. A statistician from a competing company is fully independent but typically not available for confidentiality reasons.
The practical standard for most trials is a senior statistician from a different function within the same organization—a regulatory statistics team reviewing a clinical development design, or an external consultant with no current engagement on this program—for routine design review, and an external academic statistician for trials whose design complexity or importance warrants a higher standard of independence.
Technical competence means the reviewer has sufficient expertise in the specific design elements being reviewed. A reviewer who is expert in frequentist hypothesis testing but unfamiliar with adaptive design methodology cannot meaningfully review the combination test specification for an adaptive enrichment design. A reviewer who is expert in oncology trial design but unfamiliar with the estimand framework cannot meaningfully review the intercurrent event strategy for a cardiovascular outcome trial. The reviewer’s expertise must be matched to the design’s complexity.
For complex designs—adaptive designs, NI designs with multi-regional populations, seamless phase II/III designs—the independent review may require a team of reviewers rather than a single reviewer: a statistician for the analysis plan, a clinical methodologist for the estimand and endpoint specification, and a governance expert for the DSMB and SSR charters.
The design review as a discipline, not an event
The independent review of a clinical trial design is most valuable when it is structured as a discipline—a regular, staged process that is embedded in the design timeline—rather than as a one-time event at the end of the design process.
When review is a one-time event, it occurs when the design is largely complete, the team has significant investment in the current approach, and the cost of substantive changes is high. The reviewer identifies problems; the team resists changing them because the cost of change is real and the benefit of the change—avoiding a regulatory challenge or a post-trial credibility problem—is hypothetical. The review becomes a negotiation rather than an examination.
When review is a discipline—when there are staged review points at the estimand specification stage, at the power calculation stage, at the interim analysis plan stage, and at the final document finalization stage—the design is reviewed when the cost of change is still manageable. The estimand is reviewed before the power calculation is built on it. The power calculation is reviewed before the interim analysis plan is built on it. The interim analysis plan is reviewed before the governance charters are built on it. Problems are identified and addressed before they compound.
The staged review structure requires the design team to articulate the design decisions at each stage—to make the decisions visible before they are embedded in subsequent decisions—and this articulation itself has value. A team that cannot explain, at the estimand review stage, why the chosen intercurrent event strategy corresponds to the scientific question has not fully thought through the estimand. The review forces the articulation, and the articulation forces the thinking.
After the design review: the design summary
At the completion of the design process—after the protocol has been finalized, the SAP has been locked, and the charters have been approved—a design summary document should be produced. The design summary is not a regulatory submission document; it is an internal governance record.
The design summary records, in one place, the key decisions made at each design stage: the estimand and its scientific justification, the effect measure and the rationale for its selection, the assumed effect size and the clinical team member who owns it, the power level and the justification for accepting the implied type II error, the interim analysis plan and its operating characteristics, the bias protection design and any acknowledged limitations, the claim structure and the hierarchical order with its clinical rationale, and for adaptive designs, the adaptive rules and their governance structure.
The design summary is the record of the design review’s conclusions—the documented basis for the independent reviewer’s determination that the design is defensible. It serves two purposes. First, it provides a reference document for the DSMB, the clinical operations team, and the regulatory agency that can orient each party to the trial’s design without requiring them to read the full protocol and SAP. Second, it provides the basis for the clinical study report’s account of the design rationale—the post-trial document that explains why the trial was designed as it was and how the primary analysis reflects the pre-specified design.
A trial that produces a design summary before enrollment begins is a trial whose design process was governed with sufficient discipline that the summary can be written. A trial that cannot produce a design summary—because the design decisions were made piecemeal, without documentation, by parties who are no longer available or who disagree about what was decided—is a trial whose governance system failed before the first patient was enrolled.
References: ICH E8(R1) General Considerations for Clinical Studies (2021); FDA Guidance for Industry, Establishing Effectiveness and Safety of Drugs Intended to Treat Serious Conditions (2023); Chow and Chang, Adaptive Design Methods in Clinical Trials (2011); Pocock, Clinical Trials: A Practical Approach (1983).