An FAQ on Adaptive Design for Device Trials

For manufacturers wondering if adaptive designs are the best option for their device trials, the following FAQ style presentation provides answers for several of the questions they likely have. If you still have questions after reviewing this, post them in the comments and we’ll see about securing responses for a follow-up article.
 
Q: What is the biggest benefit of adaptive designs for device manufacturers? 
Vicki Anastasi: The greatest benefits of adaptive designs are seen at the portfolio level. With adaptive designs, manufacturers can reduce the chance that errant trial parameters could cause an effective device to fail or suffer delays, thereby de-risking the entire portfolio. Adaptive designs can also provide opportunities to redirect limited resources to more productive products.
 
To achieve the portfolio-level benefits of adaptive designs requires that manufacturers determine early on in every product’s lifecycle whether it would benefit from an adaptive design. While not all trials are conducive to adaptive designs—for example, fast-enrolling, short-term trials would logistically preclude interim analyses—most device trials would benefit from simulations that identify risks that may arise over the course of a trial and aid in the development of designs that optimally address those risks.
 
This review process, however, does require an upfront investment of time and capital as well as access to biostatisticians and other experts who are specifically knowledgeable in device-relevant adaptive trials. These preparations are important, as the strategies and operational processes used for pharmaceuticals do not necessarily translate to benefits in device development. This early investment, however, will ultimately be offset by adaptations that, for example, reduce study duration by combining pilot and pivotal studies and optimize the sample size to both achieve statistical significance as well as prevent a trial from recruiting unnecessary subjects.
 
Q: What types of adaptive designs should device manufacturers consider using in their trials?
Anastasi: Manufacturers have many adaptive designs at their disposal, depending on the specific needs of the development program. These designs include:
 

• Group sequential designs. These give manufacturers the opportunity to stop a trial early if the device demonstrates effectiveness (which accelerates time-to-market) or if it demonstrates futility (which can preserve millions of dollars that would otherwise be spent on the trial that does not show promise).
• Sample size reassessment. This strategy lets manufacturers either reduce the costs and delays associated with overpowered trials and unnecessary recruitment, or reduce the potential risk that an underpowered study fails to capture a product’s true viability.
• Seamless designs. Manufacturers can alter a study’s hypothesis following an interim analysis, such that a late phase pilot study may continue into a pivotal study, thereby reducing white space and efficiently carrying over existing sites and patients.
• Changing study hypothesis. Manufacturers can combine superiority/inferiority studies, wherein if the device does not meet its superiority criteria, the trial can still demonstrate non-inferiority, increasing the chance for success and the opportunity to demonstrate the strongest claim possible.
• Adaptive enrichment. If, after an interim analysis, a manufacturer discovers that a device is more effective in a particular subgroup, they can adjust trial inclusion/exclusion criteria to include more patients for whom the device is effective. This could potentially prevent failure or underperformance due to the dilution of a device’s effect in an irrelevant patient population

 
Manufacturers may have more designs to consider, depending on the specific needs of their trials. Overall, adaptive designs remove risk from a portfolio in ways that benefit patients, regulators, and manufacturers alike.
 
Q: Can you give us any examples of how device companies have directly benefitted from using adaptive designs?
Anastasi: Many of our clients have incorporated adaptive designs into their trials, and they are seeing real results.
 
In one example, a manufacturer was developing an electrostimulation device, and they asked us for help in designing a trial that would support registration of the device as early as possible. We designed an adaptive trial that would not only yield useful results quickly, but that would also protect the manufacturer against potential delays and risks.
 
The design specified two prospectively planned interim analyses. We set thresholds for the sham control response rate and observed a treatment effect at the first and second interim analyses, respectively, that allowed the manufacturer opportunities to right-size study enrollment for optimal statistical power or to terminate the study early on efficacy or futility grounds.
 
At the second interim analysis, after enrolling only half of the total planned number of patients, ICON detected that the response-rate to the device was not meeting the pre-planned goal. The manufacturer deemed the device ineffective, and then, with the $1.9 million and five months of time that they would have otherwise spent on the ineffective device, they accelerated development of an improved treatment regime and delivery method that would offer a more promising future clinical study.
 
In another case, a company came to us with promising data on a disc replacement device from a non-inferiority trial. These data had encouraged the manufacturer to conduct an additional superiority trial, a rare feat in that indication. However, they found that it would be cost-prohibitive to start a new superiority trial from scratch with new protocols, a new recruitment strategy, and new sites.
 
To reduce costs and save time, ICON supported negotiations with the FDA to allow a midstream change to the non-inferiority trial that would enable the simultaneous collection of superiority data. ICON designed and executed an adaptive design to collect both non-inferiority and superiority data using a single, patient-sparing, efficient trial. This strategy ended up saving the company $5 million and accelerated their go-to-market timeline with a stronger value proposition.
 
Q: What effect would an adaptive design have on a manufacturer’s commercialization strategy for a device?
Anastasi: By using an adaptive design, manufacturers can deliver more innovations to market in a much shorter time span. Since adaptive designs accelerate cycle times, manufacturers can beat competitors to market and maximize the revenue potential from any device brought to market.
 
Seamless designs reduce white space between trials, group sequential designs let manufacturers stop trials early when a device demonstrates efficacy, and adaptive designs overall mitigate the risk of unnecessary setbacks during the clinical development process.
 
Q: How are regulators responding to the FDA draft guidance? Have you noticed regulatory changes in regard to device trials?
Anastasi: The FDA, EMA, and other regulatory agencies around the world are actively encouraging manufacturers to adopt adaptive designs. They are encouraged by the potential for adaptive designs to reduce risk for both patients and themselves.
 
According to the recently released FDA guidance, “adaptive designs may optimize the treatment of subjects enrolled in the study and safeguard their welfare from ineffective or unsafe treatments and interventions at the earliest possible stage.” Adaptive designs give patients the best chance of being on the right treatment and participating in a safe and secure trial.
 
To meet regulators’ desires for safe and secure trials, they stress that manufacturers update their infrastructure, from their monitoring and data management systems to how they work with IRBs to maintain data integrity. For example, they desire that manufacturers use purpose-built clinical trial management systems to handle data more securely. They also require that if manufacturers use an adaptive design, they must use statistical software that can perform Bayesian analyses. ICON’s ADDPLAN software, for example, has been validated for adaptive designs and licensed by the FDA, EMA, and PMDA for analyzing data from adaptive studies.