Bruce Johnston, Ph.D., Lead Solution Architect, MedAcuity06.06.23
The immediate reaction to a new round of regulatory guidance is to take the common view that it’s an obstacle that will hamper creativity, slow down the team, and get in the way of important, innovative medical devices getting to market. There is, however, another perspective to consider. FDA guidance can be, as the name implies, a guide to an efficient, effective path to success. Stepping deeper into the intent behind the regulation and guidance shows how they align with industry best practices that have been developed over many years and countless devices, both successful and not, to help the device manufacturer avoid the common mistakes that send development projects off the rails.
As with most regulatory guidance, the processes outlined in these two documents echo established industry best practices—in this case, from the human factors community. The idea that the design of a device’s user interface can have a direct bearing on the likelihood of users making mistakes goes back decades to seminal works by Norman3 and Vincente.4 The solution, it turns out, is simple: know your users. Talk to them, get their feedback, put prototypes in their hands. Understand their workflows, environments, and pressures. Likewise, understanding risk through effective risk assessment and management has long been recognized as key to a successful endeavor. A thorough understanding of the risk profile of a device facilitates effective decision making and allows resources to be focused on solving the important problems.
The biggest hurdle to effective use of regulatory guidance, and resulting regulatory compliance, often comes down to a lack of understanding of the risk-based approach that is an integral component of the FDA guidance. This gap in understanding often leads to one of two outcomes:
The regulatory landscape is built around safety risk, and mitigating any potential for a medical device to harm patients or users is, of course, the paramount concern in the development of a medical device. In our experience, the business risks associated with poor usability are substantial as well, and are important considerations when determining the scope of HFA that should be applied during the development of a device.
In the case of human factors, a laser focus on the users’ needs can reduce thrashing, scope creep, and development of unneeded features to significantly reduce overall development time.5 Understanding users, their needs, and their environments in order to design better, more successful devices is nothing new and the usability of a product is recognized as a desirable, marketable feature on its own. The difference when these concepts are applied to medical devices is that “better” is now interpreted through the additional lens of “safer.”
Some of the benefits associated with this perspective of regulatory guidance as a reflection of best practices include:
The time savings gained by this thinking is illusory. The work still needs to be done and our experience working with hundreds of device manufacturers and OEMs over many years has shown that, like fixing defects, fixing UI design issues gets dramatically more costly as time goes on and resources are invested in implementing those designs. Valuable information that could have steered the development effort toward early convergence and efficiency is instead coming too late in the process to have a positive effect on the outcome. Discovering this kind of information late in a program converts potential value into additional, preventable costs.
Beyond the development schedule, deprioritizing HFA activities can lead to issues and deficiencies in a regulatory submission, ultimately setting back the timeline for a product reaching the market by months or possibly even years. These kinds of delays are a problem for both device manufacturers and patients. Device manufacturers face increased costs, lost market windows, and damaged reputations. Patients are denied timely access to innovative devices that could improve their health. Even after a device gets to market, the costs resulting from recalls and associated liabilities can dwarf entire R&D budgets.
Effective application of the approaches to risk assessment and human factors assessment captured in the regulations provides a roadmap to high-value alignment of your device with your users’ needs. This alignment is a key element of a predictable path to safe and effective devices, and minimizes both safety risk and business risk.
References
Dr. Bruce Johnston has more than 25 years of experience designing and developing complex software systems and over 15 years of experience working in regulated medical software development. Much of this experience has been focused on UI design and human factors aspects of Class II and Class III medical device programs.
The Regulatory Landscape
The current regulatory landscape for human factors and usability was laid out in 2016 with the FDA’s Guidance on Applying Human Factors and Usability Engineering to Medical Devices.1 This guidance marked a shift in focus from the user as the problem (user error) to the device as the problem (use error) and provided a roadmap for applying human factors assessment (HFA) techniques to medical device design. A draft complement to the current guidance2 being circulated by the FDA seeks to provide additional guidance on the HFA process with a risk-based approach to determining human factors submission requirements.As with most regulatory guidance, the processes outlined in these two documents echo established industry best practices—in this case, from the human factors community. The idea that the design of a device’s user interface can have a direct bearing on the likelihood of users making mistakes goes back decades to seminal works by Norman3 and Vincente.4 The solution, it turns out, is simple: know your users. Talk to them, get their feedback, put prototypes in their hands. Understand their workflows, environments, and pressures. Likewise, understanding risk through effective risk assessment and management has long been recognized as key to a successful endeavor. A thorough understanding of the risk profile of a device facilitates effective decision making and allows resources to be focused on solving the important problems.
The biggest hurdle to effective use of regulatory guidance, and resulting regulatory compliance, often comes down to a lack of understanding of the risk-based approach that is an integral component of the FDA guidance. This gap in understanding often leads to one of two outcomes:
- Too little effort: Failure to identify usability risks through effective human factors assessments leads to mistaken conclusions concerning applicability of the regulations or the need to address issues. This results in the potential for a design that facilitates use error that will have to be remedied down the road and incur significant unanticipated costs.
- Too much effort: Lack of effective risk assessment can also lead to excessive and unnecessary effort. Where there is a lack of understanding of the actual risk, the reaction is often to do more to compensate. The result is the 600-page specification or report that does little to identify or reduce potential use errors.
The regulatory landscape is built around safety risk, and mitigating any potential for a medical device to harm patients or users is, of course, the paramount concern in the development of a medical device. In our experience, the business risks associated with poor usability are substantial as well, and are important considerations when determining the scope of HFA that should be applied during the development of a device.
Alignment with Best Practices
The collective experience of the medical device industry gained over many decades of successful, and not so successful, development programs has been captured and codified in industry standards, such as IEC 14971, IEC 62366, and IEC 62304. These standards, in turn, form the basis for the processes and practices recommended in much of the regulatory guidance for developing medical devices. This alignment of regulatory guidance and established best practices provides an opportunity for the developers of medical devices. The lessons learned from countless projects can be leveraged in the context of regulatory “compliance” to streamline development in a way that addresses safety risks while also reducing business risks (e.g., costs, time to market).In the case of human factors, a laser focus on the users’ needs can reduce thrashing, scope creep, and development of unneeded features to significantly reduce overall development time.5 Understanding users, their needs, and their environments in order to design better, more successful devices is nothing new and the usability of a product is recognized as a desirable, marketable feature on its own. The difference when these concepts are applied to medical devices is that “better” is now interpreted through the additional lens of “safer.”
Some of the benefits associated with this perspective of regulatory guidance as a reflection of best practices include:
- Identifying issues early: Usability problems will be found; the only question is when. The guidance steers programs toward “preliminary analyses and evaluations” to identify and address issues at the outset. The alternatives of identifying issues during development, verification testing, validation testing, or post-market come with dramatically increasing costs.
- Streamline development: Positioning UI design assessment as design input sets up cross-functional collaboration with development and other stakeholders. This provides for both effective communication of UI design intent to the development team, as well as a synergistic feedback loop to incorporate important implementation perspectives into the design.
- Building in compliance: Following the path laid out by the guidance ensures the artifacts you need as objective evidence to support a submission are built and collected along the way, eliminating the last-minute scramble (and resulting delays) associated with dealing with process debt at the end of a program.
Cutting Corners: A Time Savings Myth
Looking at regulation as a burden sets up a mindset bent toward trying to minimize effort, inevitably leading to cutting corners. Manufacturers often delay activities that are thought of as “regulatory documentation” until the end of the development effort or skip important steps deemed “not applicable” due to not fully understanding or appreciating the intent of the regulatory guidance.The time savings gained by this thinking is illusory. The work still needs to be done and our experience working with hundreds of device manufacturers and OEMs over many years has shown that, like fixing defects, fixing UI design issues gets dramatically more costly as time goes on and resources are invested in implementing those designs. Valuable information that could have steered the development effort toward early convergence and efficiency is instead coming too late in the process to have a positive effect on the outcome. Discovering this kind of information late in a program converts potential value into additional, preventable costs.
Beyond the development schedule, deprioritizing HFA activities can lead to issues and deficiencies in a regulatory submission, ultimately setting back the timeline for a product reaching the market by months or possibly even years. These kinds of delays are a problem for both device manufacturers and patients. Device manufacturers face increased costs, lost market windows, and damaged reputations. Patients are denied timely access to innovative devices that could improve their health. Even after a device gets to market, the costs resulting from recalls and associated liabilities can dwarf entire R&D budgets.
Summary
The FDA guidance and the associated industry standards for human factors and usability provides a comprehensive roadmap to an efficient and successful UI/UX design effort while reducing both safety and business risks for medical device development. Viewing this guidance as a burden creates the risk of losing the inherent value built into the guidance from the lessons learned and experience gained from years of medical device development.Effective application of the approaches to risk assessment and human factors assessment captured in the regulations provides a roadmap to high-value alignment of your device with your users’ needs. This alignment is a key element of a predictable path to safe and effective devices, and minimizes both safety risk and business risk.
References
Dr. Bruce Johnston has more than 25 years of experience designing and developing complex software systems and over 15 years of experience working in regulated medical software development. Much of this experience has been focused on UI design and human factors aspects of Class II and Class III medical device programs.