Meredith P. Vanderbilt, JD, RAC, CQA, MSE, BSE, Director of Consulting, Empirical04.04.24
As a medical device consultant doing both regulatory submissions and quality audits, I’ve had thousands of conversations with employees from manufacturing personnel to company executives. Rarely is there much enthusiasm for design verification activities required for new or modified products. This process is arguably one of the most critical on many levels, and should be a primary focus for any product development team.
According to 21 CFR 820.30(f): “Each manufacturer shall establish and maintain procedures for verifying the device design. Design verification shall confirm that the design output meets the design input requirements. The results of the design verification, including identification of the design, method(s), the date, and the individual(s) performing the verification, shall be documented in the DHF.”
A junior design engineer might ask “What is design verification?” According to 21 CFR 820.3(aa), “verification means confirmation by examination and provision of objective evidence that specified requirements have been fulfilled.” That seems like a word salad, so what does it mean in human terms? According to Merriam-Webster dictionary, it’s defined as “the act or process of verifying: the state of being verified.” That’s not very helpful, so we dig deeper to define “verify” as “to give evidence or testimony to the truth or factualness of.” Now we’re getting somewhere, but what are we proving the factualness of and how? Those answers are bigger than a breadbox, as my grandmother used to say, which is why the industry spends millions of dollars annually on verification activities and training. The technical details of verification requirements are beyond the scope of this article but the importance of a repeatable verification process is not.
Design verification is a testing or inspection activity that involves evaluating the actual or simulated product to ensure it meets a specific design requirement. The simplest version of this is that the device should be 15mm in length, so a verification would involve checking the drawing to confirm the specified length and measuring some of the prototypes and first production parts. A more complicated example is that the device should withstand mechanical forces anticipated in the human body. This verification activity includes researching forces in the body and adding a safety factor to develop acceptance criteria before executing a specifically designed mechanical test battery appropriate for characterizing the strength of the device compared to that acceptance criteria.
Piece of cake, right?
Circling back to medical devices, design verification and validation activities are where the rubber hits the road because innumerable designs have seemed good on paper but couldn’t weather the feasibility or design verification process to make it to market. From a legal perspective, the design verification activities justify allowing the product’s use in human patients. Failure to adequately verify a design can lead to patient harm and business risk.
When failures occur in the market, they impact the company with the failing devices and the industry as a whole. A 2012 article in Consumer Reports—Dangerous medical implants and devices: Most implants have never been tested for safety—made the inflammatory (and factually incorrect) statement, “For most implants and other high-risk devices brought to market, manufacturers do nothing more than file some paperwork and pay the Food and Drug Administration a user fee of roughly $4,000 to start selling a product that can rack up many millions of dollars in revenue. Often, the only safety ‘testing’ that occurs is in the bodies of unsuspecting patients—including two of the three people whose stories are told in this report.” The media perspective was that no testing was completed because clinical trials (design validation) weren’t needed to prove the device’s safety, according to FDA regulations. I cannot express the importance that our design verification activities be robust enough to reduce the chances of product failures by objectively testing all applicable design inputs and potential risks identified in the development process.
Who is responsible for design verification? The short answer: everyone on the project team and any outside entities used to support or perform verification activities. Everyone.
Each department has ideas about measurable and/or observable design inputs critical to the success of the product and project. These ideas are best merged through active discussions instead of routing a form for team members to complete. Design control procedure should include a process to bring the applicable departments and people together to create these inputs and develop a design verification plan.
Once the inputs are generated and approved, they must each be evaluated for a verification method. An example of a biocompatibility input might be, “finished device shall have no unacceptable adverse biological response resulting from contact of the component materials of the device with the body.” The accepted method of verification for this input can be found in FDA’s guidance document “Use of International Standard ISO 10993-1, ‘Biological evaluation of medical devices - Part 1: Evaluation and testing within a risk management process:’ Guidance for Industry and Food and Drug Administration Staff.” It’s rare for any medical device company to have an internal lab equipped to perform ISO 10993 testing or internal resources with the education and experience to perform a complete biocompatibility assessment. This testing/assessment is generally outsourced to a third-party lab.
On Feb. 20, 2024, the FDA released a letter to industry warning that “an increasing number of entities that contract with device firms to conduct testing on medical devices (‘third-party test labs’) are generating testing data that are fabricated, duplicated from other device submissions, or otherwise unreliable.” This should scare us all because it’s a crucial aspect of design verification; the responsible organization must have valid design verification documentation. For third-party testing, a knowledgeable team member in the company should review the reports for accuracy and, ideally, write a summary that draws conclusions based on testing results.
A good design verification process starts with valid, measurable, or observable inputs. The testing should be completed using approved protocols that include valid methods and justifiable acceptance criteria. Final, approved reports must include analysis of any deviations or failures and draw reasonable conclusions based on data generated during testing.
On May 19, 2023, Steiner Biotechnology LLC received a warning letter because its design control procedures only included identification of design inputs and did “not include procedures for design output, design reviews, design verification, design validation, design transfer, or changes, as required by 21 CFR 820.30(d) through (i).” Additionally, the FDA noted “the records you provided…do not show these studies were conducted using the OsseoConduct device. Additionally, the histology data provided does not include the study protocol or methods used to generate the data.” A design verification process without approved protocols with valid methods and acceptance criteria as well as final, approved reports that draw reasonable conclusions based on data generated during testing can lead to regulatory repercussions, as in this case.
In the previous example of a biocompatibility input of a “finished device shall have no unacceptable adverse biological response resulting from contact of the component materials of the device with the body,” patient infection is a strong possibility without completion of design verification testing. Patient infection can cause everything from a surgical site infection treated with antibiotics to implant loss and even death. Poor execution or lack of verification testing can lead directly to these outcomes.
Meredith P. Vanderbilt is an internationally known medical device regulatory affairs consultant unafraid to communicate directly and honestly with regulatory bodies and clients about strategies and submissions to provide compliant and high-quality devices to the market. She is an industry influencer through workshops, networking, presentations, articles, textbook chapters, and (coming soon) an orthopedic industry book about additive manufacturing. She has decades of experience in performing and managing medical device development, commercialization, and marketing activities. She has a high success rate for gaining regulatory clearances in relatively short periods of time and is a participant in all aspects of quality management systems and post-marketing activities.
What Is Design Verification?
The statutory requirement for these activities includes three simple, logical sentences, but executing these activities can be as expensive, time-consuming, and daunting as they are important.According to 21 CFR 820.30(f): “Each manufacturer shall establish and maintain procedures for verifying the device design. Design verification shall confirm that the design output meets the design input requirements. The results of the design verification, including identification of the design, method(s), the date, and the individual(s) performing the verification, shall be documented in the DHF.”
A junior design engineer might ask “What is design verification?” According to 21 CFR 820.3(aa), “verification means confirmation by examination and provision of objective evidence that specified requirements have been fulfilled.” That seems like a word salad, so what does it mean in human terms? According to Merriam-Webster dictionary, it’s defined as “the act or process of verifying: the state of being verified.” That’s not very helpful, so we dig deeper to define “verify” as “to give evidence or testimony to the truth or factualness of.” Now we’re getting somewhere, but what are we proving the factualness of and how? Those answers are bigger than a breadbox, as my grandmother used to say, which is why the industry spends millions of dollars annually on verification activities and training. The technical details of verification requirements are beyond the scope of this article but the importance of a repeatable verification process is not.
Design verification is a testing or inspection activity that involves evaluating the actual or simulated product to ensure it meets a specific design requirement. The simplest version of this is that the device should be 15mm in length, so a verification would involve checking the drawing to confirm the specified length and measuring some of the prototypes and first production parts. A more complicated example is that the device should withstand mechanical forces anticipated in the human body. This verification activity includes researching forces in the body and adding a safety factor to develop acceptance criteria before executing a specifically designed mechanical test battery appropriate for characterizing the strength of the device compared to that acceptance criteria.
Piece of cake, right?
Why Do We Need Design Verification?
I argue design verification is the most important area of design controls because it’s the basis for our substantiation of safety and (sometimes) effectiveness. To broaden our conversation beyond medical devices, we will reach back over 120 years to the development of the airplane. On Sept. 18, 1901, Wilbur Wright addressed a distinguished group of Chicago engineers on the subject of “some aeronautical experiments” he had conducted with his brother Orville over the previous two years. “The difficulties which obstruct the pathway to success in flying machine construction,” he noted, “are of three general classes:- Those which relate to the construction of the sustaining wings.
- Those which relate to the generation and application of the power required to drive the machine through the air.
- Those relating to the balancing and steering of the machine after it is actually in flight.”
Circling back to medical devices, design verification and validation activities are where the rubber hits the road because innumerable designs have seemed good on paper but couldn’t weather the feasibility or design verification process to make it to market. From a legal perspective, the design verification activities justify allowing the product’s use in human patients. Failure to adequately verify a design can lead to patient harm and business risk.
When failures occur in the market, they impact the company with the failing devices and the industry as a whole. A 2012 article in Consumer Reports—Dangerous medical implants and devices: Most implants have never been tested for safety—made the inflammatory (and factually incorrect) statement, “For most implants and other high-risk devices brought to market, manufacturers do nothing more than file some paperwork and pay the Food and Drug Administration a user fee of roughly $4,000 to start selling a product that can rack up many millions of dollars in revenue. Often, the only safety ‘testing’ that occurs is in the bodies of unsuspecting patients—including two of the three people whose stories are told in this report.” The media perspective was that no testing was completed because clinical trials (design validation) weren’t needed to prove the device’s safety, according to FDA regulations. I cannot express the importance that our design verification activities be robust enough to reduce the chances of product failures by objectively testing all applicable design inputs and potential risks identified in the development process.
Who is responsible for design verification? The short answer: everyone on the project team and any outside entities used to support or perform verification activities. Everyone.
Each department has ideas about measurable and/or observable design inputs critical to the success of the product and project. These ideas are best merged through active discussions instead of routing a form for team members to complete. Design control procedure should include a process to bring the applicable departments and people together to create these inputs and develop a design verification plan.
Once the inputs are generated and approved, they must each be evaluated for a verification method. An example of a biocompatibility input might be, “finished device shall have no unacceptable adverse biological response resulting from contact of the component materials of the device with the body.” The accepted method of verification for this input can be found in FDA’s guidance document “Use of International Standard ISO 10993-1, ‘Biological evaluation of medical devices - Part 1: Evaluation and testing within a risk management process:’ Guidance for Industry and Food and Drug Administration Staff.” It’s rare for any medical device company to have an internal lab equipped to perform ISO 10993 testing or internal resources with the education and experience to perform a complete biocompatibility assessment. This testing/assessment is generally outsourced to a third-party lab.
On Feb. 20, 2024, the FDA released a letter to industry warning that “an increasing number of entities that contract with device firms to conduct testing on medical devices (‘third-party test labs’) are generating testing data that are fabricated, duplicated from other device submissions, or otherwise unreliable.” This should scare us all because it’s a crucial aspect of design verification; the responsible organization must have valid design verification documentation. For third-party testing, a knowledgeable team member in the company should review the reports for accuracy and, ideally, write a summary that draws conclusions based on testing results.
A good design verification process starts with valid, measurable, or observable inputs. The testing should be completed using approved protocols that include valid methods and justifiable acceptance criteria. Final, approved reports must include analysis of any deviations or failures and draw reasonable conclusions based on data generated during testing.
What Happens if the Design Verification Process Is Inadequate?
If the Wright brothers had advertised their airplane as safe to fly before completing all their experiments, injury, death, or lawsuits could have happened. The same potential outcomes apply to medical device companies.On May 19, 2023, Steiner Biotechnology LLC received a warning letter because its design control procedures only included identification of design inputs and did “not include procedures for design output, design reviews, design verification, design validation, design transfer, or changes, as required by 21 CFR 820.30(d) through (i).” Additionally, the FDA noted “the records you provided…do not show these studies were conducted using the OsseoConduct device. Additionally, the histology data provided does not include the study protocol or methods used to generate the data.” A design verification process without approved protocols with valid methods and acceptance criteria as well as final, approved reports that draw reasonable conclusions based on data generated during testing can lead to regulatory repercussions, as in this case.
In the previous example of a biocompatibility input of a “finished device shall have no unacceptable adverse biological response resulting from contact of the component materials of the device with the body,” patient infection is a strong possibility without completion of design verification testing. Patient infection can cause everything from a surgical site infection treated with antibiotics to implant loss and even death. Poor execution or lack of verification testing can lead directly to these outcomes.
Meredith P. Vanderbilt is an internationally known medical device regulatory affairs consultant unafraid to communicate directly and honestly with regulatory bodies and clients about strategies and submissions to provide compliant and high-quality devices to the market. She is an industry influencer through workshops, networking, presentations, articles, textbook chapters, and (coming soon) an orthopedic industry book about additive manufacturing. She has decades of experience in performing and managing medical device development, commercialization, and marketing activities. She has a high success rate for gaining regulatory clearances in relatively short periods of time and is a participant in all aspects of quality management systems and post-marketing activities.