Dawn A. Lissy, Empirical09.01.17
I’m a huge fan of lists. Covering every base with an organized way to track everything from my companies’ financial progress to whether the Doritos made it into the car for the drive home from school keeps me functioning. I note things on my computer, tablet, smartphone, spiral paper planner, and a series of Post-it notes that wing up around my office like neon butterflies.
It would never occur to me to not have a test plan. But I frequently find myself explaining the benefits of a plan to clients and dispelling a series of misunderstandings about how a test plan can make the development cycle easier and less expensive. I’ve even had clients who have never heard of such a thing and wonder why anyone would ever craft one.
I have special list just for them.
“Less expensive” typically catches everyone’s attention when explaining the benefits of devising a test plan. It varies from client to client, but some have saved as much as 30 percent of overall testing time and financial costs, depending on the type and scope of project. A testing plan solid enough to be the foundation of a successful project yet fluid enough to allow for course corrections eases the path to market—particularly for Class III devices.
“Easier” is a subjective measurement, however; there’s not necessarily a bottom line or hard number attached to it. But based on what I’ve noticed over the course of my career, it’s better to form a test plan at the beginning of the development cycle. If you wait until mid-cycle, a challenge that could’ve been foreseen might sideline the project and measures must then be put in place to deal with it.
The Why
Clients have argued a test plan could limit their options and they don’t want to feel pigeon-holed or wedded to a single path. But after considering the options and plotting the course, identify where and when that plan may need reassessment. The plan will not be carved in stone. Hopefully it won’t manifest in fluorescent paper slips all over the office, but rather a happy medium that allows forward momentum and won’t trap you in a corner.
Plan Before You Plan
A failure modes and effects analysis (FMEA) is a risk analysis and starting point. This is a must for Class III devices as well as Class II devices containing a feature with the potential to cause harm, such as an intervertebral body fusion device (IBFD) with a new locking mechanism for expansion purposes. The FMEA will determine any additional testing to consider in order to prove the overall safety and efficacy of the device—benchtop testing, animal testing, or other evaluations needed in the overall test plan. For Class III devices, you’ll also want to consider feasibility testing. Again, this may apply for Class IIs with breakthrough technology applications.
Now the actual plan is ready to be developed. Feel free to dust off my article “Driving a Successful Medical Device Product Development Cycle” in the July/August 2015 edition of MPO for more details on putting that plan together, but it should include minimum required tests from relevant regulatory agencies, in addition to feasibility or other evaluations identified during the risk analysis.
More Than Guidance Documents
Some clients have argued that they don’t need a plan because there are regulatory documents that spell out required tests. While established guidelines and regulatory direction are great fodder for a plan, they’re not an actual process guide for a device. The U.S. Food and Drug Administration’s guidance documents serve as the foundation and minimum requirements, but they’re not a device-specific path to market.
It’s easy to over- or under-test if those documents aren’t read correctly. Some devices such as vertebral body replacements or cervical or lumbar IBFDs can be tested to up to six different methods per the American Society for Testing and Materials (ASTM) standards. But not every device needs all six tests depending on its indication and location. Only two of six tests have to be performed on a monolithic lumbar device. Knowing the regulatory requirements for the specific kind of device helps to put the best plan in place to use time, money, and testing wisely. With Class III devices, there may not be a standard to test a device to, thereby requiring a test plan to propose to the regulatory agency.
I’ve actually had face-palm moments when a client informs me they performed all six tests when they only needed four. They covered their bases mid-process because they didn’t have a plan, which cost them time and money they didn’t need to spend.
Let’s also consider who you’re working with. The regulatory community is full of planners. Anyone who reports to a governmental agency—regardless of what country it’s in—has a series of boxes to check. It will be easier on everyone if a document to share with reviewers at any point in the cycle is available.
Putting It in Writing
When developing a test plan, you’re making a map—creating a document that will enable the entire team to literally be on the same page. This is where you’re doing homework and will see potential liabilities as well as possible shortcuts. Once this critical baseline is obtained, a realistic timeline and the bookends of possibilities can be developed. Best-case to worse-case scenarios and where/when those issues can pop up will be known.
With this outline of major deadlines and turning points, you’ll be able to allocate resources and refine schedules to smooth the journey.
Everyone will have the deadlines, everyone will get the same information, and the whole team will work off the same document. Given the number and variety of people that must work together to get a medical device to market—regulatory, manufacturing, production development, surgeons, etc.—this is essential. If everyone understands what the steps are and that those steps can’t be skipped, precious time will not be wasted. It’s ultimately about setting and managing stakeholder expectations.
Bringing the Future to the Present
Alan Lakein, author of “How to Get Control of Your Time and Your Life,” sums it up nicely: “Planning is bringing the future into the present so you can do something about it now.”
Consider the future of the device, the team, and your goals. Then bring all of that into the present with a test plan that will move the endeavor forward to meet those goals.
Dawn Lissy is a biomedical engineer, entrepreneur, and innovator. Since 1998, the Empirical family of companies (Empirical Testing Corp., Empirical Consulting, LLC, and Empirical Machine, LLC) has operated under Lissy’s direction. Empirical offers the full range of regulatory and quality systems consulting, testing, small batch and prototype manufacturing, and validations services to bring a medical device to market. Empirical is very active within standards development organization ASTM International and has one of the widest scopes of test methods of any accredited independent lab in the United States. Because Lissy was a member of the U.S. Food and Drug Administration’s Entrepreneur-in-Residence program, she has first-hand, in-depth knowledge of the regulatory landscape. Lissy holds an inventor patent for the Stackable Cage System for corpectomy and vertebrectomy. Her M.S. in biomedical engineering is from The University of Akron, Ohio.
It would never occur to me to not have a test plan. But I frequently find myself explaining the benefits of a plan to clients and dispelling a series of misunderstandings about how a test plan can make the development cycle easier and less expensive. I’ve even had clients who have never heard of such a thing and wonder why anyone would ever craft one.
I have special list just for them.
“Less expensive” typically catches everyone’s attention when explaining the benefits of devising a test plan. It varies from client to client, but some have saved as much as 30 percent of overall testing time and financial costs, depending on the type and scope of project. A testing plan solid enough to be the foundation of a successful project yet fluid enough to allow for course corrections eases the path to market—particularly for Class III devices.
“Easier” is a subjective measurement, however; there’s not necessarily a bottom line or hard number attached to it. But based on what I’ve noticed over the course of my career, it’s better to form a test plan at the beginning of the development cycle. If you wait until mid-cycle, a challenge that could’ve been foreseen might sideline the project and measures must then be put in place to deal with it.
The Why
Clients have argued a test plan could limit their options and they don’t want to feel pigeon-holed or wedded to a single path. But after considering the options and plotting the course, identify where and when that plan may need reassessment. The plan will not be carved in stone. Hopefully it won’t manifest in fluorescent paper slips all over the office, but rather a happy medium that allows forward momentum and won’t trap you in a corner.
Plan Before You Plan
A failure modes and effects analysis (FMEA) is a risk analysis and starting point. This is a must for Class III devices as well as Class II devices containing a feature with the potential to cause harm, such as an intervertebral body fusion device (IBFD) with a new locking mechanism for expansion purposes. The FMEA will determine any additional testing to consider in order to prove the overall safety and efficacy of the device—benchtop testing, animal testing, or other evaluations needed in the overall test plan. For Class III devices, you’ll also want to consider feasibility testing. Again, this may apply for Class IIs with breakthrough technology applications.
Now the actual plan is ready to be developed. Feel free to dust off my article “Driving a Successful Medical Device Product Development Cycle” in the July/August 2015 edition of MPO for more details on putting that plan together, but it should include minimum required tests from relevant regulatory agencies, in addition to feasibility or other evaluations identified during the risk analysis.
More Than Guidance Documents
Some clients have argued that they don’t need a plan because there are regulatory documents that spell out required tests. While established guidelines and regulatory direction are great fodder for a plan, they’re not an actual process guide for a device. The U.S. Food and Drug Administration’s guidance documents serve as the foundation and minimum requirements, but they’re not a device-specific path to market.
It’s easy to over- or under-test if those documents aren’t read correctly. Some devices such as vertebral body replacements or cervical or lumbar IBFDs can be tested to up to six different methods per the American Society for Testing and Materials (ASTM) standards. But not every device needs all six tests depending on its indication and location. Only two of six tests have to be performed on a monolithic lumbar device. Knowing the regulatory requirements for the specific kind of device helps to put the best plan in place to use time, money, and testing wisely. With Class III devices, there may not be a standard to test a device to, thereby requiring a test plan to propose to the regulatory agency.
I’ve actually had face-palm moments when a client informs me they performed all six tests when they only needed four. They covered their bases mid-process because they didn’t have a plan, which cost them time and money they didn’t need to spend.
Let’s also consider who you’re working with. The regulatory community is full of planners. Anyone who reports to a governmental agency—regardless of what country it’s in—has a series of boxes to check. It will be easier on everyone if a document to share with reviewers at any point in the cycle is available.
Putting It in Writing
When developing a test plan, you’re making a map—creating a document that will enable the entire team to literally be on the same page. This is where you’re doing homework and will see potential liabilities as well as possible shortcuts. Once this critical baseline is obtained, a realistic timeline and the bookends of possibilities can be developed. Best-case to worse-case scenarios and where/when those issues can pop up will be known.
With this outline of major deadlines and turning points, you’ll be able to allocate resources and refine schedules to smooth the journey.
Everyone will have the deadlines, everyone will get the same information, and the whole team will work off the same document. Given the number and variety of people that must work together to get a medical device to market—regulatory, manufacturing, production development, surgeons, etc.—this is essential. If everyone understands what the steps are and that those steps can’t be skipped, precious time will not be wasted. It’s ultimately about setting and managing stakeholder expectations.
Bringing the Future to the Present
Alan Lakein, author of “How to Get Control of Your Time and Your Life,” sums it up nicely: “Planning is bringing the future into the present so you can do something about it now.”
Consider the future of the device, the team, and your goals. Then bring all of that into the present with a test plan that will move the endeavor forward to meet those goals.
Dawn Lissy is a biomedical engineer, entrepreneur, and innovator. Since 1998, the Empirical family of companies (Empirical Testing Corp., Empirical Consulting, LLC, and Empirical Machine, LLC) has operated under Lissy’s direction. Empirical offers the full range of regulatory and quality systems consulting, testing, small batch and prototype manufacturing, and validations services to bring a medical device to market. Empirical is very active within standards development organization ASTM International and has one of the widest scopes of test methods of any accredited independent lab in the United States. Because Lissy was a member of the U.S. Food and Drug Administration’s Entrepreneur-in-Residence program, she has first-hand, in-depth knowledge of the regulatory landscape. Lissy holds an inventor patent for the Stackable Cage System for corpectomy and vertebrectomy. Her M.S. in biomedical engineering is from The University of Akron, Ohio.
