Erik Swain09.13.10
Early Planning, Future Reward
Proper product development involves taking proactive steps to avoid pitfalls that may come when potential risks are not adequately considered.
In the medical device product development process, the most risk lies at the beginning, where mistakes and omissions can be made that fester into large and expensive problems as the product gets closer to market, and even after it is introduced.
Risk assessment and risk management are processes often associated with the later stages of product development, but they need to be employed in the earlier stages as well, so potential problems can be addressed before a design is finalized, after which changes take longer and cost more to make.
Medical device OEMs that outsource all or part of product development need to work with their partners to ensure that these early-stage pitfalls are avoided. Luckily, there are a number of contract developers that are experienced enough to know how to do that.
What follows is a discussion of some of the most common mistakes that lead to unnecessary early-stage risk, and some tips for ensuring that they are not made.
Common Mistakes
Firms shouldn’t be making as many potential mistakes in early product development as they used to, assuming that they’re following a process that’s compliant with U.S. Food and Drug Adminstration (FDA) quality system practices, said Paul Mulhauser, president of Factors NY, a New York, N.Y.-based design and development company with an emphasis on human factors. This is because most of the more common pitfalls have been well-publicized, and FDA has been quicker to step in when it senses that there are design and use problems, he says.
However, he notes that sometimes OEMs “tend to focus on addressing the manufacturing needs but fall short in addressing user needs upfront. They don’t understand that the process starts and ends with the user and that you have to identify how the product will be used and optimize functionality and performance.”
The FDA, as well as groups such as Advanced Medical Technology Association and the Association for the Advancement of Medical Instrumentation have been getting the word out that user testing is a required part of product development, but some firms still think of it as a luxury expense, says Reade Harpharm, manager of human-centric design at Battelle’s Medical Device and Diagnostics Group. Battelle, based in Columbus, Ohio, is the world’s largest not-for-profit research and development organization.
“You must have an understanding of how your product is going to be used and made safe,” said Harpharm. “Sometimes the safest design is going to be ugly. That drives engineers nuts, but that just has to be what it is.”
Putting off the decision to green-light the project can be a source of unnecessary risk also.
“A common mistake is not making the decision quickly enough to go forward with the project,” said Ron Sully, director of marketing for Omnica, an Irvine, Calif., firm that specializes in product development for complex medical devices. He explains that firms often do not adjust their timetables for getting the product to the market when they delay their decisions about moving forward. That leads to an accelerated development schedule that cuts corners and can result in an inferior product—or a delayed introduction when fixes must be made late in the process.
Another issue that creates risk is picking the wrong development partner because of a misunderstanding about the scope of the project, Sully said. “If you get two bids, but one is more than 25 percent less than the other, someone is not understanding the scope of the project. That’s a red flag,” he explained.
He adds that while manufacturing considerations should not be overlooked, they also should not trump everything when it comes to choosing a development partner.
“Designing and manufacturing cannot coexist under the umbrella of product development,” Sully said. “They are two separate disciplines that don’t go together. Contract manufacturers might be able to give you a lower bid to do product development because they get money from the manufacturing contract at the end. But there may be conflicts between what is best for the client and the needs of the contract manufacturer. They perform services that are extensions of their primary business and have a tendency to design what they can build.”
Also, he noted that those firms may have expertise in manufacturing support engineering but not principal engineering. By contrast, he added, product development firms are not constrained to specific manufacturing techniques.
Sometimes firms run into trouble when they have not developed an appropriate proof of therapy, said Keith Checca, director of business development for Cirtec Medical Systems, a development and manufacturing company based in East Longmeadow, Mass. “They’ve got the technology they want to employ, but don’t fully understand how to prove the therapy. As a contract organization, we need direction from them on how to test it.”
The problem is that some firms want to apply technology for technology’s sake, says Sean MacLeod, president of Seattle, Wash.-based Stratos Product Development. “This means they are not doing product definition, they do not know the unmet medical needs, they do not know the user needs, and they do not know the use case,” he said.
Others, according to MacLeod, don’t want to go through iterations of the device but want to jump right in to product development. “Building on the fly doesn’t work in medical devices,” he said. “At the end of the day, you will have to back up and do it all again anyway. This is ultimately a highly iterative process. Trying to do development in one pass is shortsighted. When you do an iteration, you can see the risks firsthand, and they can be mitigated in the next iteration.”
Doing too many iterations also can be a problem, because that can tie up resources on something that might have to be changed later on anyway, said Chuck Brower, president of Aubrey Group, Inc., and Jahnavi Lokre, director of software engineering and marketing. Aubrey Group is a product development and contract manufacturing firm based in Irvine, Calif.
“One mistake is trying to do too much in the first generation of the product; after the product is in the field, customers will offer feedback on what should be improved, and it often is not what the OEM anticipates during the development phase,” they said.
Failure to consider reimbursement issues creates financial risk. Small OEMs are prone to not thinking through reimbursement strategies before proceeding with product development, according to Ron Podraza, CEO of Reimbursement Principles. His firm, based in Littleton, Colo., does product development with a focus on reimbursement strategies and gathering clinical evidence.
“The problem is the thinking that because something is a better version of a product that’s already out there, you can just get money for it,” he said. “But their customers may not be able to recoup some of the funds for the premium price of the device.”
Indeed, there are a number of perspectives that must be incorporated into early discussions and failure to do so risks the creation of an unsafe or ineffective product, said MacLeod.
“Too many times, we have seen firms that do not have a fully integrated team, so R&D is doing product development without the clinical, regulatory, and reimbursement people involved,” he said. “If you are missing a voice that needs to be there, you are in trouble. Even if it’s only one voice, that will be the area that bites you. There is no replacement for an experienced team that works well together. You want a team that understands when pitfalls might arise and knows how to avoid them. A dysfunctional team adds risk to the process. You have to communicate as formally and informally as you can and as much as you can.”
Addressing the Risks
There are a number of steps that can be taken and procedures that can be followed to ensure that these problems are much less likely to arise.
Before product development can begin, product definition must occur, says MacLeod. “That is where you define market requirements, do voice-of-the-customer and user-needs research, review intellectual property, and evaluate the technology for commercialization,” he said. “This informs both the start of and the results of product development.”
Potential risks should be identified at the very beginning, according to Brower and Lokre.
“We help the client identify the risks in product development, prioritize them and then develop solutions to address the specific risks,” they said. “The successful outcome of this stage leads to the start of the product development process. Sometimes it can change the course of development for the project to a path that was not anticipated earlier.”
An essential component of a good process is to have all product requirements properly delineated from the beginning.
“A good statement of requirements is necessary,” Sully said. “If we have that, we can find out exactly what they need, and have in-depth conversations from there.”
Checca agreed. “We see so many requirements documents that are hurried,” he says. “It can cause rough relationships with clients when they are not clear on the requirements.”
The product requirements and specifications, along with the risk analysis, should be discussed at the beginning of the project, said Brower and Lokre. Defining the requirements of the product is very important to developing a project plan and ensuring that the development process runs smoothly and efficiently, they said. Another thing that should be discussed early is the technology development risk. For example, is enough of the technology developed and understood to enable the team to engineer a product that meets the desired specifications? If there are areas where the path to engineering reliable product features is not clear, it may be advisable to first engage in a concept or feasibility development phase, the duo noted. Doing some early work on the key engineering unknowns will make it easier to plan for and carry out the development of the final product. This also will minimize the risk of spending time and money on development paths that later need to be abandoned due to critical design changes, they said.
Sully noted that “sometimes companies come to us with a core technology or intellectual property and want us to make it into a medical device, but they’re not sure if it will work as a medical device. So we do a lot of feasibility studies, and they have to be part of the scope of the project.”
Similar planning and communication must happen throughout the project, said Mulhauser. “The first activity is that you have to lay out a plan that includes all steps from preparation of product for everything from the development specifications through to the production implementation,” he said. “You must fully understand the market needs, the user needs, and the technology before you ever start designing.”
Richard Grant, manager of medical devices for Invetech, a product development firm based in Melbourne, Australia, and San Diego, Calif., agreed.
“We have a comprehensive system definition process that lays the groundwork for a successful development by drawing out both stated and unstated requirements and embodying these in a comprehensive requirements document and ensuring that the development team understands totally what success looks like to the client,” he explained. “We also use a range of risk analysis tools and a risk-based development approach that tests high-risk elements of the design/product early in the development to ensure the practicality of the product and maximum progress for the development dollar. An early prototyping and testing program is key to achieving this risk reduction.”
Realistic projections for time and cost need to be part of this process, added MacLeod. That includes creative time. “There needs to be a discussion about how much time should be spent to do something innovative,” he said. “If you don’t do that, you’re not creating value. But you have to put a product on the market, so you have to determine at what point you pick up your pencil and move on to full development.”
Podraza says that one of the first steps should be figuring out who will use the product and how they will get paid for it. “If you understand those two things, then you will have a product with a positive value proposition for the provider,” and that leads to market success, he said. “It’s necessary to work with physicians in development, but you can’t lose sight that the hospital is the one that pays for it. So what is the interplay of those two in the reimbursement scenario?”
Therefore, he said, it helps to perform a baseline study that assesses the reimbursement outlook for the product. That can help determine early on whether the product needs to be modified, whether it might be used in different medical scenarios than was previously anticipated, and whether clinical trials need to be redesigned, he noted.
Understanding Use Issues
OEMs that skimp on or omit user research and use testing in the early stages of development are adding tremendous risk to their products. Medical devices that are designed without regard to how they will be used in the field run the risk of being unsafe and/or ineffective.
“For surgical products, interview surgeons and key opinion leaders. Observe how they are used in the [operating room], noting gaps between what’s said and what’s actually done,” Mulhauser explained. “Your research should be broad enough to give sufficient answers, and not just be tailored to a few opinion leaders, though.”
The design also should be sufficiently flexible to work effectively and safely even if it is being misused or used in an unintended manner, he added.
The importance of user input cannot be understated, said Harpharm. “I have seen some of the best technologies fall flat on their face because users were not considered early enough in the development process,” he said.
It is essential to understand the environment of use, he emphasized. Who will be using the product and for what purpose? What are the expected outcomes from the use of the device?
“You have to understand the workflow of the procedure and how the product impacts it,” he added. This information is best obtained before the formal steps guided by design controls get underway. “Once the design is defined, it is harder to see what needs to be done from the user perspective,” he said. “Try to get that input in as early as possible to reduce the risk management process downstream. It helps get the team and the requirements aligned moving forward. Risk reduction and human factors go hand in hand.”
Indeed, said MacLeod, the development team should tend to risk assessment through product requirements. “Risk assessment is not a one-time or distinct event. It is a living action throughout the process,” he said. “The team should constantly be going back to review all properties and requirements, including the assumptions that the program was based on.”
Not locking in to a design too early is another way to mitigate risk, according to Grant.
“If budget allows, running two or more parallel development approaches—maturing multiple device concepts in parallel—can help minimize risk,” he said.
Taking steps such as these may cost time early on, but failing to do them may cost time later on if FDA has objections to the design or last-minute design changes are needed, said Checca. “If the right approach is a 15-month timeframe, the client may say they want it done in nine months, but after three delays, it’s 15 months anyway,” he said. And it is much less expensive to make changes early on in the process than later on.
The timetable also must reflect an accurate assessment of when the product will become profitable, Podraza noted. “For truly novel products, regardless of the clinical value they provide, it will take a few years before reimbursement becomes routine and automatic,” he said. “People betting on these products need to have a realistic timeframe as to when the market is going to ramp up.”
For start-up companies, that means recruiting investors who understand that it may take novel products more than a year on the market before they become profitable.
Risks addressed early also can help prevent further risks that can occur after product introduction, when slight design changes might be made to improve manufacturability or use.
“It is best to stay in touch with the people who created the product with you,” said Mulhauser. “Should cumulative in-process changes go awry (or be desired), five years down the road, we easily assure the changes maintain original intent. That information is in the Design History File, but can be tedious to extract.”
Medical device OEMs need to keep in mind that product development is not merely the path from getting an idea to going to market and making money. It also involves extensive research and testing to prove that the product is safe and effective. If these aspects of product development are not considered, or if the wrong outsourcing partner is chosen, then there are substantial risks to patients, users, and the company.
“At the end of the day, mitigating risks early on will increase product safety and reduce adverse events,” said MacLeod. “Investment in proper development today pays off in spades later on.”
Erik Swain is a freelance writer based in Phillipsburg, N.J. He has covered the medical device industry for 12 years.
Proper product development involves taking proactive steps to avoid pitfalls that may come when potential risks are not adequately considered.
In the medical device product development process, the most risk lies at the beginning, where mistakes and omissions can be made that fester into large and expensive problems as the product gets closer to market, and even after it is introduced.
Risk assessment and risk management are processes often associated with the later stages of product development, but they need to be employed in the earlier stages as well, so potential problems can be addressed before a design is finalized, after which changes take longer and cost more to make.
Medical device OEMs that outsource all or part of product development need to work with their partners to ensure that these early-stage pitfalls are avoided. Luckily, there are a number of contract developers that are experienced enough to know how to do that.
What follows is a discussion of some of the most common mistakes that lead to unnecessary early-stage risk, and some tips for ensuring that they are not made.
Common Mistakes
Firms shouldn’t be making as many potential mistakes in early product development as they used to, assuming that they’re following a process that’s compliant with U.S. Food and Drug Adminstration (FDA) quality system practices, said Paul Mulhauser, president of Factors NY, a New York, N.Y.-based design and development company with an emphasis on human factors. This is because most of the more common pitfalls have been well-publicized, and FDA has been quicker to step in when it senses that there are design and use problems, he says.
However, he notes that sometimes OEMs “tend to focus on addressing the manufacturing needs but fall short in addressing user needs upfront. They don’t understand that the process starts and ends with the user and that you have to identify how the product will be used and optimize functionality and performance.”
The FDA, as well as groups such as Advanced Medical Technology Association and the Association for the Advancement of Medical Instrumentation have been getting the word out that user testing is a required part of product development, but some firms still think of it as a luxury expense, says Reade Harpharm, manager of human-centric design at Battelle’s Medical Device and Diagnostics Group. Battelle, based in Columbus, Ohio, is the world’s largest not-for-profit research and development organization.
“You must have an understanding of how your product is going to be used and made safe,” said Harpharm. “Sometimes the safest design is going to be ugly. That drives engineers nuts, but that just has to be what it is.”
Putting off the decision to green-light the project can be a source of unnecessary risk also.
“A common mistake is not making the decision quickly enough to go forward with the project,” said Ron Sully, director of marketing for Omnica, an Irvine, Calif., firm that specializes in product development for complex medical devices. He explains that firms often do not adjust their timetables for getting the product to the market when they delay their decisions about moving forward. That leads to an accelerated development schedule that cuts corners and can result in an inferior product—or a delayed introduction when fixes must be made late in the process.
Another issue that creates risk is picking the wrong development partner because of a misunderstanding about the scope of the project, Sully said. “If you get two bids, but one is more than 25 percent less than the other, someone is not understanding the scope of the project. That’s a red flag,” he explained.
He adds that while manufacturing considerations should not be overlooked, they also should not trump everything when it comes to choosing a development partner.
“Designing and manufacturing cannot coexist under the umbrella of product development,” Sully said. “They are two separate disciplines that don’t go together. Contract manufacturers might be able to give you a lower bid to do product development because they get money from the manufacturing contract at the end. But there may be conflicts between what is best for the client and the needs of the contract manufacturer. They perform services that are extensions of their primary business and have a tendency to design what they can build.”
Also, he noted that those firms may have expertise in manufacturing support engineering but not principal engineering. By contrast, he added, product development firms are not constrained to specific manufacturing techniques.
Sometimes firms run into trouble when they have not developed an appropriate proof of therapy, said Keith Checca, director of business development for Cirtec Medical Systems, a development and manufacturing company based in East Longmeadow, Mass. “They’ve got the technology they want to employ, but don’t fully understand how to prove the therapy. As a contract organization, we need direction from them on how to test it.”
The problem is that some firms want to apply technology for technology’s sake, says Sean MacLeod, president of Seattle, Wash.-based Stratos Product Development. “This means they are not doing product definition, they do not know the unmet medical needs, they do not know the user needs, and they do not know the use case,” he said.
Others, according to MacLeod, don’t want to go through iterations of the device but want to jump right in to product development. “Building on the fly doesn’t work in medical devices,” he said. “At the end of the day, you will have to back up and do it all again anyway. This is ultimately a highly iterative process. Trying to do development in one pass is shortsighted. When you do an iteration, you can see the risks firsthand, and they can be mitigated in the next iteration.”
Doing too many iterations also can be a problem, because that can tie up resources on something that might have to be changed later on anyway, said Chuck Brower, president of Aubrey Group, Inc., and Jahnavi Lokre, director of software engineering and marketing. Aubrey Group is a product development and contract manufacturing firm based in Irvine, Calif.
“One mistake is trying to do too much in the first generation of the product; after the product is in the field, customers will offer feedback on what should be improved, and it often is not what the OEM anticipates during the development phase,” they said.
Failure to consider reimbursement issues creates financial risk. Small OEMs are prone to not thinking through reimbursement strategies before proceeding with product development, according to Ron Podraza, CEO of Reimbursement Principles. His firm, based in Littleton, Colo., does product development with a focus on reimbursement strategies and gathering clinical evidence.
“The problem is the thinking that because something is a better version of a product that’s already out there, you can just get money for it,” he said. “But their customers may not be able to recoup some of the funds for the premium price of the device.”
Indeed, there are a number of perspectives that must be incorporated into early discussions and failure to do so risks the creation of an unsafe or ineffective product, said MacLeod.
“Too many times, we have seen firms that do not have a fully integrated team, so R&D is doing product development without the clinical, regulatory, and reimbursement people involved,” he said. “If you are missing a voice that needs to be there, you are in trouble. Even if it’s only one voice, that will be the area that bites you. There is no replacement for an experienced team that works well together. You want a team that understands when pitfalls might arise and knows how to avoid them. A dysfunctional team adds risk to the process. You have to communicate as formally and informally as you can and as much as you can.”
Addressing the Risks
There are a number of steps that can be taken and procedures that can be followed to ensure that these problems are much less likely to arise.
Before product development can begin, product definition must occur, says MacLeod. “That is where you define market requirements, do voice-of-the-customer and user-needs research, review intellectual property, and evaluate the technology for commercialization,” he said. “This informs both the start of and the results of product development.”
Potential risks should be identified at the very beginning, according to Brower and Lokre.
“We help the client identify the risks in product development, prioritize them and then develop solutions to address the specific risks,” they said. “The successful outcome of this stage leads to the start of the product development process. Sometimes it can change the course of development for the project to a path that was not anticipated earlier.”
An essential component of a good process is to have all product requirements properly delineated from the beginning.
“A good statement of requirements is necessary,” Sully said. “If we have that, we can find out exactly what they need, and have in-depth conversations from there.”
Checca agreed. “We see so many requirements documents that are hurried,” he says. “It can cause rough relationships with clients when they are not clear on the requirements.”
The product requirements and specifications, along with the risk analysis, should be discussed at the beginning of the project, said Brower and Lokre. Defining the requirements of the product is very important to developing a project plan and ensuring that the development process runs smoothly and efficiently, they said. Another thing that should be discussed early is the technology development risk. For example, is enough of the technology developed and understood to enable the team to engineer a product that meets the desired specifications? If there are areas where the path to engineering reliable product features is not clear, it may be advisable to first engage in a concept or feasibility development phase, the duo noted. Doing some early work on the key engineering unknowns will make it easier to plan for and carry out the development of the final product. This also will minimize the risk of spending time and money on development paths that later need to be abandoned due to critical design changes, they said.
Sully noted that “sometimes companies come to us with a core technology or intellectual property and want us to make it into a medical device, but they’re not sure if it will work as a medical device. So we do a lot of feasibility studies, and they have to be part of the scope of the project.”
Similar planning and communication must happen throughout the project, said Mulhauser. “The first activity is that you have to lay out a plan that includes all steps from preparation of product for everything from the development specifications through to the production implementation,” he said. “You must fully understand the market needs, the user needs, and the technology before you ever start designing.”
Richard Grant, manager of medical devices for Invetech, a product development firm based in Melbourne, Australia, and San Diego, Calif., agreed.
“We have a comprehensive system definition process that lays the groundwork for a successful development by drawing out both stated and unstated requirements and embodying these in a comprehensive requirements document and ensuring that the development team understands totally what success looks like to the client,” he explained. “We also use a range of risk analysis tools and a risk-based development approach that tests high-risk elements of the design/product early in the development to ensure the practicality of the product and maximum progress for the development dollar. An early prototyping and testing program is key to achieving this risk reduction.”
Realistic projections for time and cost need to be part of this process, added MacLeod. That includes creative time. “There needs to be a discussion about how much time should be spent to do something innovative,” he said. “If you don’t do that, you’re not creating value. But you have to put a product on the market, so you have to determine at what point you pick up your pencil and move on to full development.”
Podraza says that one of the first steps should be figuring out who will use the product and how they will get paid for it. “If you understand those two things, then you will have a product with a positive value proposition for the provider,” and that leads to market success, he said. “It’s necessary to work with physicians in development, but you can’t lose sight that the hospital is the one that pays for it. So what is the interplay of those two in the reimbursement scenario?”
Therefore, he said, it helps to perform a baseline study that assesses the reimbursement outlook for the product. That can help determine early on whether the product needs to be modified, whether it might be used in different medical scenarios than was previously anticipated, and whether clinical trials need to be redesigned, he noted.
Understanding Use Issues
OEMs that skimp on or omit user research and use testing in the early stages of development are adding tremendous risk to their products. Medical devices that are designed without regard to how they will be used in the field run the risk of being unsafe and/or ineffective.
“For surgical products, interview surgeons and key opinion leaders. Observe how they are used in the [operating room], noting gaps between what’s said and what’s actually done,” Mulhauser explained. “Your research should be broad enough to give sufficient answers, and not just be tailored to a few opinion leaders, though.”
The design also should be sufficiently flexible to work effectively and safely even if it is being misused or used in an unintended manner, he added.
The importance of user input cannot be understated, said Harpharm. “I have seen some of the best technologies fall flat on their face because users were not considered early enough in the development process,” he said.
It is essential to understand the environment of use, he emphasized. Who will be using the product and for what purpose? What are the expected outcomes from the use of the device?
“You have to understand the workflow of the procedure and how the product impacts it,” he added. This information is best obtained before the formal steps guided by design controls get underway. “Once the design is defined, it is harder to see what needs to be done from the user perspective,” he said. “Try to get that input in as early as possible to reduce the risk management process downstream. It helps get the team and the requirements aligned moving forward. Risk reduction and human factors go hand in hand.”
Indeed, said MacLeod, the development team should tend to risk assessment through product requirements. “Risk assessment is not a one-time or distinct event. It is a living action throughout the process,” he said. “The team should constantly be going back to review all properties and requirements, including the assumptions that the program was based on.”
Not locking in to a design too early is another way to mitigate risk, according to Grant.
“If budget allows, running two or more parallel development approaches—maturing multiple device concepts in parallel—can help minimize risk,” he said.
Taking steps such as these may cost time early on, but failing to do them may cost time later on if FDA has objections to the design or last-minute design changes are needed, said Checca. “If the right approach is a 15-month timeframe, the client may say they want it done in nine months, but after three delays, it’s 15 months anyway,” he said. And it is much less expensive to make changes early on in the process than later on.
The timetable also must reflect an accurate assessment of when the product will become profitable, Podraza noted. “For truly novel products, regardless of the clinical value they provide, it will take a few years before reimbursement becomes routine and automatic,” he said. “People betting on these products need to have a realistic timeframe as to when the market is going to ramp up.”
For start-up companies, that means recruiting investors who understand that it may take novel products more than a year on the market before they become profitable.
Risks addressed early also can help prevent further risks that can occur after product introduction, when slight design changes might be made to improve manufacturability or use.
“It is best to stay in touch with the people who created the product with you,” said Mulhauser. “Should cumulative in-process changes go awry (or be desired), five years down the road, we easily assure the changes maintain original intent. That information is in the Design History File, but can be tedious to extract.”
Medical device OEMs need to keep in mind that product development is not merely the path from getting an idea to going to market and making money. It also involves extensive research and testing to prove that the product is safe and effective. If these aspects of product development are not considered, or if the wrong outsourcing partner is chosen, then there are substantial risks to patients, users, and the company.
“At the end of the day, mitigating risks early on will increase product safety and reduce adverse events,” said MacLeod. “Investment in proper development today pays off in spades later on.”
Erik Swain is a freelance writer based in Phillipsburg, N.J. He has covered the medical device industry for 12 years.