James Shore, Director of Quality, Titan Medical Inc. and John Freije, Principal Consultant and Owner, Freije Quality Engineering LLC10.12.16
The international standard ISO 13485:2016 for Medical Devices quality management systems was published in March 2016. One of the key changes included the “strengthening of supplier control processes” to be more harmonized with the U.S. Food and Drug Administration’s (FDA) 21 CFR 820.50 Purchasing controls. Now, OEMs need to ensure all supplier quality-related procedures are compliant. This article will provide the necessary information to perform a gap analysis, as well as identify areas for making existing systems more robust. It will also provide a tool that can be used to assist in the reduction of this potential risk.
Historical Information of FDA Warning Letter Citing
Purchasing controls have always been among the areas of focus by the FDA. An analysis from 2004 to 2014 shows, on average, 40 percent of all warning letters (WL) cite inadequate purchasing controls (PC) (Graph 1). With the recent changes being more harmonized with 21 CFR 820.50 and the increase in outsourcing, supplier management will continue to be an area of focus for regulatory bodies.
Graph 2 provides a breakdown of specific areas that are being cited within the warning letters. By focusing on the areas most cited, one could expect to reduce occurrence and risk involved with noncompliance.
ISO 13485:2016 Revision—Major Changes to Supplier Quality
Fortunately, the changes made to the ISO standard are more harmonized with 21 CFR 820.50. For those companies that understand and comply with the quality system regulation (QSR), this transition should be a straightforward, easy gap analysis to complete.
The major changes made in this ISO standard revision were made to the three areas under sub-clause 7.4:
Following is a comparison of the ISO standard and the FDA QSR regarding the purchasing process.
ISO 13485, Para 7.4.1
Gap Assessment
One way to identify, evaluate, and monitor suppliers is to create a category matrix that will associate the risk levels for each type of supplier to the product or service received. It’s also recommended to have the minimum level of controls within this matrix, as well as the suppliers not requiring any controls (Table 1). The types and extent of controls based on the individual assessment of each supplier shall be documented. The matrix should be the minimum requirements that shall be met, and if a supplier that does not meet minimum requirements is to be used, additional controls are needed.
7.4.2 Purchasing Information
Following is the comparison of the ISO standard and the FDA QSReg regarding purchasing information.
ISO 13485, Para 7.4.2
Gap Assessment
Generic supplier quality agreement documents are a good starting point, but ensure it reflects what is required for that specific supplier and situation—one size does not fit all. The agreements should cover the scope of work that the supplier is providing and the regulations that are required.
7.4.3 Verification of Purchased Product
Following is the comparison of the ISO standard and the FDA QSR regarding the verification of purchased product/receiving acceptance activities.
ISO 13485, Para 7.4.3
The other major change to the ISO standard is the requirement to assess the impact of risk when changes are made. Again, this is not new for the medical device industry following 21 CFR 820; as part of design controls, the manufacturer shall establish and maintain procedures for the identification, documentation, validation, or where appropriate, verification, review, and approval of design changes before their implementation.
Gap Assessment
Throughout all of these supplier quality elements, the need to identify risk and assess suppliers has been a common theme. In order to overcome the challenges of comparing one supplier to another, one needs a tool that can be used to assess key quality and business elements. One tool developed by the authors is the Total Risk Factor (TRF).2
The value of conducting this risk assessment is not only to identify high risk suppliers, but to mitigate the risk they may impose on you. For example, if a supplier’s financial situation is not stable, they may not be able to pay for services critical to the product or obtain raw material in time to meet the production schedule. If the supplier has cash flow issues and is on credit hold with its suppliers, items such as calibration services could be impacted (pushed out or not performed at all).
There are many factors that can be considered, but for this example, five of the most common elements are reviewed.
Severity of the product provided—The most effective method would be to use the Design FMEA severity rating. If that isn’t possible, consider using the commodity category and assign a severity for each type.
Supplier’s quality system detection—A scale of 1 through 5 can be used to assign the level based on the supplier’s ability to detect non-conforming product, last quality audit, or their quality performance for the past 18 months. A “1” would be assigned for a supplier that has had a very good performance or a great audit report and a “5” for a supplier with poor performance or a poor audit report.
Financial stress factor—This factor looks at the supplier financial stress score that can be easily obtained through Dunn & Bradstreet.3 The financial stress factor predicts the likelihood of business failure over the next 12 months. The rating uses “five distinct risk groups where a one (1) represents businesses that have the lowest probability of financial stress, and a five (5) represents businesses with the highest probability of financial stress.”
Lead time—This factor looks at the supplier’s ability to meet the required delivery date. This is not to be confused with the promise date or the date the supplier agreed to ship. If the product is required on Tuesday but the supplier can’t ship it until Friday, that’s not aiding with the management of internal operations. Using a scale of 1 through 5, assign a value based on supplier performance. If “need date” is not in the system, use the supplier’s delivery performance for the past 18 months.
Order capacity—Order capacity refers to how much of the supplier’s business a company will have in annual revenue. The rule of thumb—it should never be more than 20 percent of anyone’s business—should be considered, but may not always apply. This may be the case with a company to have greater focus from a supplier, particularly for critical components or with a finished medical device being outsourced to a contract manufacturer. The scale 1 through 5 can be applied based on the percentage of the supplier’s annual revenue with a company.
Regardless of which factors are selected, it is important to be consistent with the values and how suppliers are rated. Data integrity is critical. Also, keep the scale small and well defined so there is less opportunity for different departments to disagree with which level is assigned to a supplier. If there is not enough discrimination of the data to make good decisions, then select one or two factors and experiment with the results.
Putting the TRF into Action
Table 2 offers an example of three suppliers being evaluated to determine their TRF levels. Once the values are assigned to each attribute, the totals are summed. The system is set up to keep the calculations as simple as possible while providing the most information. For most cases, this simple approach should answer 90 percent of all supplier risk evaluations.
This basic approach will help clarify the bigger picture with minimal effort to reach it. It’s obvious that supplier B is the best choice based on the TRF value. This is based, however, on the premise that each factor is equally important. This is one of the weaknesses of systems that are one-dimensional.
The Weighted Approach
If the basic approach accomplishes the evaluation, no further action is required. With a small investment of time, however, better information can be gained to make a more reliable decision. The TRF uses a weighted approach based on the premise that not all factors are equally important. The advantage of the weighted approach is that the value can be changed to fit a company’s needs, so it provides not only structure but also flexibility.
The TRF weighted approach (Table 3) takes the same suppliers and attribute ratings but adds the weighted factor for each attribute. One size does not fit all, so it will be up to a company to determine the individual weighted factors. The calculation is the attribute multiplied by the weighted value, resulting in the risk. All of the risk values are summed together to determine the TRF levels.
When used to evaluate a company’s current supplier base, threshold values should be set to determine further action. For example, a monthly report of suppliers could be reviewed and any supplier with a TRF value over 25 would require documented investigation and possible corrective action. Any supplier below that value, however, would not require any action (and documented investigation). This approach focuses resources on the higher risk suppliers, rather than trying to cover all of them.
Another use of this TRF model is for supplier selection. In this example, the direction would be to select the supplier with the lowest TRF value. In this example, supplier B would be the first choice, followed by supplier A, and then C. Supplier B would be the primary supplier while Supplier A may be considered as the backup or secondary supplier.
Conclusion
This article provided the necessary tools to perform an internal gap assessment of a company’s procedures against the ISO changes. In addition, higher level approaches to evaluate the overall supplier risk profile and ideas to reduce them were presented.
The tools should be modified to match a company’s risk profile and business practices. The TRF models are very flexible and offer a great start to evaluate business risks, as well as to avoid unnecessary noncompliance reports from an ISO registrar and external regulators.
References
1 “Analysis of Warning Letters, 2004 to 2014 for Purchasing Controls Citing,” Courtesy of Freije Engineering LLC. PC cites include both 820.50 and 820.80 (b). Source of information: FDA’s Electronic Freedom of Information Reading Room for Warning Letters and Responses. Accuracy ±4% based on last posted Transparency Report by FDA.
2 “Proactive Supplier Management in the Medical Device Industry,” Quality Press, 2016. James Shore and John Freije.
3 http://bit.ly/mpo161004
James Shore is the director of quality and regulatory affairs at Titan Medical Inc., a developer of advanced robotic surgical technologies such as the SPORT Surgical System. He has 25 years of quality and supplier management experience working in medical devices, semiconductor, aerospace, and defense (Lake Region, Nypro Healthcare, Boston Scientific, Aspect Medical, ACMI, Brooks Automation, and Raytheon). His professional certifications include ASQ Certified Six Sigma Black Belt, ASQ Certified Quality Manager/Operations Excellence, Certified Quality Auditor, and Certified Mechanical Inspector and ASQ Senior Member. Shore is also a certified welding inspector from AWS and obtained the Lean Bronze certification from the Association for Manufacturing Excellence. He is a veteran of Operation Desert Storm (1991), having served in the United States Marine Corps for more than 15 years and was honorably discharged at the rank of Gunnery Sergeant (E-7).
John Freije is the principal consultant and owner of Freije Quality Engineering LLC, which he started in 2007. In this capacity, he works with medical device companies and suppliers to develop and implement quality systems exceeding the expectations of ISO 13485 and FDA’s Quality System Regulation 21 CFR 820. Freije has been in the workforce for more than 34 years with over 23 in the medical device and pharmaceutical industries maintaining a primary focus on supplier development. His experience includes support of quality systems and various product lines at Roche Diagnostics; projects for the Special Forces and Direct Fire Control at Raytheon Technical Services Company; and various engineering and quality positions with increasing responsibilities at Eli Lilly and Company supporting medical device commercialization, quality, and manufacturing. He was onorably discharged at the rank of Staff Sergeant (E-6) after serving eight years in the U.S. Army as a field artillery systems mechanic. Freije is the 2016 Chair of the ASQ Biomedical Division and is a certified quality engineer (CQE).
Historical Information of FDA Warning Letter Citing
Purchasing controls have always been among the areas of focus by the FDA. An analysis from 2004 to 2014 shows, on average, 40 percent of all warning letters (WL) cite inadequate purchasing controls (PC) (Graph 1). With the recent changes being more harmonized with 21 CFR 820.50 and the increase in outsourcing, supplier management will continue to be an area of focus for regulatory bodies.
Graph 2 provides a breakdown of specific areas that are being cited within the warning letters. By focusing on the areas most cited, one could expect to reduce occurrence and risk involved with noncompliance.
ISO 13485:2016 Revision—Major Changes to Supplier Quality
Fortunately, the changes made to the ISO standard are more harmonized with 21 CFR 820.50. For those companies that understand and comply with the quality system regulation (QSR), this transition should be a straightforward, easy gap analysis to complete.
The major changes made in this ISO standard revision were made to the three areas under sub-clause 7.4:
- 7.4.1 Purchasing Process
- 7.4.2 Purchasing Information
- 7.4.3 Verification of Purchased Product
Following is a comparison of the ISO standard and the FDA QSR regarding the purchasing process.
ISO 13485, Para 7.4.1
- Criteria for evaluation and selection of suppliers include performance and risk.
- Performance monitoring as part of re-evaluation process
-
Each manufacturer shall establish and maintain procedures to ensure that all purchased or otherwise received product and services conform to specified requirements.
-
(a) Evaluation of suppliers, contractors, and consultants. Each manufacturer shall establish and maintain the requirements, including quality requirements, that must be met by suppliers, contractors, and consultants.
- (1) Evaluate and select potential suppliers, contractors, and consultants...on ability…
- (3) Establish and maintain records of acceptable suppliers, contractors, and consultants.
-
(a) Evaluation of suppliers, contractors, and consultants. Each manufacturer shall establish and maintain the requirements, including quality requirements, that must be met by suppliers, contractors, and consultants.
Gap Assessment
- Does your company use risk levels and performance in selecting suppliers?
- Does your company monitor performance of your suppliers at regular intervals? If so, what are the limits and what action is taken if those limits are exceeded?
- How is this documented?
One way to identify, evaluate, and monitor suppliers is to create a category matrix that will associate the risk levels for each type of supplier to the product or service received. It’s also recommended to have the minimum level of controls within this matrix, as well as the suppliers not requiring any controls (Table 1). The types and extent of controls based on the individual assessment of each supplier shall be documented. The matrix should be the minimum requirements that shall be met, and if a supplier that does not meet minimum requirements is to be used, additional controls are needed.
7.4.2 Purchasing Information
Following is the comparison of the ISO standard and the FDA QSReg regarding purchasing information.
ISO 13485, Para 7.4.2
- Information shall describe or reference the product to be purchased…Product specifications, requirements for acceptance, requirements for personnel, QMS
- Communication to the supplier, written agreement regarding change notification, as applicable
- Each manufacturer shall establish and maintain data that clearly describe or reference the specified requirements, including quality requirements…
- Purchasing documents shall include, where possible, an agreement…to notify the manufacturer of changes…so that manufacturers may determine whether the changes may affect the quality of a finished device.
Gap Assessment
- Do you have clear quality requirements including acceptance methods documented and are they shared with your suppliers?
- Do you have SQAs in place for your suppliers or at least an SQA to notify you of any changes to the product or service?
- Do your agreements include notification of change approval prior to making those changes?
- If your product has CE mark approval, do your critical supplier SQA documents include the unannounced audit requirements?
Generic supplier quality agreement documents are a good starting point, but ensure it reflects what is required for that specific supplier and situation—one size does not fit all. The agreements should cover the scope of work that the supplier is providing and the regulations that are required.
7.4.3 Verification of Purchased Product
Following is the comparison of the ISO standard and the FDA QSR regarding the verification of purchased product/receiving acceptance activities.
ISO 13485, Para 7.4.3
- Establish and implement the inspection…for ensuring that purchased product meets specified purchase requirements.
- Extent of activity is based on supplier evaluation and related to risk.
- If changes occur, the changes are assessed.
- Each manufacturer shall establish and maintain procedures for acceptance of incoming product. Incoming product shall be inspected, tested, or otherwise verified as conforming to specified requirements.
- Acceptance or rejection shall be documented…shall document acceptance activities required by this part.
The other major change to the ISO standard is the requirement to assess the impact of risk when changes are made. Again, this is not new for the medical device industry following 21 CFR 820; as part of design controls, the manufacturer shall establish and maintain procedures for the identification, documentation, validation, or where appropriate, verification, review, and approval of design changes before their implementation.
Gap Assessment
- Do you have a method to identify which items require inspection? Are they clearly identified in a risk matrix such as Table 1?
- For those inspection plans, is the inspection level based on the risk identified in the Risk Management File, such as the Design FMEA or Hazard Analysis?
- Does your supplier quality agreement clearly state how suppliers are supposed to communicate changes to you?
- Do you have procedures in place to assess supplier changes and how are they handled through your inspection process? Do these procedures address if your supplier refuses to sign an agreement with you or provide you with changes prior to the change taking place and receipt of product?
Throughout all of these supplier quality elements, the need to identify risk and assess suppliers has been a common theme. In order to overcome the challenges of comparing one supplier to another, one needs a tool that can be used to assess key quality and business elements. One tool developed by the authors is the Total Risk Factor (TRF).2
The value of conducting this risk assessment is not only to identify high risk suppliers, but to mitigate the risk they may impose on you. For example, if a supplier’s financial situation is not stable, they may not be able to pay for services critical to the product or obtain raw material in time to meet the production schedule. If the supplier has cash flow issues and is on credit hold with its suppliers, items such as calibration services could be impacted (pushed out or not performed at all).
There are many factors that can be considered, but for this example, five of the most common elements are reviewed.
Severity of the product provided—The most effective method would be to use the Design FMEA severity rating. If that isn’t possible, consider using the commodity category and assign a severity for each type.
Supplier’s quality system detection—A scale of 1 through 5 can be used to assign the level based on the supplier’s ability to detect non-conforming product, last quality audit, or their quality performance for the past 18 months. A “1” would be assigned for a supplier that has had a very good performance or a great audit report and a “5” for a supplier with poor performance or a poor audit report.
Financial stress factor—This factor looks at the supplier financial stress score that can be easily obtained through Dunn & Bradstreet.3 The financial stress factor predicts the likelihood of business failure over the next 12 months. The rating uses “five distinct risk groups where a one (1) represents businesses that have the lowest probability of financial stress, and a five (5) represents businesses with the highest probability of financial stress.”
Lead time—This factor looks at the supplier’s ability to meet the required delivery date. This is not to be confused with the promise date or the date the supplier agreed to ship. If the product is required on Tuesday but the supplier can’t ship it until Friday, that’s not aiding with the management of internal operations. Using a scale of 1 through 5, assign a value based on supplier performance. If “need date” is not in the system, use the supplier’s delivery performance for the past 18 months.
Order capacity—Order capacity refers to how much of the supplier’s business a company will have in annual revenue. The rule of thumb—it should never be more than 20 percent of anyone’s business—should be considered, but may not always apply. This may be the case with a company to have greater focus from a supplier, particularly for critical components or with a finished medical device being outsourced to a contract manufacturer. The scale 1 through 5 can be applied based on the percentage of the supplier’s annual revenue with a company.
Regardless of which factors are selected, it is important to be consistent with the values and how suppliers are rated. Data integrity is critical. Also, keep the scale small and well defined so there is less opportunity for different departments to disagree with which level is assigned to a supplier. If there is not enough discrimination of the data to make good decisions, then select one or two factors and experiment with the results.
Putting the TRF into Action
Table 2 offers an example of three suppliers being evaluated to determine their TRF levels. Once the values are assigned to each attribute, the totals are summed. The system is set up to keep the calculations as simple as possible while providing the most information. For most cases, this simple approach should answer 90 percent of all supplier risk evaluations.
This basic approach will help clarify the bigger picture with minimal effort to reach it. It’s obvious that supplier B is the best choice based on the TRF value. This is based, however, on the premise that each factor is equally important. This is one of the weaknesses of systems that are one-dimensional.
The Weighted Approach
If the basic approach accomplishes the evaluation, no further action is required. With a small investment of time, however, better information can be gained to make a more reliable decision. The TRF uses a weighted approach based on the premise that not all factors are equally important. The advantage of the weighted approach is that the value can be changed to fit a company’s needs, so it provides not only structure but also flexibility.
The TRF weighted approach (Table 3) takes the same suppliers and attribute ratings but adds the weighted factor for each attribute. One size does not fit all, so it will be up to a company to determine the individual weighted factors. The calculation is the attribute multiplied by the weighted value, resulting in the risk. All of the risk values are summed together to determine the TRF levels.
When used to evaluate a company’s current supplier base, threshold values should be set to determine further action. For example, a monthly report of suppliers could be reviewed and any supplier with a TRF value over 25 would require documented investigation and possible corrective action. Any supplier below that value, however, would not require any action (and documented investigation). This approach focuses resources on the higher risk suppliers, rather than trying to cover all of them.
Another use of this TRF model is for supplier selection. In this example, the direction would be to select the supplier with the lowest TRF value. In this example, supplier B would be the first choice, followed by supplier A, and then C. Supplier B would be the primary supplier while Supplier A may be considered as the backup or secondary supplier.
Conclusion
This article provided the necessary tools to perform an internal gap assessment of a company’s procedures against the ISO changes. In addition, higher level approaches to evaluate the overall supplier risk profile and ideas to reduce them were presented.
The tools should be modified to match a company’s risk profile and business practices. The TRF models are very flexible and offer a great start to evaluate business risks, as well as to avoid unnecessary noncompliance reports from an ISO registrar and external regulators.
References
1 “Analysis of Warning Letters, 2004 to 2014 for Purchasing Controls Citing,” Courtesy of Freije Engineering LLC. PC cites include both 820.50 and 820.80 (b). Source of information: FDA’s Electronic Freedom of Information Reading Room for Warning Letters and Responses. Accuracy ±4% based on last posted Transparency Report by FDA.
2 “Proactive Supplier Management in the Medical Device Industry,” Quality Press, 2016. James Shore and John Freije.
3 http://bit.ly/mpo161004
James Shore is the director of quality and regulatory affairs at Titan Medical Inc., a developer of advanced robotic surgical technologies such as the SPORT Surgical System. He has 25 years of quality and supplier management experience working in medical devices, semiconductor, aerospace, and defense (Lake Region, Nypro Healthcare, Boston Scientific, Aspect Medical, ACMI, Brooks Automation, and Raytheon). His professional certifications include ASQ Certified Six Sigma Black Belt, ASQ Certified Quality Manager/Operations Excellence, Certified Quality Auditor, and Certified Mechanical Inspector and ASQ Senior Member. Shore is also a certified welding inspector from AWS and obtained the Lean Bronze certification from the Association for Manufacturing Excellence. He is a veteran of Operation Desert Storm (1991), having served in the United States Marine Corps for more than 15 years and was honorably discharged at the rank of Gunnery Sergeant (E-7).
John Freije is the principal consultant and owner of Freije Quality Engineering LLC, which he started in 2007. In this capacity, he works with medical device companies and suppliers to develop and implement quality systems exceeding the expectations of ISO 13485 and FDA’s Quality System Regulation 21 CFR 820. Freije has been in the workforce for more than 34 years with over 23 in the medical device and pharmaceutical industries maintaining a primary focus on supplier development. His experience includes support of quality systems and various product lines at Roche Diagnostics; projects for the Special Forces and Direct Fire Control at Raytheon Technical Services Company; and various engineering and quality positions with increasing responsibilities at Eli Lilly and Company supporting medical device commercialization, quality, and manufacturing. He was onorably discharged at the rank of Staff Sergeant (E-6) after serving eight years in the U.S. Army as a field artillery systems mechanic. Freije is the 2016 Chair of the ASQ Biomedical Division and is a certified quality engineer (CQE).