A Critical Look
Conducting a thorough evaluation of a supplier can help OEMs reduce risk and increase their competitive advantage in the marketplace.
Michael Barbella, Managing Editor
The most serious mistakes are not being made as a result of wrong answers. The truly dangerous thing is asking the wrong question.
— Peter F. Drucker
Most legends are born in death. A few, however, find renown within their lifetimes.
Peter F. Drucker was one of those rare legends. Well before his death in 2005, Drucker had been crowned the inventor of modern business management. Though he moved fluidly among roles as journalist, professor, historian, economics commentator and raconteur, Drucker was perhaps best known for his business acumen.
Much of Drucker’s success can be attributed to his keen observation skills and his ability to find patterns among seemingly unconnected disciplines. A BusinessWeek cover story that was published shortly after his death described Drucker’s mind as “an itinerant thing, able to wander in minutes through a series of digressions until finally coming to some specific business point.”
Finding the right outsourcing partner is not an easy task. Companies can achieve significant performance improvements by awarding additional business to suppliers that consistently meet their goals. |
In the course of his career, Drucker helped countless numbers of managers—including some of the most celebrated CEOs of his era—focus on the opportunities at their respective companies rather than existing problems. He did that in a very simple way, too—by asking questions.
“My job,” he once told a consulting client, “is to ask questions. It’s your job to provide answers.”
While Drucker’s approach was brusque, his direct delineation of duties is not limited to managers in any specific industry. The breakdown is certainly applicable to the medical device field, where CEOs and quality control directors are constantly striving to improve business processes and better manage the supply base.
Sourcing directors, in particular, would be well-served by Drucker’s penchant for questions (and his ability to learn by listening). These directors usually are charged with finding the right outsourcing partner for their company, a job that can be both daunting and revealing for the device firm and the partner.
Finding the right outsourcing partner, however, is not an easy task. It is particularly difficult at original equipment manufacturers (OEMs), where sourcing directors can choose from tens of thousands of potential suitors. Most companies that face such a vast selection of suppliers (and those that have far fewer choices) whittle down their options through an evaluation system designed to produce a long-lasting and profitable partnership.
“The evaluation process provides a data-driven basis for selecting the best fit supplier for a project or product,” said Joe Kelly, a director in the general manufacturing division of Tefen USA, an international consulting firm with U.S. headquarters in New York City.“When I’m looking at choosing a partner, I have to remember that a piece of that partner is going to end up in the DNA of my finished product. I have to be very confident that the piece of that supplier that ends up in my finished product is not going to harm anyone or my company’s reputation. That’s really the root of it. Certainly we want good performance but it’s really about making sure we maintain the integrity of the finished product and the integrity of our good name.”
The Motivating Factors Behind Evaluations
While integrity certainly is important to the product development process, it is not the only reason companies evaluate vendors. It also is mandated by law.
Title 21 of the U.S. Food and Drug Administration’s (FDA) Code of Federal Regulations (CFR) Part 820.50a requires manufacturers to “evaluate and select potential suppliers, contractors, and consultants on the basis of their ability to meet specified requirements, including quality requirements.” Evaluations must be documented.
The FDA also requires companies to define the type and extent of control they will exercise over the product, services, suppliers, contractors, and consultants, based on evaluation results. In addition, manufacturers must create and maintain records of acceptable suppliers, 21 CFR Part 820.50a states.
Manufacturers are held to similar standards by the Global Harmonization Task Force (GHTF). A guidance document issued by the GHTF in December 2008 recommends that companies assess the business and operational capabilities of potential vendors to ensure top quality, performance and reliability. “A potential supplier’s business conduct, practices, reputation and financial viability may provide useful information about the business capabilities of that supplier,” the GHTF document states. “The operational capability should be investigated to determine whether the supplier is able or willing to adapt and respond to performance indicators required by the manufacturer, such as lead times, on-time delivery, response time, etc.”
The GHTF guidance document is not compulsory for manufacturers. Nevertheless, FDA officials consider it a good reference guide for evaluating and validating suppliers. “It’s not the law,” noted Melissa Torres, biomedical engineer in the Office of Compliance at the FDA’s Center for Devices and Radiological Health. “The law is the [21 CFR Part 820.50]. The GHTF guidance is a good guideline for companies to use. It’s more of a suggested way of doing things.”
Whether suggested or mandated, vendor evaluation is an integral part of the outsourcing process. Not only can evaluations improve visibility into supplier performance, they also can help companies reduce risk and increase their competitive advantage, industry experts told Medical Product Outsourcing.
Improving suppliers’ performance visibility can help device firms better manage their supply chain. Plus, assessing vendors can help drive performance improvement—suppliers that are required to meet specific performance goals, experts noted, usually will meet them and try to surpass them. Companies can achieve significant performance improvements by awarding additional business to suppliers that consistently meet their goals.
Vendor evaluations also can help remove hidden waste and cost drivers in the supply chain. Some waste is caused by poor communication between device manufacturers and their outsourcing partners, while other inefficiencies stem from bad business practices at the supplier that result in quality problems, increased inventory, and slow deliveries. Assessments of supplier quality can help device companies remove the costs associated with inspections and supplier non-conformances, industry insiders explained.
“Regardless of what the actual documents and regulations say, you have to have a certain level of a quality system in order to make a business decision,” said Steven Walfish, president of Statistical Outsourcing Services, a Rockville, Md.-based consulting firm that provides statistical analysis and training to FDA-regulated industries. “Companies have to look and say to themselves ‘this particular supplier is not delivering value to me. I’m paying too much or I’m spending too much to evaluate their product, or the quality of their product is not very good, so I have to conduct more inspections and that is costing me money.’ There are a lot of reasons companies should evaluate vendors above and beyond the regulations.”
One of those reasons is the insight manufacturers can gain about better leveraging their supply base. Companies that understand their suppliers’ capabilities and levels of performance can better plan new products and services, industry experts said. Such an understanding, for example, can help manufacturers determine whether they should outsource a particular product or service or make it themselves.
A thorough understanding of a vendor’s capabilities not only helps improve the quality of a manufacturer’s products and services, it also can lead to a better alignment of practices between the two entities. Organizations that share the same business ethics are much more likely to have a lasting partnership than those with conflicting outlooks, sourcing directors noted.
“You want to make sure the businesses are aligned properly,” said Benjamin G. Harrison, president and CEO of Kaysun Corporation, a designer and manufacturer of plastic injection-molded products and assemblies based in Manitowoc, Wis. “You have to be selective in who you choose to partner with because you’re going to live with the consequences if you choose the wrong companies to supply. The mentality of ‘as long as the money is green we’re happy’ is the road to ruin because you clearly won’t have the type of control and focus that you need to grow a business profitably.”
Evaluation Methods
Many of the managers and CEOs who worked with Drucker during the course of their careers recall not only his wisdom but his unusual consulting style. Dan W. Lufkin, a co-founder of Wall Street investment firm Donaldson Lufkin & Jenrette, encountered Drucker’s unconventional style during their first meeting in the early 1960s.
“I asked him if he thought we should sell a certain product or do a certain strategy, but all he said was ‘I don’t know’ to every question I posed,” Lufkin told The Wall Street Journal after Drucker’s death. “So finally I asked, ‘what am I hiring you for?’“
Drucker’s reply was direct and simple: “I’m not going to give you any answers, because there are always many different ways to approach problems, but I’m going to give you the questions you should ask.”
Lufkin was lucky. Medical device manufacturers have no such guru to consult for suggestions on the kinds of questions they should ask suppliers. Had he been asked though, Drucker most likely would have answered such a query from device firms with a question of his own: “What kind of information do you want from the supplier?”
Deciphering the answer to that question can help companies determine the best way to evaluate vendors. Numerous factors, such as knowledge of the industry, the talent of staff members and the type of services offered, can have an impact on the evaluation process. Experience can play an important role as well. New vendors, for example, usually are subjected to more scrutiny than well-established suppliers that have experience working with OEMs.
“One of the things companies must keep in mind when they are evaluating vendors is what they are purchasing,” Walfish noted. “There are critical raw materials, critical components and then there are less critical things that can be purchased. And, there’s a whole bunch of different goods and services that you can purchase. I might be qualifying a new lab supplier, someone who does lab testing for me. They don’t deliver anything to me. How do I evaluate that supplier? For new suppliers, there are so many avenues I can go down.”
One of those avenues leads to an evaluation system known as the requirements-based assessment, or RBA. Used primarily to assess new vendors, this evaluation method typically is developed by a team of professionals from various disciplines within a company (quality, business and technical, for example). The team builds the RBA by identifying its company’s business needs, determining the attributes that support the business needs, establishing ways to measure those attributes, and developing questions from the measures used in planning the assessment.
Some companies supplement the RBA with an assessment that focuses on product risk. Boston Scientific Corp., for instance, uses an assessment system called vendor process certification (VPC) to evaluate the processes used by a supplier to make its product.
“After selecting a supplier using RBA, we use a method called vendor process certification in combination with typical key performance indicators such as metrics management,” explained Todd McCaslin, Boston Scientific’s global sourcing director, metals. “The VPC is a product-focused risk-based assessment technique. This is driven from the product specifications, safety considerations and indications of use. It is a method for controlling the key elements of the manufacturing process used to make the product and assure the product quality. We work hand in hand with the supplier to perform a thorough risk assessment on the entire production process, resulting in a program that will develop the necessary controls in order to ensure continued compliance to our requirements.”
Some of the more traditional methods of ensuring product quality include site visits and rating grids that act as supplier “report cards.” Site visits can provide companies with very effective insights and information about a supplier, particularly when they are conducted by well-trained, knowledgeable personnel from the evaluating organization. However, they are not always economically feasible. Small companies with numerous overseas vendors, for example, may not have the resources to visit each supplier. In those cases, companies can request information through email, though industry experts disagree on whether electronic records and desktop audits are effective vendor evaluation tools.
Questionnaires and RFIs (request for information) also have become a source of conflict within the industry. Some companies still use questionnaires to elicit basic information about potential outsourcing partners, while others have phased out the method, having recognized the disadvantages of such an information-gathering process. Questionnaires and RFIs, whether sent electronically or by regular mail, present several challenges, experts said.
First and foremost, the company sending out the questionnaire must know the kind of information it wants and how it will assess that information. Getting suppliers to complete the questionnaire in a timely manner can be challenging, and once collected, the data can be hard to access. Plus, questionnaires do not give companies an accurate assessment of a supplier because the document is often completed by one person—a quality manager or the owner—who may not be qualified to answer all questions.
“Accellent is seeing increased numbers of RFIs from our customer base,” said Jim Reed, vice president of sales, corporate customers, at Accellent Inc., a contract manufacturer based in Wilmington, Mass. “These RFIs are most often extremely detailed and require us to provide very comprehensive responses to all aspects of our operational organization. In my opinion, the completion of a supplier quality questionnaire is no longer effective. Today’s market dynamics require a more comprehensive evaluation to ensure the vendor can truly operate as an extension of the OEM’s manufacturing operations.”
For many companies, however, a questionnaire or RFI is merely a starting point in the evaluation process. St. Jude Medical sends out a survey to its potential suppliers to extract basic information about them such as ownership, quality systems, ISO certifications (if applicable) and financial performance. Once the survey is completed, the St. Paul, Minn.-based OEM starts a dialogue with the supplier to better understand the product, process or service it will provide, said Bryan Szweda, senior director of operations at St. Jude. Once a dialogue has begun, the company may conduct an on-site visit, depending on the nature and criticality of the service the supplier will provide.
One of the more traditional approaches to vendor evaluation is the rating grid. Used by companies such as ATEK Medical, a contract manufacturing firm in Grand Rapids, Mich., rating grids enable the assessing organization to objectively compare similar vendors.
But like questionnaires, rating grids have shortcomings, too: They cannot always adequately assess such hard-to-define qualitative traits as business philosophy or ethics. Companies can remedy such a shortcoming by making sure their evaluation process is quantitatively and qualitatively balanced.
“Some suppliers score really well. The show well, they look good but you just feel something isn’t there,” explained Chris Oleksy, ATEK Medical president. “I’ve been surprised by that before. It can be one of those softer qualitative things that are hard to assess. For example, how to you grade ethical behavior? You can put it on your scorecard but if you ask someone if they’re ethical they’re going to tell you yes. You may pick a supplier that fails across the board on the ethical side of things but the mathematical grid won’t pick up on that. So you always need to make sure you have a mathematical approach coupled with a qualitative approach that includes references, checking into their ethical behavior in dealing with customers, and whether they pay bills on time. Any evaluation process should have both a qualitative and quantitative approach.”
The Evolution of Evaluations
Evaluations have always played an important role in the supplier selection process. But their significance has become more imperative in recent years as the FDA has increased its scrutiny of the device approval process and the overall quality of medical products.
When the medical device industry was in its infancy, the supplier evaluation process often consisted of a site visit by a quality engineer and sourcing director. The assessment made by both individuals was usually all that was needed to make a final decision about a potential supplier, industry experts noted.
“Over the years we’ve witnessed a significant change in how current and potential customers assess our company,” Accellent’s Reed said. “Years ago it may have been a simple site visit and/or a quick quality assessment. Today’s evaluations are typically much more comprehensive and often touch all functional areas of the organization, including quality and regulatory, research and development, engineering, manufacturing, finance, IT and supply chain. Most of our larger customers also are requesting a significant amount of information regarding Accellent’s Lean initiatives to ensure we are constantly improving processes and eliminating waste.”
Much of the emphasis on the supply chain can be traced to FDA regulations and well-publicized product recalls (the Heparin, lead paint and peanut butter fiascos immediately come to mind). Such events have resulted in a more rigid regulatory environment and higher expectations (particularly among OEMs) of suppliers.
Such high expectations extend into the deepest levels of the supply chain, too. Though OEMs usually rely on their Tier 1 suppliers to evaluate and effectively manage their own vendors, it is not uncommon now for large device firms such as Boston Scientific to visit a Tier 2 or Tier 3 supplier to ensure the quality of its manufacturing capabilities.
“It’s really a criticality-based decision. If a supplier is making a critical component and they’re using someone else we would probably go out to the Tier 2 ourselves with the other supplier,” Boston Scientific’s McCaslin said. “We do, however, expect all our Tier 1 suppliers to have very good supplier controls processes. We expect them to understand how their suppliers are validating and controlling their suppliers and we expect them to have that same level of expertise in process validation as we do.”
***
Developing an effective, easy-to-deploy method of evaluating suppliers has become a crucial business competency in the medical device industry. Manufacturing firms that evaluate their supply base reap numerous benefits from the practice, including better visibility of vendor performance, reduced risk and cost, and an increased competitive advantage.
Perhaps the most important benefit, however, can be boiled down to just one word: quality.
As ATEK Medical’s Oleksy put it, “Quality is a given, it’s an absolute requirement. I don’t care if you’re making a screw or you’re making a pacemaker, quality is absolutely number one. Quality starts in every single thing everybody should be doing in this space. It is why we are all here in the first place.”
Vetting Vendors
Most medical device manufacturers understand the importance of evaluating vendors. Besides eliminating hidden costs in the product development process, evaluations can help suppliers improve their performance, which eventually benefits the organization performing the assessment.
And while the U.S. Food and Drug Administration (FDA) and the Global Harmonization Task Force have released documents that discuss the kinds of selection criteria manufacturing firms can use in their evaluations, a great deal of debate exists about which metrics are best.
“It starts with determining what is appropriate for the type of relationship,” said Joe Kelly, a director in the general manufacturing division of Tefen USA, an international consulting firm with U.S. headquarters in New York City. “For instance, in a transactional relationship, where I’m basically buying a catalogue item, do I need more than operational performance? When measuring a strategic partner, should I also consider a value add metric to capture cost improvements such as material, tooling and processes, or financial strength index, or even a behavioral index related to collaboration efforts?”
While the measurement tools will indeed vary depending on the company, product, and type of relationship that exists between the manufacturer and supplier, most vendor evaluations include the following basic metrics:
•Internal quality assurance program
•On-time delivery
•Cost-effectiveness
•Experience in the medical device industry
•Experience of the supplier’s team
•Technical capability
•Pricing
•Parts availability
•First pass acceptance rates
“What is the right metric for evaluating a vendor? There’s really not a one-size-fits-all kind of answer to that question,” said Steven Walfish, president of Statistical Outsourcing Services, a Rockville, Md.-based consulting firm that provides statistical analysis and training to FDA-regulated industries. “Companies have to determine the kinds of metrics that are best for them from a business perspective and regulatory perspective.”
—M.B.