Setting the Benchmarking Bar
In the final installment of our two-part outsourcing best-practices series, OEMs and suppliers discuss how to continue to improve even the mostefficient and effective collaborations.
Contributing Writer
Best practices are essential for retaining OEM and supplier relationships. Just like the idea that Lean initiatives can always get leaner, best practices can always get better—but only if meaningful metrics are measured and benchmarks set. It’s critical to select metrics that give you and the client the best quantitative information regarding overall process performance, including design and engineering, quality, production, supply chain management and cost.
It’s impossible to benchmark without a starting point. The first step is to clearly define the expectations and responsibilities for both parties.
“One of the ways Ximedica does this is by using comprehensive quality plans with our clients,” said Stephen Perez, vice president of quality for the design, engineering and advanced manufacturing services provider in Providence, R.I. “These documents go well beyond the standard contract language of terms and conditions to define more tactical-level items. Some of the more significant discussion points usually center on whose design control process will be followed and where the design history file documentation will be held. We have devised our systems and processes to be flexible to accommodate product development programs utilizing a client’s design control system, our own or a hybrid of each.”
Keith Checca, director of business development for CIRTEC Medical Systems, a contract design, development and manufacturing firm in East Longmeadow, Mass., agreed, noting that the best relationships result when the work activities and the quality/system responsibility are defined clearly up front through a detailed quality plan. CIRTEC has developed a number of template documents for different types of devices, Checca said.
A face-to-face meeting, including both the engineering and the quality teams, is essential at this stage because “we commonly see issues arise when the QA (quality assurance) team comes in late in the development,” he said. “This helps identify the mind set of our clients and their concern areas early in the process. It also allows us to better deal with the unknown and any changes that might occur. Integrated manufacturing can be very challenging, so we like to drive a clear requirement starting point.”
Selecting Metrics and Benchmarks
Once the starting point (quality plan) has been defined, the next step is deciding what metrics are required and how to best go about benchmarking them.
For many companies, risk is a major driver that requirescareful tracking.
“Because risk tolerance varies widely from company to company, and from product to product, one of the important lessons of benchmarking risk is to assure you can always relate data back to business risk,” added Perez. “In my experience, organizations are very good at collecting and analyzing risk data related to products, patients, users, etc. However, few companies do a good job of translating that data into things like customer satisfaction or repeat/new business.”
Minneapolis, Minn.-headquartered Medtronic Inc. has created its own internal supplier-selection tool that is based on a detailed risk analysis.
“We consider this process of selecting suppliers for a new project to be a best practice,” commented Saleem Ganiy, a principal product engineer for supplier quality based at the company’s cardiovascular facility in Santa Rosa, Calif. “In this process we identify potential suppliers by major commodities and benchmark them against criteria that we consider to represent business and performance risks.”
These criteria were developed after consulting with several Medtronic divisions. Three key criteria that define the “foundation” of a supplier’s business are financial stability, customer diversification and disaster recovery plans. Moving on to the “structural” risks a supplier may bring to a project, Medtronic identified the following supplier characteristics as critical for controlling risk:
• Production and service capabilities (abilities to take the project from concept to full volume, considering facilities, equipment and people skills);
• Level of understanding of the medical device business (how vested they are in this industry);
• Technological competence level (leader, laggard or a follower); and
• Cost competitiveness (cost leader, competitive or low-cost producer).
“All suppliers must be vetted and accepted through this process to be considered as final candidates,” said Ganiy. “The process maps the ratings, which are crisply defined, on a model that was created and calibrated against well-known suppliers. We have been using this tool for a few years and it has been very useful in making rational decisions and avoiding the pitfalls of engaging with the wrong suppliers. Now we debate the merits of the suppliers we engage with at an objective level and at the earliest stages of a project, rather than come back years later to conduct lessons learned.”
Real-time metrics are essential for the best process control.
“Whether sharing nonconformance information or, just as importantly, complaint handling and reliability data, real-time metrics provide critical insight into how the product holds up to the demands of the field,” said Tim Mohn, an industry principal with Sparta Systems, a quality management software firm in Holmdel, N.J. “With OEMs, the key is visibility into your supply chain and product quality. Often, a company’s supplier agreement only requires a subset of nonconformance information found by the OEM to be raised to the company. This is okay in theory, but what if the non-conformances really weren’t low risk? It’s best for organizations to be able to track and trend through all issues on a single lot, and across lots, as part of the complaint investigation process.”
Mohn suggests the following steps for identifying bestmeasurements:
• Building a cross-functional team to define what should be measured;
• Determining whether it truly is feasible to measure those items—all too often, the proposed measures aren’t available and compromises must be made; and
• Developing the measurement system with clear definitions that also are readily available.
“Companies frequently do the first two steps well, but then fail to publicize the metric definitions that drive the scorecard, causing employees make the wrong decisions/assumptions based on the data,” Mohn said. “One example I see frequently is when businesses develop a measuring system that results in an ultimate grade, either numerical or alphabetical. When employees are asked how it’s measured, they can talk through the high-level concepts but fail to understand how it’s actually measured. This may sound minor, but given device companies’ increased dependence on OEMs, it’s critical that your oversight mechanisms are completely understood.”
If you aren’t totally confident with your own internal abilities to set metrics, or are interested in a broader view of how other companies in the industry benchmark similar processes, hire a consultant to survey other firms.
“We discuss the issues they are having trouble with and then create a survey that allows them to see how other companies might approach the same issues,” said Cameron Tew, executive director of research and advisory services for Best Practices LLC, a research, consulting and advisory firm based in Chapel Hill, N.C., that specializes in best-practice benchmarking for the pharmaceutical and medical device industries. “We aggregate the responses we receive and utilize that data to establish the benchmarks. From meetings and discussions with clients and the research participants we then assess what is deemed normal operating parameters and can share with our clients what benchmarks can be seen as excelling best practices.”
Flexibility is the Name of the Game
Usually, most companies’ quality and manufacturing groups set the traditional metrics for any process. Because each client, however, has a different set of concerns, it’s best to stay flexible.
“We try and keep the tracking isolated to a few critical metrics,” said Checca. “We commonly look at process times/cost and a lot on yield at various processes. Statistical process control is usually not reviewed, unless there are specific reject areas that all parties are worried about. A lot of time is also spent on the general number of nonconformances and corrective action requests, with an aim to reduce those since they are a target during audits.”
Joe Horvath, vice president and general manager for Sanbor Medical, a medical device contract manufacturer in Allentown, Pa., said his company uses a limited set of metrics that is easy to monitor and maintain.
“For suppliers, we utilize a limited set of metrics that is easy to monitor and maintain,” Horvath said. “We measure timeliness of deliveries, percentage of lot acceptance, robustness of their quality system and competitiveness of pricing. We do this twice per year, as we find this period allows for a realistic top-level view of performance within a reasonable timescale.”
One of the biggest challenges for many suppliers and contract manufacturers is the variety of their customers and products; no two are alike or look at the same data in the same way. Furthermore, no two products run exactly alike either, which means processes often need to be adjusted or customized—even more reason to stay flexible.
“For the most part,” added Horvath, “we don’t have formal best practices set up with either our suppliers or customers, aside from the performance metrics we’ve established to monitor our supplier base. Being a custom manufacturer, we work daily with thousands of individual components and raw materials. These are sourced from around the world and come from a very diverse base of suppliers. We’re in the same position with each customer, since every product we make is virtually unique. With this broad range of contact, we need to deal with an extensive set of interactive expectations. This makes it impossible to standardize our supplier and customer interactions, and makes broad-based benchmarking very difficult. We compensate by having flexible procedures that govern our external working relationships.”
For Checca, his project managers are instrumental in maintaining flexibility with clients and getting feedback to the project teams. What are the key traits for an effective project manager?
“Our PMs are a combination leader, quality auditor and customer service rep,” he said. “They have a ‘let’s find a way to all win’ attitude. They can’t be overly biased toward the contractor. They also can’t cave to the customer needs or will lose internal respect. Add in a ‘I did what I said I would do’ requirement and you have an excellent project manager.”
Celestica, a global company with headquarters in Toronto, Canada, provides supply chain and technology solutions for the medical device market. Recently, one of its customers needed to respond more rapidly and be more flexible to its ever-changing customer demands.
“We collaborated with them to improve the flexibility of their supply chain,” said Sandra Ketchen, vice president and general manager for healthcare. “A big part of the work was designing metrics to specifically measure component lead-time improvements. By designing metrics for improved supply chain flexibility the customer reduced material costs, increased spend leverage, reduced risk in the supply chain and mitigated excess and obsolete materials. Their supply chain ultimately became far less vulnerable to possible lead-time extensions or supply disruptions that could delay time to market.”
Adjust as Needed
Metrics only are useful if they are periodically reviewed. They should be part of the formal management review process, where the executive level of the organization determines whether the corporate goals and objectives are being met. As business conditions, objectives and program requirements change, metrics sets need to be revisited and revised as necessary.
Checca said his company conducts a monthly review ofmetrics across all clients, which typically leads to a “most useful” and well-received list.“We then spread that back to other clients and team members and improve the reviews across the board,”he said.
Horvath believes metrics should always be subject to review.“We first started this type of analysis more than 10 years ago and have been using our current set for three or four years now,” he said. “As business evolves, some metrics will lose their significance. Others will arise that weren’t considered in prior periods, due to new technologies or general changes to the industry. It’s always important to do routine assessment of any set of metrics you’re using. Be sure you are getting the information you want from each data point and that the individual metric is relevant to your overall analysis.”
OEM Audits
Regulatory agencies across the globe are placing increased scrutiny on the use of outsourced services. In response, OEMs are making sure they clearly understand what systems and processes their suppliers and contract manufacturers have in place to meet and exceed their internal requirements. One of the best ways for an OEM to assess these systems is through an on-site audit.
“Audits are a key component to any company selling products or services to an OEM,” said Horvath. “The OEM needs to make sure the contract manufacturer’s quality system is robust and can support the OEM as they market their product. An audit will test the supplier’s quality system and highlight key areas for improvement. This is a valuable tool both for the OEM, to gain assurance of the supplier’s reliability, and for the supplier. Being subjected to a steady schedule of audits from a number of different OEMs puts suppliers in a frequent position of having their quality system challenged. This helps to maintain rigorous adherence to the practices prescribed by the quality system and leads to regular improvement whenever a deficiency is pointed out.”
From an ongoing business perspective, audits are critical to ensuring that a partner/supplier is meeting all regulatory and compliance requirements. “However,” said Ketchen, “if you have established mutual goals at the outset of your relationship and structured metrics to manage toward specific outcomes, an audit may not be needed to assess performance. This can be achieved through the quarterly and annual review process. If you are proactively managing the relationship well, there should be no surprises that an audit could uncover.”
Staying in Touch
As important as metrics are, sometimes the easiest way to find out what needs improving is to ask the customer.
“Since our mission is to provide exceptional customer service to our OEM medical device customers, we conduct lots of surveys at the end of each year or after each customer visit,” said Steve Wychulis, director of corporate development at CEA Technologies, a contract manufacturer of medical devices in Colorado Springs, Colo. “Did we meet their needs? Was delivery to their satisfaction? Would they recommend us to their peers? We also have a section for written comments. Our management team reviews this information and makes strategic decisions about any improvements that may need to be made. Any data that is collected, through metrics or surveys, is discussed by top management and shared throughout the company.”
CIRTEC Medical Systems conducts quarterly to annual reviews with its clients to discuss the business and how they can mutually improve. “So rather than set the metrics and agenda,” said Checca, “we simply set the meeting and allow the approach and discussion to vary over time.”
During one of these meetings, Checca recalls getting challenged by a customer regarding scheduling issues. “We were constantly pushing for more time, and the customer was pushing to speed us up,” said Checca. “We provided a breakout of the lead times by process and were able to uncover that the real issue was a long dwell process right up front, combined with the client not giving us consistent pre-warning of the parts arriving. Now we get a seven-day advance ship notice and can schedule that dwell process ahead of time.”
Celestica’s healthcare division also implements an ongoing review process that typically gets refreshed annually with customers as they go through their planning process and reset their objectives.
“Recognizing that each customer’s needs are constantly evolving, we conduct quarterly customer interlocks that involve a formal progress review that’s designed to obtain feedback and understand changes in their priorities,” said Ketchen. “We also meet to reset objectives on an annual basis and update metrics on our customer scorecards accordingly.”
When it comes to managing customer relationships, one size definitely does not fit all. Every OEM faces unique challenges, which makes close collaboration with every supply chain partner critical to success. Successful partnerships are jointly defined at the outset of the relationship and it is crucial that both companies, at all levels of each organization, are aligned on specific partnership objectives.
“Objectives must be realistic and not overwhelming,” said Checca. “The two companies should agree on several key objectives that are deemed most important, then manage performance through benchmarking metrics toward achieving specific goals, whether it iscost reduction, optimizing the supply chain or improving time-to-market.”
Ketchen agreed. “Don't ‘over-metric’ but pick the critical few,” she said. “Keep them simple. Ask what is most critical to the business and re-evaluate these metrics on a regular basis. The more effective metrics tend to be leading indicators—for example, the component lead time will ultimately have a positive effect on the flexibility and delivery metrics.”
Setting metrics and benchmarking results aren’t only useful for real time, short term, or annual objectives; they also can be applied to longer-term visions. “Our research findings typically depict a three-to-five year future horizon,” said Tew. “Large corporations require such cycle times to implement complex programs with extended learning curves and need quantitative data to keep them on course.”
Benchmarking best practices is a continuous process in forward-thinking companies because market dynamics, regulations, personnel and corporate thinking is always changing.
“We have customers with which we have been working together for eight or more years on complex systems manufacturing, and we are continually refining best practice,” said Checca.
Of course, as best practices continue to improve, it gets harder to improve because there are fewer places for waste and error to hide. However, “every incremental improvement that a company ekes out or makes in its processes and efficiencies can save time and money and ultimately lead to better revenues,” said Tew.
Finally, don’t forget to acknowledge and reward these gains when they happen. “Celebrate successes when goals are achieved,” said Ketchen. “This helps motivate the team.”
Mark Crawford is a full-time freelance business writer based in Madison, Wis. He can be reached at mark.crawford@charter.net.