• Login
    • Join
  • FOLLOW:
  • Subscribe Free
    • Magazine
    • eNewsletter
    Checkout
    • Magazine
    • News
    • Opinions
    • Top 30
    • Research
    • Supply Chain
    • Device Sectors
    • Directory
    • Events
    • Resources
    • Microsites
    • More
  • Magazine
  • News
  • Opinions
  • Top 30
  • Research
  • Supply Chain
  • Device Sectors
  • Directory
  • Events
  • Resources
  • Microsites
  • Current / Back Issues
    Features
    Editorial
    Digital Edition
    eNewsletter Archive
    Our Team
    Editorial Guidelines
    Reprints
    Subscribe Now
    Advertise Now
    Top Features
    Seeking a Revolution in Clinical Care Through AI

    Diversifying Supply Chains: New Opportunities Post Pandemic and After Brexit

    Addition by Subtraction: The Latest of Machining for Medtech

    Innovation Incubators: An R&D Roundtable

    Virtual Allies: Electronics Manufacturing Services Providers Offer Real Solutions
    OEM News
    Supplier News
    Service / Press Releases
    Online Exclusives
    Press Releases
    People in the News
    Product & Service Releases
    Supplier News
    Medtech Makers
    Technical Features
    International News
    Videos
    Product & Service Releases
    Live From Shows
    Top News
    January AI Raises $8.8 Million in New Funding

    FDA Clears Ava Fertility Tracking Wearable Device

    First Patient in Europe Treated With Diamondback 360 Coronary Orbital Atherectomy System

    Artificial Intelligence, Machine Learning Adoption in Healthcare to Grow Considerably

    Japan to Lead APAC Assisted Reproductive Technology Devices Market
    From the Editor
    Blogs
    Guest Opinions
    Top Opinions
    Seeking a Revolution in Clinical Care Through AI

    Diversifying Supply Chains: New Opportunities Post Pandemic and After Brexit

    Addition by Subtraction: The Latest of Machining for Medtech

    Innovation Incubators: An R&D Roundtable

    Virtual Allies: Electronics Manufacturing Services Providers Offer Real Solutions
    Top 30 Medical Device Companies
    Market Data
    White Papers
    Top Research
    Rethink Your Medtech ASC Strategy, Part 1

    Pandemic-Level Playing Field: How COVID-19 Made Space for Progress

    Tips for Reigniting Your MDR/IVDR Preparation

    Three Major Innovation Opportunities for Biomedical Textiles

    Use Vertical Integration to Increase Productivity, Drive Innovation
    3D/Additive Manufacturing
    Contract Manufacturing
    Electronics
    Machining & Laser Processing
    Materials
    Molding
    Packaging & Sterilization
    R&D & Design
    Software & IT
    Testing
    Tubing & Extrusion
    Cardiovascular
    Diagnostics
    Digital Health
    Neurological
    Patient Monitoring
    Surgical
    Orthopedics
    All Companies
    Categories
    Company Capabilities
    Add New Company
    Outsourcing Directory
    PTI Engineered Plastics Inc.

    Unicep

    FUTEK Advanced Sensor Technology Inc.

    maxon

    MW Life Sciences
    MPO Summit
    Industry Events
    Webinars
    Live From Show Event
    Industry Associations
    Videos
    Career Central
    eBook
    Slideshows
    Top Resources
    Refurbished Medical Equipment Testing and Certification Under the NEC

    Taking the First Step with Metal Components

    4 Trends Driving Medical Manufacturing in 2021

    Pioneering AI Solutions to Reinforce Healthcare Services

    The 8 Major Types of PPE Supply Chain Fraud
    Companies
    News Releases
    Product Releases
    Press Releases
    Product Spec Sheets
    Service Releases
    Case Studies
    White Papers
    Brochures
    Videos
    Outsourcing Directory
    MW Life Sciences

    Medbio LLC

    Forefront Medical Technology

    Spectrum Plastics Group

    Concise Engineering
    • Magazine
      • Current/Back Issues
      • Features
      • Editorial
      • Columns
      • Digital Editions
      • Subscribe Now
      • Advertise Now
    • News
    • Directory
      • All Companies
      • ALL CATEGORIES
      • Industry Associations
      • Company Capabilities
      • Add Your Company
    • Supply Chain
      • 3D/Additive Manufacturing
      • Contract Manufacturing
      • Electronics
      • Machining & Laser Processing
      • Materials
      • Molding
      • Packaging & Sterilization
      • R&D & Design
      • Software & IT
      • Testing
      • Tubing & Extrusion
    • Device Sectors
      • Cardiovascular
      • Diagnostics
      • Digital Health
      • Neurological
      • Patient Monitoring
      • Surgical
      • Orthopedics
    • Top 30 Company Report
    • Expert Insights
    • Slideshows
    • Videos
    • Podcasts
    • Resources
    • eBook
    • Infographics
    • Whitepapers
    • Research
      • White Papers
      • Case Studies
      • Product Spec Sheets
      • Market Data
    • MPO Summit
    • Events
      • Industry Events
      • Live From Show Events
      • Webinars
    • Microsite
      • Companies
      • Product Releases
      • Product Spec Sheets
      • Services
      • White Papers / Tech Papers
      • Press Releases
      • Videos
      • Literature / Brochures
      • Case Studies
    • About Us
      • About Us
      • Contact Us
      • Advertise with Us
      • eNewsletter Archive
      • Privacy Policy
      • Terms of Use
    Columns

    The Importance of Usability/Human Factors Testing

    When is the work of human factors over, and how does your team know they are prepared for summative testing?

    The Importance of Usability/Human Factors Testing
    Maria Shepherd, President and Founder, Medi-Vantage03.04.20
    When is the work of human factors over, and how does your team know they are prepared for summative testing? Fact is, human factors is never complete, it is only part of the iterative cycle FDA wants us to use for new product development of medical devices. There is always so much more to learn about your product. Even if the team has completed their analyses on the end-users who will be using the device, assessed the environment(s) where the medical device will be used, and characterized key device user interfaces, your work is still not finished. Is your device ready? Have your design controls prepared your device for summative usability testing?

    Why This Is Important
    A recent study from Johns Hopkins estimates greater than 250,000 Americans die each year due to medical error, making it the third leading cause of death after heart disease and cancer.1 Other studies state this estimate should be much higher, asserting the number of medical-error-related deaths is as high as 440,000 (Table 1).2 The authors state the discrepancy exists because medical examiners, coroners, physicians, and funeral directors seldom note human errors and system failures on death certificates. Death certificates are the source of information for the CDC when estimating statistics for deaths nationwide. It’s not easy to get this information onto a death certificate.


    Table 1: Leading causes of deaths in the U.S. (2017)1-3

    Test Early, Test Often
    A step many medical device companies skip is formative testing. This is a mistake made for cost control reasons and can have serious implications when it comes time to perform the summative study. Risk errors that may not have been anticipated can and will emerge in the summative (Table 2). Formative usability testing serves to validate the chosen user interface design controls are sufficient to eliminate as many risks to use safety and effectiveness as possible, bringing device performance to an acceptable level. Usability testing enables a medical device design team to improve the usability of their device to meet acceptable standards of risk.


    Table 2: Top medical device causes for recalls (Q4, 2018)4

    Successful formative testing typically requires five to eight end-users, making the number of iterations flexible for the product development team. Use-related errors can be seen early in the formative usability process. In the early stages of product development, many use-errors can be identified utilizing a methodology that recruits end-users with no prior experience with the device to test how intuitive the device interface is. In the later stages of product development, it is recommended to perform all usability testing with experienced end-users to ensure the product performs well against competitive devices on the market. To control costs, many medtech companies perform early formative testing in house.

    A series of formative tests can ensure the end-user will be able to manage use of the device successfully for its intended purpose. When designing formative studies, try to expose as many risks as possible on the road to a successful summative usability test.

    What to Avoid
    One of the biggest mistakes made in formative usability testing happens when the development team over-trains participants before testing, reducing the potential for use-errors to occur. Also, be sure to capture what end-users find easy and intuitive in order to note design features that do not need to be changed.

    The Magic Number
    Use-errors are a type of systematic error, so it is difficult to calculate their frequency of occurrence. Table 3 (i.e., K.1 of ISO 62366-2) illustrates the probability of observing at least one instance of a use-error as a function of sample size and underlying population use-error rates.5 It is important to note the underlying population use-error rate can never be known and must be estimated.


    Table 3 (i.e., K.1 of ISO 62366-2): Number of test participants needed in a usability test5

    ISO 62366-2 states many usability problems can be uncovered with sample sizes from five to eight end-users, driving the industry standard in medical device formative studies.

    For summative testing, the ISO standards state 15 end-users per distinct user group (the minimum number recommended by the FDA) is likely to provide the data needed to measure risk. This number is based upon a study conducted in 2003 by Faulkner6 where empirical data was collected from a sample of 60 individuals. The study results indicated a sample of 15 people was enough to find a minimum of 90 percent (average 97 percent) of all problems with the product being tested.

    IRB or Not?
    When should you perform usability testing under institutional review board (IRB) oversight? An IRB is an organizational body with the goal to protect the rights and welfare of human research subjects participating in research studies. The IRB will want to see the following in your submission:
    • The usability testing protocol
    • Informed consent and assent form
    • Recruiting screeners with inclusion and exclusion criteria
    • The CV of your designated principal investigator
    • Research locations
    • Details of your device
    The IRB will evaluate the ethicality of your research based on the information you provide to ensure the methodology minimizes end-user risk and all recruited end-users are fully informed of the full scope of the research to be conducted before the end-user decides to participate. Of equal importance is the principal investigator chosen is qualified to manage the usability study.

    Purpose of Submitting to IRB
    Pre-approval by the IRB is important when potential risks are introduced by the usability study. An obvious example is usability testing of an injection device. This introduces the risk of a needlestick, and an IRB should be utilized to ensure the risk of a needlestick is minimized. Another example is a transdermal patch, where the end-user could have an allergic reaction to patch material. The IRB will ensure the protocol implements protection measures for the end-user participants.7

    The Medi-Vantage Perspective
    We are frequently asked if there are usability tests that do not require IRB submission. This a decision made on a case-by-case basis and there are reasons to waive an IRB submission, such as if the device is very low risk and the organization managing the usability test has taken all applicable protection measures. 

    References
    1. bit.ly/mpo200301
    2. bit.ly/mpo200302
    3. bit.ly/mpo200303
    4. bit.ly/mpo200304
    5. bit.ly/mpo200305
    6. Faulkner, L. (2003). Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, and Computers, 35(3), 379-383.
    7. bit.ly/mpo200307


    Maria Shepherd has more than 20 years of experience in medical device marketing in small startups and top-tier companies. After her industry career, including her role as VP of marketing for Oridion Medical, where she boosted the company valuation prior to its acquisition, director of marketing for Philips Medical, and senior management roles at Boston Scientific Corp., she founded Medi-Vantage. Medi-Vantage provides marketing, business strategy, and innovation research for the medical device, diagnostic, and digital health industries. The firm quantitatively and qualitatively sizes and segments opportunities, evaluates new technologies, provides marketing services, and assesses prospective acquisitions. Shepherd has taught marketing and product development courses and is a member of the Aligo Medtech Investment Committee (www.aligo.com). She can be reached at 855-343-3100. Visit her website at www.medi-vantage.com.
    Related Searches
    • technologies
    • data
    • diagnostic
    • boston scientific
    Related Knowledge Center
    • Testing

    Related Columns

    • Testing
      Going Beyond the Regulations to Evaluate Relevant Medical Device Risk

      Going Beyond the Regulations to Evaluate Relevant Medical Device Risk

      Companies need to review their testing plan so it supports a successful submission, saves time, and ensures the long-term success of their product.
      Sherry Parker, Ph.D., Senior Director of Regulatory Toxicology, WuXi AppTec Medical Device Testing 06.04.20

    • Materials | Testing
      Particle Particulars: Why this Analysis Is Foundational for Successful Devices

      Particle Particulars: Why this Analysis Is Foundational for Successful Devices

      In the mid-2000s, medical device experts were hopeful they’d found one solution to longer-lasting implants.
      Dawn A. Lissy, President & Founder, Empirical 03.04.20

    • R&D & Design | Testing
      FDA Guidelines for Employing X-ray Inspection in Medical Device Development

      FDA Guidelines for Employing X-ray Inspection in Medical Device Development

      Different materials exhibit different radiopacity as a function of material composition (i.e. atomic weight) and thickness.
      Gil Zweig, President, Glenbrook Technologies 01.29.20


    • Testing
      Managing Risks for Medical Devices:  A Guide to Understanding ISO 14971

      Managing Risks for Medical Devices: A Guide to Understanding ISO 14971

      The intent of ISO 14971 is to provide a general “bird’s-eye view” approach to designing and implementing a risk management plan for medical devices.
      Helin Räägel, Ph.D., Biocompatibility Expert, Nelson Labs 10.08.19

    • Testing
      Preparing for MDR: Test In-House or Outsource?

      Preparing for MDR: Test In-House or Outsource?

      As the EU MDR deadline draws near, manufacturers are under pressure to bring products into compliance and get submissions in front of regulators ASAP.
      Sandi Schaible, Senior Director, Analytical Chemistry and Regulatory Toxicology, WuXi Medical Device Testing 10.08.19

    • Testing
      Why Big Companies With Internal Labs Should Outsource Testing

      Why Big Companies With Internal Labs Should Outsource Testing

      ...
      Dawn A. Lissy, President & Founder, Empirical 03.07.19


    • Packaging & Sterilization | Testing
      Coming Clean: Developing Standards for Cleaning Reusable Devices and Instruments

      Coming Clean: Developing Standards for Cleaning Reusable Devices and Instruments

      ...
      Dawn A. Lissy, President & Founder, Empirical 07.26.18

    • 3D/Additive Manufacturing | Testing
      The Final Four: My Top Headlines for 2017

      The Final Four: My Top Headlines for 2017

      ...
      Dawn A. Lissy, President & Founder, Empirical 11.07.17

    • Packaging & Sterilization | Testing
      Packaging Validations—Looking Beyond ISO 11607

      Packaging Validations—Looking Beyond ISO 11607

      ...
      Christopher Scott, Vice President, Eurofins Medical Device Testing 10.05.17


    • Testing
      So We’re All on the Same Page: You Need a Test Plan

      So We’re All on the Same Page: You Need a Test Plan

      ...
      Dawn A. Lissy, Empirical 09.01.17

    • 3D/Additive Manufacturing | R&D & Design | Testing
      Pro Tips for Prototypes: The Smart Way to Manage Device Iterations

      Pro Tips for Prototypes: The Smart Way to Manage Device Iterations

      ...
      Dawn A. Lissy, Empirical 07.26.17

    • Digital Health | Software & IT | Testing
      Recent Cyber Attack Enough to Make Medtech WannaCry

      Recent Cyber Attack Enough to Make Medtech WannaCry

      ...
      Michael Barbella, Managing Editor 06.06.17


    • R&D & Design | Testing
      The Human Factor

      The Human Factor

      ...
      Steve Maylish and Dave Hines, Fusion Biotec 05.01.17

    • Testing
      Peruvian Prospects:  The Medical Device Industry of a Country Founded on Precious Metals

      Peruvian Prospects: The Medical Device Industry of a Country Founded on Precious Metals

      ...
      Dawn A. Lissy, President & Founder, Empirical 04.03.17

    • Testing
      Extractable/Leachable Chemistry Testing Strategies for Medical Devices

      Extractable/Leachable Chemistry Testing Strategies for Medical Devices

      ...
      Matthew R Jorgensen, Ph.D., and Thor Rollins, BS, Nelson Laboratories; Allison Komiyama, Ph.D., RAC, AcKnowledge RS 04.03.17


    Trending
    • Hardy Diagnostics Releases Rapid Test For COVID-19
    • Portable, Home-Use Device Quickly Measures Inflammation Levels
    • 5 Ways Plastics Revolutionized The Healthcare Industry
    • Common Issues And Their Solutions In Health Application Development
    • Top 10 Trends In The Medical Device And Equipment Industry
    Breaking News
    • January AI Raises $8.8 Million in New Funding
    • FDA Clears Ava Fertility Tracking Wearable Device
    • First Patient in Europe Treated With Diamondback 360 Coronary Orbital Atherectomy System
    • Artificial Intelligence, Machine Learning Adoption in Healthcare to Grow Considerably
    • Japan to Lead APAC Assisted Reproductive Technology Devices Market
    View Breaking News >
    CURRENT ISSUE

    January/February 2021

    • Seeking a Revolution in Clinical Care Through AI
    • Diversifying Supply Chains: New Opportunities Post Pandemic and After Brexit
    • Addition by Subtraction: The Latest of Machining for Medtech
    • Innovation Incubators: An R&D Roundtable
    • Virtual Allies: Electronics Manufacturing Services Providers Offer Real Solutions
    • View More >

    Cookies help us to provide you with an excellent service. By using our website, you declare yourself in agreement with our use of cookies.
    You can obtain detailed information about the use of cookies on our website by clicking on "More information”.

    • About Us
    • Privacy Policy
    • Terms And Conditions
    • Contact Us

    follow us

    Subscribe
    Nutraceuticals World

    Latest Breaking News From Nutraceuticals World

    Preliminary Study Links Markers of Impaired Bone Health to Vegan Diet
    Refined Grains Linked to Heart Attack Risk, Early Death
    Kappa Bioscience Extends Research Partnership For COVID-19 Research
    Coatings World

    Latest Breaking News From Coatings World

    Nippon Paint Marine Wins Korea Export Award
    Universal Display Corporation, PPG Expand Global Production of UniversalPHOLED Materials
    PPG Introduces Premium Iso-free Primer Surfacer, Catalyst
    Medical Product Outsourcing

    Latest Breaking News From Medical Product Outsourcing

    January AI Raises $8.8 Million in New Funding
    FDA Clears Ava Fertility Tracking Wearable Device
    First Patient in Europe Treated With Diamondback 360 Coronary Orbital Atherectomy System
    Contract Pharma

    Latest Breaking News From Contract Pharma

    Spark Therapeutics Appoints CTO
    Cryoport Launches ESG Program
    U.S. Govt. Purchases 100k Doses of Lilly's COVID-19 Antibody Cocktail
    Beauty Packaging

    Latest Breaking News From Beauty Packaging

    The Top Beauty Products of 2021, According to HelloGiggles
    Estée Lauder Taps Ana de Armas as Global Brand Ambassador
    Sephora Announces Significant Expansion
    Happi

    Latest Breaking News From Happi

    What You're Reading on Happi.com
    Cosmetic Chemical Sales To Post CAGR 5.8% To 2025
    ACI Urges Senate to Approve Regan as EPA Administrator
    Ink World

    Latest Breaking News From Ink World

    Etiflex Enters New Markets with Nilpeter FA-22 Installation
    Hydrocarbon Solvents Market to Surpass $8.1 Billion by 2030
    Marshall & Bruce Adds Koenig & Bauer Rapida 106 41-Inch Seven-Color Press
    Label & Narrow Web

    Latest Breaking News From Label & Narrow Web

    FTA converts Forum & INFOFLEX to virtual events
    Xeikon launches two new digital label printing presses
    UFlex adds capacity in packaging films
    Nonwovens Industry

    Latest Breaking News From Nonwovens Industry

    Texas Medical Technology Partners with My Protect Kit
    Mexico City Single-Use Plastics Ban Leads to Tampon Shortage
    Believe Diapers Launch in U.S.
    Orthopedic Design & Technology

    Latest Breaking News From Orthopedic Design & Technology

    Fusion Robotics Receives FDA Clearance for Spinal Navigation and Robotics System
    Kaia Health Unveils Next-Gen Complete MSK Care Solutions
    First Implantation of CTL Amedica's Minimally Invasive Flex Tower
    Printed Electronics Now

    Latest Breaking News From Printed Electronics Now

    Comercial Kywi Improves Customer Service, Front-Store Operations with Zebra Mobile Solution
    Global Printed Circuit Board Market Projected to Reach $69.32 Billion by 2027
    Global Smart Glass Market to Register 6.8% CAGR Between 2021-28: Grand View Research

    Copyright © 2021 Rodman Media. All rights reserved. Use of this constitutes acceptance of our privacy policy The material on this site may not be reproduced, distributed, transmitted, or otherwise used, except with the prior written permission of Rodman Media.

    AD BLOCKER DETECTED

    Our website is made possible by displaying online advertisements to our visitors.
    Please consider supporting us by disabling your ad blocker.


    FREE SUBSCRIPTION Already a subscriber? Login