Over the past year, our research group has been focused on the development and real-world validation of three new advanced human factors testing methodologies. The genesis for these methodologies flows from client requests that have identified specialized research needs focused on complex problems and cost reduction efforts. Below are short summaries of each problem and the new methodology that provides the related solution set. All three methodologies have been validated in large-scale client projects.
1 Applied Cognitive Task Analysis (ACTA) for Measuring User’s Emotional and Functional Response to Advanced Feature Sets for Complex New Medical Devices/Surgical Systems Utilizing Simulated User Experience Protocols.
The Problem: The development of advanced and complex new medical devices including surgical intervention robots is a costly and high-risk undertaking. The primary problem that teams face early in development is the identification of feature sets combined with an actual descriptive user interface that is reliable in terms of objectively improving clinical outcomes. Failure to define proper features/functions and optimize the balance between automation and human intervention can lead to a massive waste of development resources, an increased risk of rejection by the intended users of the system, and diminished reputation of the developer in the marketplace. Traditional market research methods have often failed to solve this complex and business-critical problem.
The Solution: By combining a validated and professional human factors research methodology known as Applied Cognitive Task Analysis with advanced feature set simulation frameworks, MUS has produced a robust and reliable means of critically assessing user response to advanced medical device/system concepts. This validated methodology employs a combination of robust psychophysical data capture methods including micro-facial expression analysis (MFEA), high-performance eye-tracking, and cognitive workload assessment along with high-fidelity animated simulations of proposed feature set combinations. The integrated data streams are used to determine with high levels of reliability whether or not new medical device feature sets and early-stage human-machine interaction design meet the needs and limitations of healthcare professionals who will purchase and utilize the final production system. The methodology probes deeply into the user’s emotional response, ratings of risk vs. reward, cognitive workload, and perceived impact on clinical outcomes. The methodology is specifically designed to produce reliable data on the impact of new system concepts on cognitive workload, risk management, training, transfer of learning, and error management. The system is also designed to identify and rate new feature set combinations as well as entirely new product innovation opportunities. The methodology is based on the recruitment of highly skilled subject matter experts such as neurosurgeons who we engage with in our specialized lab-based testing facility for extended live data capture sessions. The methodology produces an extensive roadmap for the development of systems that have both emotional resonance and human factors benefits to those who will utilize the new medical device platforms. This is a risk reduction and feature set optimization methodology.
2 Advanced Methodology for Measuring and Optimizing PFS Newtonian Injection Forces for Large Molecule Drugs
The Problem: In the drug development space, there is a massive new push toward the creation and utilization of large molecule drugs that offer unprecedented clinical advantages. However, these new liquid chemical drug configurations have surprisingly high viscosity and tend to be temperature sensitive. Such drug attributes present critical usability and human factors performance problems for those who must ultimately deliver such drugs, either to themselves as self-administering patients, or as healthcare professionals. The primary problem is a dramatic increase in the physical force required to execute a successful injection utilizing standard syringe-type devices known technically as pre-filled syringe (PFS) injection devices. So high are some injection forces that certain percentiles of the user population simply cannot undertake a successful injection without experiencing serious levels of risk and pain. For development teams who are not aware of the complexity of this problem and how it can be solved, it is possible to make assumptions and decisions about the usability of PFS devices that simply will not work in the actual marketplace. Such an oversight can delay a critical drug by months, if not years, depending on how the FDA responds to complaints from the marketplace.
The Solution: To address this vexing problem, MUS has developed and fully validated a robust force measurement platform system that captures Newtonian forces during injection sequences combined with physiological data from live user populations. In addition to force and physiological data, the system also tracks needle and device excursion during injections utilizing a specialized 3D spatial tracking system. The system has been utilized in over a dozen recent studies to determine the acceptability of various drug viscosity levels on multi-axis force distributions. The system combines advanced data capture, robust insights, and statistical analysis of anthropometric user population data to ensure that your drug can be delivered to the proper distribution of users based on population analysis and human factors best practices. The system can capture data directly from existing PFS devices, new device configurations, and/or competitive products to produce reliable engineering data and insights on exactly how drug viscosity impacts the quality of the drug delivery experience and the range of users that will be able to successfully execute large molecule drug injections.
3 AGILE Medical Device Summative Human Factors/Usability Testing Focusing on Rapid IFU Optimization and Cost Reduction.
The Problem: Medical devices undergoing final summative FDA human factors (HF) testing require professional testing expertise and careful tracking of user behavior. To provide valid and supportable data for FDA submission, the HF performance of the total product in production form including packaging, instructions for use (IFU), product labeling, and the product itself must together deliver high levels of usability and HF performance. However, it is a fact that complex medical devices do arrive at summative user testing with usability problems not resolved in prior formative studies. When this happens, it is imperative that the HF testing team and the corporation producing the medical device rapidly identify the extent and nature of such problems and take action as early as possible to halt study execution, update the necessary device systems, and, to the extent possible, address such issues before restarting the summative study sequence. Ignoring critical usability problems that surface during summative testing is both costly and dangerous since the study will produce data that shows poor HF performance. A far better approach is to constantly monitor on a daily basis HF outcomes, take immediate action to resolve critical problems, and finish the study with improved and reliable HF performance data for FDA submission.
The Solution: Having experienced these types of critical HF testing problems over the past 20 years, MUS has developed a robust and structured methodology that is focused on four critical aspects of this problem: 1) Daily tracking and detailed reporting of all primary HF performance variables for all respondents in the study in rapid succession, 2) Early and aggressive identification of error states that are consistent across primary user profiles that may constitute a usability problem trend, 3) Rapid and early identification of the root causes of such errors, and 4) Identification and schematic development of possible solution sets for resolving such critical problems. It is often the case that modifications to the IFU design, content structure, and flow will provide the necessary improvements required to meet FDA HF approval. To rapidly update the primary IFU flow, MUS has developed a formal IFU optimization methodology based on application of structured cognitive task analysis combined with rapid-response pilot testing of IFU updates BEFORE returning to the final stages of summative testing. Our methodology is specifically focused on resolution of all critical root cause of error variables through rapid redesign of the IFU systems, followed by actual pilot-session respondent testing with the optimized medical device system. Only after the medical device system refinements are validated is summative HF testing restarted and completed. This approach has saved clients hundreds of thousands of dollars in study re-do fees, not to mention reduced delays for critical FDA submissions. This methodology is a collaborative effort with the medical device development, regulatory, and product management teams.
Please feel free to contact our research team via my email below for more information.
Thank you.
Charles L. Mauro CHFP
President / Founder (1975-Present)
Cmauro@MauroUsabilityScience.Com