![iso 14971]()
Editor's note: This is a guest post authored by William Storage, VP LiveSky Inc., Visiting Scholar, UC Berkeley History of Science. The article was originally published here and is being republished here with premission.
The novel alliance between security research firm MedSec and Muddy Waters LLC’s short-seller Carson Block brought medical device risk into the news again this summer. The competing needs of healthcare cost-control for an aging population, a shift toward population-level outcomes, med-tech entrepreneurialism, changing risk-reward attitudes, and aggressive product liability lawsuits demand a rational approach to medical-device risk management. Forty-six Class-3 medical device recalls have been posted this year.
Medical device design and manufacture deserves our best efforts to analyze and manage risks. ISO 14971 (including EU variants) is a detailed standard providing guidance for applying risk management to medical devices.
For several years I’ve been comparing different industries’ conceptions of risk and their approaches to risk management in my work with UC Berkeley’s Center for Science, Technology, Medicine and Society. In comparison to most sectors’ approach to risk, ISO 14971 is stellar.
My reasons for this opinion are many. To start with, its language and statement of purpose is ultra-clear. It’s free of jargon and ambiguous terms such as risk scores and risk factors – a potentially useful term that has incompatible meanings in different sectors.
Miscommunication between different but interacting domains is wasteful, and could even increase risk. Precision in language a small thing, but it sets a tone of discipline that many specs and frameworks lack. For example, the standard includes the following definitions:
- Risk– combination of the probability of occurrence of harm and the severity of that harm
- Hazard– potential source of harm
- Severity– measure of the possible consequences of a hazard
Obvious as those may seem, defining risk in terms of hazards is surprisingly uncommon; leaving severity out of its definition is far too common; and many who include it define risk as an arithmetic product of probability and severity, which often results in nonsense.
ISO 14971 calls for a risk-analysis approach that is top-down. I.e., its risk analysis emphasizes functional hazard analysis first (ISO 14971 doesn’t use the acronym “FHA”, but its discussion of hazard analysis is function-oriented).
Hazard analyses attempt to identify all significant harms or unwanted situations – often independent of any specific implementation of the function a product serves – that can arise from its use. Risk analyses based on FHA start with the hypothetical harms and work their way down through the combinations of errors and failures that can lead to that harm.
Despite similarity of the information categories between FHA and Failure Mode Effects Analysis (FMEA), their usage is (should be) profoundly different. As several authors have pointed out recently, FMEA was not invented for risk analysis, and is not up to the task. FMEAs simply cannot determine criticality of failures of any but the simplest components.
Further, FHA can reasonably accommodate harmful equipment states not resulting from failure modes, e.g. misuse, and mismatched operational phase and operating mode, and other errors. Also, FHAs force us to specify criticality of situations (harm to the device user) rather than trying to tie criticality to individual failure modes.
Again, this is sensible for complex and redundant equipment, while doing no harm for simple devices. While the standard doesn’t mention fault trees outright, it’s clear that in many cases the only rational defense of a residual risk of high severity in a complex device would be fault trees to demonstrate sufficiently low probability of hazards.
ISO 14971 also deserves praise for having an engineering perspective, rather than that of insurers or lawyers. I mean no offense to lawyers, but successful products and patient safety should not start with avoidance of failure-to-warn lawsuits, nor should it start with risk-transfer mechanisms.
The standard is pragmatic, allowing for a risk/reward calculus in which patients choose to accept some level of risk for a desired benefit. In the real world, risk-free products and activities do not exist, contrary to the creative visions of litigators. Almost everyone in healthcare agrees that risk/reward considerations make sense; but it often fails to make its way into regulations and standards.
14971 identifies a proper hierarchy of risk-control options that provide guidance from conceptual design through release of medical devices. The options closely parallel those used in design of life-critical systems in aerospace and nuclear circles:
- Inherent safety by design
- Protective measures
- Information for safety
As such, the standard effectively disallows claiming credit for warnings in device instructions as a risk-reduction measure without detailed analysis of such claims.
A very uncommon feature of risk programs is calling for regression-analysis on potential new risks introduced by control measures. Requiring such regression analysis forces hazard analysis reports to be living documents and the resulting risk evaluations to be dynamic.
A rough diagram of the risk management process of ISO 14971, based on one that appears in the standard with minor clarifications (at least for my taste) appears below.
This standard also avoids the common pitfalls and fuzzy thinking around “detection”(though some professionals seem determined to introduce it in upcoming reviews). Presumably, its authors recognized that if monitors and operating instructions call for function-checks then detection is addressed in FHAs and FMEAs, and is not some vague factor to be stirred into risk calculus (as we see in RPN usage).
What’s not to like? Minor quibbles only. Disagreements between US and EU standards bodies address some valid, often subtle points. Terminology issues such as differentiating “as low as reasonably practicable” vs “as far as possible” bring to mind the learning curve that went with the FAA AC 25.1309 amendments in commercial aviation. This haggling is a good thing; it brings clarity to the standard.
Another nit – while the standard is otherwise free of risk-neutrality logic flaws, Annex D does give an example of a “risk chart” plotting severity against probability. However, to its credit, the standard says this is for visualization and does not imply that any conclusions be drawn from the relative positions of plotted risks.
Also while severity values are quantified concretely (e.g., Significant = death, Moderate = reversible or minor injury, etc.), Annex D.3.4 needlessly uses arbitrary and qualitative probability ranges, e.g., “High” = “likely,” etc.
These are small or easy-to-fix concerns with a very comprehensive, systematic, and internally consistent standard. Its authors should be proud.
