The Intersection of Medical Device Usability and Risk Management

Today we’re going to be talking to our good friend Mike Drues from Vascular Sciences about the intersection of usability and risk management. 

There are so many tips and great pointers that you will not want to miss this show! Have a pen and paper or your note-taking device available, because this episode is chock-full of information that you won’t want to forget.

Listen Now:

Like this episode? Subscribe today on iTunes or Spotify.

Some highlights of this episode include:

• What usability means and how it applies to the medical device production process.
• Where the FDA stands on the topic of usability and what types of guidance documents are available.
• How the topic is handled by international and world agencies in addition to the FDA, as well as how cultural differences can impact usability studies.
• The difference between a usability study and a clinical trial. A usability study is focused on the user and a clinical trial is focused on the patient, but today, the two are often one and the same.
• Why usability needs to be more “real-world.” Many times, what the engineer intends is not how the doctor or surgeon will actually use the device.
• The importance of instructions for use and labeling, and how that relates to risk management.
• The relationship between usability and product liability.

Links:

Mike Drues on LinkedIn

Vascular Sciences

IEC 62366

ISO 14971

MD&M East (Jon & Mike will both be speaking)

Should design controls have prevented the recent reprocessing tragedy: Do we need new regulation?

Greenlight Guru 

Quotes by Mike:

“Usability and usability testing is even more important if the device is going to be used by the general public as opposed to a trained medical professional.”

“People in one part of the world might use a device one way, while in another part of the world, it’s used another way.”

“Usability needs to be thought of in a real-world sense.”

Transcription:

Announcer: Welcome to the Global Medical Device podcast where today's brightest minds in the medical device industry go to get their most useful and actionable insider knowledge, direct from some of the world's leading medical device experts and companies.

Jon Speer: Hello and welcome to the Global Medical Device podcast. Super excited about this episode, we just finished recording it. Mike Drues from Vascular Sciences, and I, we talked about the intersection of usability and risk management. It was a wonderful conversation. Lots of good tidbits and pointers and tips laced throughout this podcast. So please sit back and relax, and grab a pen and paper or whatever device that it is that you use to take notes, because there are quite a few items here that you're going to want to make note of.

Jon Speer: Hello, and welcome to the Global Medical Device podcast. This is your host, the founder and VP of Quality and Regulatory at greenlight.guru, Jon Speer. And I am super excited this morning, I've got my good friend, Mike Drues from Vascular Sciences, to join me once again on the podcast. Good morning, Mike.

Mike Drues: Good morning, Jon. Thanks for the opportunity to be with you and your audience today as always.

Jon Speer: Absolutely, and folks, for those of you who are listening, Mike and I, we always try to find topics that, well, will be interesting for a discussion, but also timely and relevant. And I'm going to have to say, I think we found a good one today, Mike. And that topic is, we're going to talk about the intersection of usability and risk management. And let's just dive into it, because I think these topics, usability, risk management, there's... Well, risk management, I've obviously talked a great deal about in the past, we've got a lot of content. Mike's done webinars, has podcasts, other guides and content on this topic as well, but usability is not something we've really delved into too deeply. So let's just talk about usability first, what does that mean? When you hear that term, what does it mean, and how do I do it, and how do I apply it, and so on. So just, I guess, give a 30,00 foot view of usability, Mike.

Mike Drues: Well, that's a great place to start, Jon. So just going to Google and typing in, "What is usability?", here's what we find, "The degree to which something is able or fit to be used." The degree to which something is able or fit to be used. I'm not sure how useful that definition of usability is, so let me give you my definition of usability. Usability has everything to do with the user's ability to use our product, in this particular case, our medical device, in the way that it was designed, in the way that the engineers had intended it to be used. And there's a lot of interesting aspects of this that we're going to get into in this discussion. One of them is, who is our user? And historically, in the medical device community, usually the user is thought of as the physician, or the surgeon, or the nurse, or perhaps somebody who... A medical professional of some kind. But now, Jon, as we've talked about before, more and more, the users of our device are the patient themselves.

Jon Speer: Of course, yeah.

Mike Drues: A growing trend for more and more devices, especially home-based devices to be used directly by the patient. And in those cases, and perhaps we can give some examples later, but in those cases, I think, usability and usability testing is even more important if the device is going to be used by a John Q. Public, as opposed to a trained medical professional. So that's a starting point in terms of usability.

Jon Speer: Okay, yeah, I think that's very helpful. I know sometimes there's a lot of terms that get thrown around, that sometimes people use synonymously. Things like usability and human factors and ergonomics, and I think there is a correlation, and I don't expect that we'll get too much into that today, but what is the FDA stand on this topic of usability? I know they have guidances and things of that nature. I know they've been big in human factors and things of that nature, but where do you think the FDA stands on the topic of usability?

Mike Drues: Well, it's an interesting question from a regulatory perspective, Jon. First of all, in terms of guidance, FDA has thus far issued 15 guidance documents in the area of usability testing or human factors. Most of them coming from CDRH. There are a couple from CDER and CBER. By the way, a topic of a more advanced discussion might be the usability issues around combination products.

Jon Speer: Oh, yeah.

Mike Drues: And that's something that I'm getting more and more questions on. FDA has put out a number of guidances, but most interesting to me is that all 15 of those guidances have come out with just within the last three or four years.

Jon Speer: You're right.

Mike Drues: As a matter of fact, historically, FDA has not regulated the usability of medical devices, because one could easily argue that how a physician uses a device is the practice of medicine, and of course, as we all know, FDA does not regulate the practice of medicine. This all changed with one precipitating event, one, what I call a watershed moment, in the history of the medical device industry. It was the, as I refer to it, the infusion pump fiasco that happened about seven or eight years ago. Long story short, there were some 80 recalls from infusion pumps from a variety of different manufacturers, largely because they were not being used properly. And this is why FDA issued now some 15 guidances in this general area. This is not something that was done before. And on a personal note, I might say I'm a little surprised that the industry has not pushed back on FDA to say, "Without specific congressional authorization, you have no business trying to regulate usability, because it is the practice of medicine."

Mike Drues: Now, I want to make sure that the audience doesn't misunderstand what I'm saying here. I'm not saying that usability is not important, and I'm not saying that we should not do usability or human factor or ergonomic studies. On the contrary, I think we definitely should, but to me that's basic engineering, and the question becomes, "Do we need the FDA or our government telling us to do something that we all as engineers know we should be doing anyway?"

Jon Speer: Right, it's an excellent point. And I know you mentioned some of the FDA perspectives on the topic of usability, it's also been a huge topic outside the US as well. And for those of you listening, you might also want to check out... There's an industry standard IEC 62366, which also delves into this topic of usability as well. And I know that's a big deal for products outside of Europe. The big question is, "Hey, did you address 62366?" And it's very common, especially for electrical medical devices. There's an element of going through the 60601 process to show electrical safety, there's a process of addressing usability and going through that as well. So it's bigger than just FDA, it's certainly outside the US borders as well. It's a topic that's hitting the world hot and heavy right now in the med device community.

Mike Drues: I think that's an excellent point, Jon. A subtle nuance under the umbrella of international usability is the differences in usability of a given device in different parts of the world because of cultural differences. In other words, we all know that there are cultural differences. In some places in the world, it's customary to shake hands, in other places it's not, language differences and so on. So this is probably beyond the scope of our conversation today, but one thing we should be keeping in mind when we talk about usability, especially usability around the globe, is what kind of cultural differences do we need to factor in? It might be customary, for example, for people in one part of the world to use a device one way, whereas people in another part of the world to use the same device in another way, so those are things that we need to take into account as well.

Jon Speer: Absolutely, and I think it gets back to your comment of prudent engineering. If I understand the markets where my product will be used and the users in those markets, then I should be doing that anyway, right?

Mike Drues: Absolutely, that's exactly right. And just one other thing I would add, Jon, in terms of the regulatory and specifically the FDA view on usability, a common question that I get is, "What's the difference between a usability study and a clinical trial?" And simply put, here's the way I like to differentiate them. In a clinical trial the focus is on the patient, in a usability or in an ergonomic study, the focus is on the user. Now, as I said earlier, historically, the user was typically the physician or the surgeon, and the patient was the one that they were treating. But nowadays, with more and more devices being designed and intended to be used by the patient themselves, there is no difference anymore. So, in many of the devices that I'm working on now, the usability study and the clinical trial is exactly one and the same.

Jon Speer: Wow, yeah, I know it's... And I think that's... It's kind of like that perception or that perspective is important here, because like you said, the clinical trial, you're looking at how does this product treat the condition or a diagnosis of the situation or whatever the case may be, like that patient perspective. Usability is all about... I'll simplify it, at least in Jon's terms, usability is about all the steps and interactions that the user needs to go through. Turn it on, plug it in, do this, do that. All those things is about how the product works. It's, I guess, from my perspective... Well, maybe it's a question rather, do you see usability as really assessing whether or not the product address the needs of the end user, or more specifically, address and tackle the indications for use of that product? What's the correlation there?

Mike Drues: Well, that's a good question, Jon. So if we bring in what the design controls tell us or remind us, as you know and as we've talked about before, one of the basic tenets of the design controls is to make sure that we design our device to meet the needs of the user. But now it comes back to the question, as I mentioned a moment ago, is who is our user. Is our user a physician, or surgeon, or nurse, or pharmacist, or is our user the patient? And in some cases, in one of our previous discussions, Jon, we talked about the reprocessing fiasco, where certain kinds of medical devices, duodenoscopes, it's a type of endoscope, were not cleaned and reprocessed properly, and as a result, a number of people at UCLA and a few other places were made sick and in a few cases died, because they were not reprocessed properly. Well, if you have a medical device, a reusable medical device, that is both designed as well as labeled to be reused, then it seems to me that one of your users needs to be the reprocessor.

Mike Drues: And so, clearly in those kinds of situations, we did not follow the regulation, because we did not meet the needs of our intended user, in this particular case, the reprocessor. So as we've talked about before, Jon, many people say we need to create new regulation. Well, I really think we don't. I think we have enough regulation, what we need to be doing more is getting people to think, "What does the regulation really mean?" Not what does the words say, but what is the intent of the regulation?

Jon Speer: You know, it's a really good point. And I like the design control tie in here, because really, there's a strong relationship between usability and design validation. Design validation is all about ensuring that the product that you've designed and developed meets the needs of the end user. And to your point, Mike, the "end user" might be the patient, it might be the clinician, it could be the reprocessing firm that's taking the product and getting it ready and fit for another use. So lots of different end users that need to be addressed from a design control perspective, and it gets back to that concept of prudent engineering. When I design the product, and when I establish my requirements, and I do the necessary testing and performance criteria on my product, I need to consider all of those end users. It's not just the doctor, it's not just the nurse.

Mike Drues: That's exactly right, Jon. And not to be nitpicky, but I hear many people, they describe or they think of usability in terms of the ability of the physician or whoever the user is, to use the device. In my opinion, that's not good enough. That's too simple. I would put a caveat on that, "Use the device in the way that the engineer or the designer intended it to be used."

Jon Speer: Yes.

Mike Drues: And the reason why I mentioned that is because there have been a number of medical devices, I have three or four case study examples that I use in my training and in my teaching, where the device worked perfectly fine when it was used to the way that the engineers intended it to be used, but the way that the engineers intended it to be used was not the way that the physicians or the surgeons actually used it, and as a result, they caused problems. And in those three or four cases, all of those devices were taken off the market. So usability needs to be thought of, in my opinion, in a real world sense. People talk today about real world evidence. Well, how about real world usability testing?

Jon Speer: Right. It's really... I've been that engineer who, I'll admit this, that once upon a time I'm like, "I'm an engineer, I'm a smart person going to design and develop this product." But I knew enough about the practice of medicine to be dangerous, and I made a lot of assumptions like, "Oh, well, clearly these are the steps that one would go through to use this product. And I've got these handy dandy instructions for use that I'm going to include with every single device that I provide." [chuckle] And lo and behold, you put it in the hands of a trained physician, and it wasn't used exactly the way I thought. So that was a process that I went through, that I learned the hard way. Fortunately, no one got hurt, and there was no injuries, but I learned, "Hey, bring that end user in sooner into my process." Even if I have a rough proof of concept prototype, get real world opinions, put the product in the person's hands and have them go through some simulated activities to really learn from that. I think that's a real important aspect of usability.

Mike Drues: I could not agree with you more, Jon. I think you're very much preaching to the choir, and you and I have learned these lessons over the many years we've been playing this game. This is one of the reasons why we do these podcasts, to share with audience, especially for those that haven't had quite the experience yet that we have, so that they can learn from our mistakes. But let's use this as an opportunity to tie in the relationship between risk and usability.

Jon Speer: Absolutely.

Mike Drues: And you used the phrase "Instructions for use", or what I prefer to call "Directions for use", which is the low level labeling. In the many companies that I've worked with over the years, when it comes to either risk management plans for design controls, or risk mitigation strategies for regulatory submissions, rarely ever have I seen in their risk analysis the risk of the user, let's assume it's a surgeon, for example, the risk of the user not following the directions for use. And let me give you one not so hypothetical scenario. Recently, I was at a conference, and just coincidentally, the person that was speaking at the conference before me was talking about usability testing, usability studies. And she said that one of the criteria in her usability study was that the surgeon had to read and follow the directions for use.

Jon Speer: Oh, no. [laughter]

Mike Drues: And I raised my hand, you know how I am Jon, because I wanted to get a discussion going there. I said, "Well, you realize that you have now totally invalidated your usability study, because we all know that the vast majority of people never read any of that stuff." And she said, "Yes, you're right, but it passed muster at the FDA." [laughter] So now we get to the fundamental question of, "Okay, we're doing usability studies, but how realistic are they?" In my opinion, just like a clinical trial or any other kind of test for that matter, if it's not realistic, then why are we wasting our time and money doing it?

Jon Speer: No, it's a funny story, because I think so many product developers may take a similar approach. And don't mishear us folks, there still is importance in having those directions for use with your product. The point that Mike is illustrating here is, it should be... Best case scenario, it should be obvious, when you put your product in the hands of a user who more or less what they need to do and how they need to do it. If it's a medical device that's going to be to put inside the body, obviously there's an expectation that you're working with a trained physician. If it's a home use device, you have to figure out, "Does somebody know which buttons to push, and in what order?" and consider that they may not have or ever read those directions for use. I'm an engineer, I'm a smart guy, and Mike, I know you're a smart guy, whenever I purchase something that requires some assembly, I'll admit, I rarely read those directions for use. I think that I can...

Mike Drues: They never do, Jon. [laughter] As a matter of fact, perhaps I shouldn't admit this on a recording that's going to be going out to many, many people, but as a PhD in engineering myself, my philosophy is, if I have to read the instructions on a product, some engineer has not done their job.

Jon Speer: Well, it's a valid point, and I think that's even a more important twist or aspect of risk management, specifically ISO 14971. There was a version that came out in Europe a few years ago, EN ISO 14971:2012. And one of the interesting things about that version of the 14971 standard was on this topic of labeling and instructions for use, that you cannot use that as a risk mitigation strategy to reduce and lower risk. The explanation is that labeling, and instructions for use, and directions for use, that's expected to be part of your product, but those labeling cannot be used as a risk mitigation strategy. And I think sometimes we fall back on that, and sometimes we expect that, "Hey, somebody's going to read these directions for use, and they're going to know how to use our product." And I think that's where, if we're doing prudent engineering and good risk management, that we would identify these things as issues that we need to address.

Mike Drues: Well, you're exactly right, Jon. And that is in fact one of the things that the ISO approach to risk says, that you cannot mitigate risk via labeling. I think personally that's an oversimplification. Part of this depends on your definition of risk, which as we've talked about before, in my world is much, much broader than most other people's, but I'll be honest with you, one of my favorite ways to mitigate risk is via labeling, because other than spending a few minutes of time in wordsmithing, it doesn't cost anything. So here's what I would modify that ISO to mean, mitigating risk via labeling should not be our primary mitigation measure for that particular risk.

Jon Speer: Okay.

Mike Drues: We should do all that we can to mitigate the risk in ways other than just simply labeling. So that would be my two bits on that one. But there's one other aspect of risk and usability that I wanted to tie in to here, Jon, because it's not something that a lot of regulatory folks think about. It is something that engineers should think about, and that is the relationship between usability and product liability.

Jon Speer: Sure.

Mike Drues: We just talked about, from a regulatory perspective, we can, although it's not desirable, mitigate risks from a labeling perspective. But I can tell you, when it comes to product liability, and I'm extremely fortunate a small but actually growing part of my business is doing expert witness testimony for medical device product liability cases, including a couple of very, very high profile ones right now. Attorneys, a good product liability attorney, has a field day. You don't have to have a JD from Harvard Law after your name to appreciate that you could easily make a case for liability if the usability was contingent on the physician reading the instructions or something like that.

Jon Speer: Sure, sure.

Mike Drues: So one of the take away messages for your audience here, I think, is for me, anyway, is usability testing is obviously very important, as we've both said, it's basic engineering, it's prudent engineering, but I want to do what I call real world usability testing. And I do not make the assumption, with all due respect to my many friends at FDA and the other regulatory agencies around the world, I do not make the assumption that just because my usability study passes muster at FDA or Health Canada or somewhere else, means that it's a good or a realistic study. I do not make that assumption. And so what I like to do, many companies are leery of doing this, obviously, is put the device in the hands of the user for the very first time, and don't say or do anything, and watch exactly how they use the product. And if one of the things, one of the first things they do, is they pull the DFU out of the box and read it, then note that down in your usability study. But if one of the first things they do is pull that DFU out of your box and throw it in the trash, [laughter] then you should note that down as well. [laughter]

Mike Drues: And now, we have to be a little careful, because if we design a usability study to be truly realistic, as I just discussed, now we get into the possibility of, "Well, if they start using our product in a way that it was not intended, is that off-label use?" And now we're talking about collecting usability data potentially to submit to the FDA for off-label use. Believe me, this is a complicated issue, but simply put, I think there are better ways that we can skin this proverbial cat than we're doing today.

Jon Speer: Sure. Absolutely. And bringing in this concept of risk, as Mike suggests, putting that product in the hands of the end user and really just being in observation mode is important. And with that being said, as a product developer or a medical device designer, you don't want to wait until the very end to learn these things. You don't want to be at the point where you're two weeks away from a 510 [k] submission, for example, and put the product in the hands of the end user and learn that, "Oh, wow, this is not being used the way we thought it was going to be used." This is the benefit of doing some prototyping earlier in your process and making some concept designs and so on and so forth, and really getting that end user, whether it's a doctor or a patient themselves, involved in your process, because that's going to help drive and influence what you do from a design and development perspective. And it really is really an important aspect of how risk management gets integrated and infused as part of your design and development process, as well as the usability.

Mike Drues: Once again Jon, that's an excellent point. I could not agree more. And one of the last things I would like to bring up here before we start to wrap this up is, and I hinted at this earlier, it's one thing to talk about the importance of usability testing for devices that are being used by trained medical professionals, in other words, people that have graduated from medical school or nursing school or dental school or what have you, but it's quite another thing when you talk about usability for a device that's being used by the individual patient in an over-the-counter device, or in a wellness product, or even in regulated devices that are intended to be used by the patient themselves. In those particular cases I would argue that usability is even much more important than in devices that are going to be used by trained medical professionals. Because at least somebody who has gone to medical school, you would like to think that they know what they're doing, although that's a huge assumption. Somebody that I used to teach medical school myself back in the day, I was constantly reminded of the old joke, "What do you call the person that graduates last in medical school?" [laughter]

Jon Speer: You call them doctor, you call them doctor. [laughter]

Mike Drues: That's right, you call them doctor. Or the same thing can be said about attorneys, and about engineers, and about regulatory professionals.

Jon Speer: Of course, of course.

Mike Drues: So usability in OTC devices, in wellness devices, in devices that are going to be used by an untrained professional, that, in my opinion, is infinitely more important. And one more time, coming back to risk, I think that has to be tied into your risk mitigation strategy.

Jon Speer: Absolutely.

Mike Drues: As a matter of fact, as you know, Jon, as we've talked about before, I spend a lot of my professional time helping companies going through the pre-submission or the pre-sub process, when we prepare the pre-sub package to submit to the FDA prior to the actual meeting. One of the last things that we do is we go through and we establish links, hyperlinks throughout the documents. And the number of links that we're creating in the last dozen or more pre-sub packages, that I've been involved in putting together, is just amazing. The number of links between usability, and the testing matrix, and risk mitigation, because everything is interrelated, everything depends on everything else.

Jon Speer: Absolutely, absolutely. And Mike, that's a big part of why we exist at greenlight.guru, because you got to keep all of this information connected throughout the entire design and development process. Being able to trace how your user needs relate to your design inputs and your design outputs, and verification, and validation, and usability, and risk management, and ultimately what you're going to be compiling in some sort of submission or technical file, being able to tell that story clearly, and succinctly, with clear objective evidence, and demonstrating your prudent engineering is so, so important.

Jon Speer: So if this is something that you want a little bit of help with, I do encourage you to go to greenlight.guru, and learn a lot more about what it is that we do. And if you want to reach out to us, click the button, and we'll be happy to give you a ring and help answer all the questions that you might have. Also, Mike Drues, Vascular Sciences, this is what Mike does for a living. So any time you have questions, reach out to him, it's pretty easy to find on the internet. If you type in 'Mike Drues', D-R-U-E-S, you'll be able to find his wonderful content and work all over the place in this industry. You can find him on LinkedIn as well, but Mike's happy to connect with you and have a conversation.

Jon Speer: Speaking of conversations with Mike and with Jon, be sure to check us out here in a... Well, I guess it's right around the corner, Mike, MD&M East. We're both going to be part of the design control topic track at the MD&M East show in New York in June. I'm looking forward to connecting with you again face to face, it should be a good time.

Mike Drues: Thank you Jon, I look forward to seeing you as well. And for anybody in the audience who's listening, who happens to be at the show in New York, please do come up and introduce yourself to Jon and I. It's amazing to me, Jon, how many followers we're gaining now as a result of the podcast series, so I think that's terrific.

Mike Drues: And I just want to thank you again for all of your kind words and the opportunity to have this discussion. It's a very important and a very timely topic, usability, and specifically the interception between usability and risk management and design controls and so on. And like many of the discussions that we've had, we've talked about some great things, but this is just scratching the surface.

Jon Speer: You got it Mike.

Mike Drues: There's a lot more meat on these bones [chuckle] that we haven't got to, and I look forward to discussing more of that as we continue in the future.

Jon Speer: Absolutely. And folks, you're right, Mike's right, this is the number one medical device podcast in the world. So, tell your friends they should be listening to this as well. And Mike, thank you again for being a guest, and this has been Jon Speer, the host, founder, and VP of Quality and Regulatory at greenlight.guru and you have been listening to the Global Medical Device Podcast.

About The Global Medical Device Podcast:

The Global Medical Device Podcast powered by Greenlight Guru is where today's brightest minds in the medical device industry go to get their most useful and actionable insider knowledge, direct from some of the world's leading medical device experts and companies.

Like this episode? Subscribe today on iTunes or SoundCloud.