Tips for Running Better Management Reviews

How efficient are your management reviews? Could improvements be made? It might be time to reevaluate and reconfigure your system processes to eliminate burdens and extract utmost value from these critical checkpoints while still satisfying compliance needs.

In this episode of the Global Medical Device Podcast, Jon Speer talks to Taylor Brown, senior medical device guru at Greenlight Guru.

Together, Taylor and Jon discuss the nuances of management reviews to help medical device managers and executives make better decisions for teams to achieve greater outcomes.

LISTEN NOW:

Like this episode? Subscribe today on iTunes or Spotify.

Some highlights of this episode include:

• Learn the must-haves of management reviews, such as inputs, feedback, complaints, reporting, and audits during planned intervals, by following ISO 13485 and FDA 21 CFR Part 820 Quality System Regulation.
• How often should management reviews be performed? The industry standard is once a year, but a lot could happen in 12 months. Greenlight Guru recommends management reviews twice a year, especially in the early parts of your business.
• Internal audits and management reviews are two activities that offer a snapshot in time of the overall picture of the health of your QMS. Then, the management review is a platform to escalate issues.
• Best practices for a management review include where more research, time, and effort should go. Don’t mistake a management meeting for a review. Pulling off a management review last minute is a lot easier than an internal audit.
• Your company’s key performance indicators (KPIs) and goals should be put on the agenda as a checkbox activity for the management review. Yet the list of topics/objectives should be discussed to assess how you handle your business.
• Document any decisions and actions to maintain the suitability, adequacy, and effectiveness of your QMS. In meeting minutes, include an action plan and product improvements related to customer requirements and correct problems.
• You can share your management review meeting minutes with notified body auditors, but not the FDA.
• If you have to hurry up and hustle to get data, is it reality data that you need? Is it that important? Automate, divide, and conquer.
• Design your systems and processes in such a way that having a management review doesn’t become a burden.

Links:

ISO 13485 - Medical Devices

FDA - 21 CFR Part 820 Quality System (QS) Regulation

Taylor Brown on LinkedIn

Greenlight Guru Academy

The Greenlight Guru True Quality Virtual Summit

MedTech True Quality Stories Podcast

Greenlight Guru YouTube Channel

Greenlight Guru

Memorable quotes from this episode: 

“The standard is pretty general on how often you should be doing management reviews.” Taylor Brown

“Management review is really a platform to escalate issues.” Taylor Brown

“Pulling off a management review last minute is a lot easier than trying to pull off all of your internal audits last minute.” Jon Speer

“Executive management needs to be aware of and involved with the health of the quality system.” Jon Speer

Transcription:

Announcer: Welcome to the Global Medical Device Podcast, where today's brightest minds in the medical device industry go to get their most useful, and actionable insider knowledge direct from some of the world's leading medical device experts and companies.

Jon Speer: Man, I just had a great time catching up with Taylor Brown, senior medical device guru on the customer success team at Greenlight Guru. She is a quality system nerd, folks. And I mean that with the highest of compliments. On this episode, we talked about management review. I mean, that was bananas. Who has that much fun talking about management review? Well, Taylor loves it. I had a great time. So, I hope you enjoy this episode of the Global Medical Device Podcast. Hello and welcome to the Global Medical Device Podcast. This is your host and founder at Greenlight Guru, Jon Speer, and joining me today is senior medical device guru from the customer success team at Greenlight Guru Taylor Brown. Taylor, welcome back.

Taylor Brown: Hi, Jon. Thanks so much for having me.

Jon Speer: Absolutely. And we were just catching up a little bit beforehand, so and that fan it's going to come to a stop. I think so bear with us. We'll get there. You have a wealth of knowledge on a lot of things, or many things quality system related. And I reached out to you the other day like, "Hey, can we talk about management review?" And at first I'm like, "Is there a lot to talk about there?" And then the more I thought about I'm like, "Oh, yeah, nobody gets this right." So, yeah, let's dive in. So you just like started to share a little bit of a story, do mind being a little vulnerable for a moment and sharing one of your stories? You can say no.

Taylor Brown: No, oh, my God.

Jon Speer: ...putyou on the spot.

Taylor Brown: Learn from my experience, dear listeners, heed this warning. So, I was sharing with Jon in my previous position, I was in medical device distribution and I was a quality engineer. And for whatever reason, I got put in charge of management review and I was so excited. This was going to be the world's best management review. We were going to set records with how good this presentation was, it was going to be bananas. But so I worked really hard and put these slides together, got my meeting minutes. I worked with an analytics team to pull charts and graphs and everything. Managements flying in, just the site, like lunch is on the way this is going to be awesome. And we put through, it was crazy like a four hour management review. I'm not even really sure what we talked about, but I came out of that meeting just, I was walking on air. It went so well, and we made really great decisions, cut to a couple of months later, looking at management review and our notified body audit for ISO 13485 and the auditor looking, and she asks for the slides and the meeting minutes. And she goes, "Where did you talk about your complaints?" We didn't really talk about complaints. We talked about how like nonconformances are doing, and customer surveys that we've put out, but no, there's really no complaint dated. She's like, "How are you supposed to report to management review the health of your customers and their happiness with the product and regulatory reporting if you're not talking about complaints?" I'm like, "Oh, man. Yeah, you're right." I really shouldn't have spent those 30 slides talking about like audit etiquette and the body audit we had coming up. I should have just gone through the standard and done what the standard told me to do. I went above and beyond as I tend to do, but it kind of bit me this time.

Jon Speer: Well, but it was a good experience. It sounds like.

Taylor Brown: It was a good experience. Yeah.

Jon Speer: So, I guess for those listening let's remind them where they can learn more about what are the must haves in a management review. And I think there are two sources here that are probably the most relevant. Certainly it's pretty well defined in 1345. I think it's also decently defined and FDA 820. So, do you mind reminding folks, these are the "thou shalts" from a management review perspective.

Taylor Brown: Right. And you're correct. The FR, and then also ISO 13485. I think ISO has, honestly, more of a checklist type of view. So it says your inputs they're in section 5. 6. 2, I have it up in front of me. I don't have it memorized. It's feedback, complaints, reporting, audit, all of that. So, the standard is pretty general on how often you should be doing management reviews. I think it says planned intervals, which, I mean, you could take once a month, you could take once every five years, I'd say industry standard is once a year. However, once a year that's every 12 months a lot could happen in 12 months. So I think here at Greenlight, we really recommend doing it twice a year, especially in the early parts of your business.

Jon Speer: Yeah. And from an FDA perspective, I think this is one of the areas. There are a lot of areas between 820, and 1345 where they're very much in sync. And I think there's a lot of similarities, or there's nothing that's an ISO that is not an FDA and vice versa. I think the FDA requirement, if I recall, and it's been a minute or two since I've read this, but I believe the expectation is that it's done at least annually. So, but to your point, the periodic, I think I've had quite a few experiences with management review and I've seen them done annually by a company, because that was the minimum criteria. And I've also seen companies that they just go through a checklist and I'm like, "Hmm, this isn't really getting it. I'm not..." This is just one of those moments. I think unfortunately, a lot of companies, they treat it as that checkbox while we got to do it, so schedule it in September or whatever. And we'll go through a little checklist and be done with it.

Taylor Brown: December 21st management review. I was going to say what are you talking about on December 21st? Like no one is mentally in the office at that point, I think internal audit and management review. Those are the two things that I bring up when I hear a company is going... Their stage one or stage two audit is coming up. I say, "Have you done your internal audit? Have you done your management review?" And I group both of those activities together, because it is a snapshot in time of the overall picture of the health of your QMS, right? Management review then is a bit different because management review is really a platform to escalate issues. If your director of quality is sitting in the management review, this is not their first time looking at the CAPA system. This is not their first time looking at it.

Jon Speer: I hope not.

Taylor Brown: I hope not. Let's add that caveat there, but for the CEO, who's not helping crunch data and not looking in the analytics display, they want a snapshot of what's happening and they need to know where they need to be concerned, where more resources should go where more time and effort should go. So just documenting the escalation points there.

Jon Speer: Yeah. Let's maybe talk about some best practices or things that make the management review more effective. To your point, I think it is one of those forgotten activities, strangely, that right there with internal audits and I've seen it happen so many times. A company's like, "Oh, crap, it's December. We haven't done our management review yet this year." Got to squeeze that in. And then you get into the holidays and the end of the year and all that sort of thing. And it's like, it becomes a challenge which pulling off a management review last minute is a lot easier than trying to pull off all of your internal audits last minute, but we've talked about that, I think, before, and we can certainly talk about that in a different episode. But so you're saying at a minimum two times a year?

Taylor Brown: A minimum of two times a year. Yeah. And a lot of companies will say, "Let's do it monthly," Don't mistake, a management meeting for a management review. You can meet with management as much as you want, please by all means set up a meeting every Monday morning with management, but a management review, per ISO 13485 per the CFR has a very specific set of requirements. So, that's my first step is to create a meeting minute template that consists of those requirements. So, and you'll never hear us say it's a checkbox activity, but the agenda is kind of a check box activity. It's more of a list of things you need to talk about. There's a lot of conversation there, but don't be me and not talk about complaints because you didn't include it on the agenda.

Jon Speer: Yeah, so the agenda is cookie cutter, as far as the topics. Now, for example, you're going to talk about complaints. You're going to talk about CAPAs and nonconformances et cetera, et cetera, but within each of those topics that is not cookie cutter because you're going to talk about the new business from the last time that you had a management review to this time and make sure you're holistic and comprehensive regarding to that.

Taylor Brown: Exactly. And I think you talk about what your KPIs are, your key performance indicators, but also talking about what the goal is. I think sometimes companies say here's our CAPA closure data, and it's just this big bar chart. But what's your goal? What are you comparing that information to? Are you taking too long? Are you doing a great job of closing CAPA's? So typically it's in your analysis of data SOP, or your quality objectives to see what goals are you meeting, and that's really your threshold for analytics in your management review.

Jon Speer: You saying that reminded me, I'm pouring another cup of coffee, but you saying that reminded me of the time I got dinged an ISO audit. In ISO speak one of the key parts of management review, and frankly the health of your quality system is defining your quality objectives. And I got dinged because I don't remember specifically what the verbiage was, but they were like warm, fuzzy, ambiguous quality objectives. Like we need a good quality system, but we didn't put in any like KPIs or metrics to that. And so I encouraged those management representatives and heads of quality out there, evaluate your quality objectives and make sure that they are stated in such a way that they're measurable define those KPIs. I mean, there's a methodology that we use internally at Greenlight from time to time called OKR. So that's a way. O stands for objectives, KR, key result. But there's lots of other ways to do that too, but just make sure that you have criteria defined for specific key processes. And I think with that though, I think I see a lot of people make mistakes on the KPIs. Like you mentioned CAPA like they'll say close the CAPA in 30 days. And that's not really a great KPI.

Taylor Brown: Yeah. Not every CAPA is going to be closed in 30 days, but though the work in the CAPA should almost dictate when the due date is. So if you, and this isn't for management review, but just in general, if you find yourself going over these goals that you have in mind, first of all, document that rationale. If anyone who's talked to me in a Greenlight call, or webinar, they know I love memos. So write your rationale before your auditor asks you about it. And then also use management review as a way to say, are these the goals that we want, or we're consistently meeting them. I mean, should we have more challenging goals at that point? Or should we reel it in and say, you know what? That was a good idea when we had one product on the market, but now that we have five, we need to really adjust.

Jon Speer: Yeah, and I'll stick with that for a moment, too, because I think I've seen a couple of... And I'll stay on the CAPA thread from a management review perspective where a company has a KPI close CAPA in 30 days, or whatever it is. And the data shows that they're actually doing that. But it's like, okay, now you've got to peel a little bit deeper, peel back another layer. How many repeat infractions, how many repeat CAPAs have you had? How are you closing CAPAs just for the sake of hitting that metric, but you're not really correcting, or preventing the issue from happening again. Right? So just be critical of your KPIs and look at your quality system as a way that described how you do business. Now, there are some people out there that are going to have a vehement negative reaction to that like, "Oh, no, that's not how it," but it is all right? These are the key processes that describe what you do, how you document things, how you review and approve things? What happens when there's a complaint or a systemic issue, these are all described within your quality system. So the management review becomes a moment in time where you're assessing the health of your business, I think. I don't know. What do you think?

Taylor Brown: No, I think so. And we talked about the inputs from ISO 13485. There's also the output section, which I think some people think, "Oh, an output of my management review is the meeting minutes." Like physically, yes, that is the output, but what's more important there is in ISO 13485, you're supposed to document any decisions and actions. So you don't need to take action today. You can come up with an action plan, but what needs to be done to maintain the suitability adequacy effectiveness of your QMS? I think Jon, that's what you're relating to. If have the same CAPA of three times, we're going to get really good at solving it, but that's not really maintaining the effect of this. That's not a suitable goal anymore because we're really just doing the same thing. So, in our meeting minutes, we had a separate section specifically saying what needs to be done to maintain the suitability, adequacy, effectiveness, and we'd come up with an action plan. There's also a question there on any product improvements related to customer requirements, that was its own section changes needed for regulatory. And then do we have enough resources? Yes or no? And if the answer was no, we came up with an action plan and really documented it much like we would a CAPA.

Jon Speer: Right. So let's talk a little bit about MRB management review board, or CAPA review board or quality review board. There's a few different names. Yeah. How are these the same or different from managers?

Taylor Brown: I think the smaller boards would be almost an end point input into management review, rather your CAPA review board, the CAPA team. They're going to know everything about CAPA, but they still need to see that management review. Your material, or review board. They know everything about incoming inspection and what's coming through the door, what the product needs. They need to see that management review.

Jon Speer: And those smaller groups meeting probably every month, right? Those are the teams that are meeting with more frequency.

Taylor Brown: Right. And that's why I think management review it's, I mean, we made fun of it like is management review of fun topic, but it's really is a fantastic idea and a very noble quest to get all of these people who know everything about their department, all in the same room, talking about what they need. That's really the intent for management review.

Jon Speer: Do you know that diagram from the FDA QSIT guide? There's a diagram in there that shows all the major subsystems. Do you remember the one that's at the center? It says management, right? I think that the intent of the regulations and the requirements of the standard is executive management needs to be aware of, and involved with the health of the quality system. So I think this is the intent behind management review is to, or I'm using air quotes here, force management to have a meeting so that they're involved with the health of the quota system.

Taylor Brown: Exactly. And I think that's why my team in my previous life, we got so excited to do this big presentation, because we're in the trenches. We're in the trenches, and CEOs flying in to listen to the presentation. So, we said, "What do we want to make the CEO aware of, so they can help us?" And we were a little dramatic in the sequence of events, but the intent was there. And I think if I could go back in time and just make it a little more straightforward, make it less of a event and more of a meeting with measurable results, and measurable actions, then we would have had a better result.

Jon Speer: All right. So I'm going to ask your opinion on a couple of things. So, we talked a little bit about KPIs. I think there's an expectation that you're defining your critical processes and things of that nature, and you probably should have metrics assigned. I think it was Deming that said, "If you don't measure it, you can't improve it," or something like that. It might not have been Deming, but that's one of those...

Taylor Brown: Believable.

Jon Speer: It's one of these quality gurus from back in the day. And I think it's true. If you're not measuring it, how do you know if it's getting better, worse, the same, et cetera. So assigning a good metrics, and good metrics, that's a whole topic all by itself. We're not going to dive into that today, but if I have these KPIs that are defined, can I create dashboards, and things that I can make available and visible to my team to the executive team? And between management reviews, what are your thoughts about that?

Taylor Brown: Yeah, I think definitely it goes back to the idea this shouldn't be the first time we're looking at it. And if it is, we have bigger issues at that point, but just keeping track monthly, quarterly, of where you are in the process. So then when you get to management review, A, you're not having to pull together 12 months worth of data, in a three- day period. That doesn't sound too enjoyable. And B you already know what you need. So when you go into the management review meeting and CEO is listening, you can say, "Here's where our data is today. Here's where it needs to be. This is what I need to get there.”

Jon Speer: And I think you just hinted at another problem or challenge with management review. A lot of times companies don't have systems or mechanisms in place that where they're keeping the data and information up to date, close to real time. I remember preparing for one of these and I swear to you, it took me a week to go into all the different systems and spreadsheets. Sometimes I had to go like do manual data collection just in order to prepare information on how things were trending. And if you find yourself in that situation, any words of wisdom, or advice to folks that might be in that stuck in that role right now?

Taylor Brown: Greenlight Guru has a analytics source. No, I think coming up with... We talk about in compliance, meeting the regulation and just that, not going above and beyond and having 12 management review meetings a year when you only need the one or the two. Same would go supplier evaluation. If you have a KPI for supplier management, let's not put an onsite audit every six months. So if you are finding yourself having to hurry up and hustle to get this data, ask yourself, is this really the data we need? Will I subset for management review? How important is all of this? And maybe it's all important, in which case I would tell you systems like Greenlight Guru can help you automate it. Dividing and conquering between different teams, of course.

Jon Speer: I mean, I think that's the key. Design your life and your systems and your processes in such a way where having a management review doesn't become a burden. I mean, I think this is maybe a little bit utopian thinking, but design your systems and your processes in such a way that you can have a management review, anytime, ad hoc. If somebody said, "Let's have one tomorrow," hopefully your systems are robust enough that you can actually do that tomorrow if, and when you need it to, right?

Taylor Brown: Yep. Yep. Stay ready so you don't have to get ready.

Jon Speer: Yeah.

Taylor Brown: ...aswell.

Jon Speer: And yeah, there's all sorts of cliches and metaphors different sayings on that. Let's also unpack this because I know this is sometimes a myth, and I'm not trying to trip you up. So I don't know if you know, but can an auditor see your management review minutes?

Taylor Brown: I have to give a shout out to my partner in crime, Sarah Adams, because I knew this question was going to come up and I messaged her this morning. I'm like, "Hey, I'm doing a podcast with Jon, can you just double check me?" So, you're notified body audits. Management review will come up, obviously. You can show them your management review, meeting minutes, FDA off the table. When we did our FDA audit, we showed them like, first page of like, "This is when we had the meeting. This is who was there. That's it. That's all they got to see." So the FDA cannot get into your dirty laundry. Notified bodies are going to be all in it.

Jon Speer: Yeah. A hundred percent, correct. So, I think, from an FDA perspective, you need to demonstrate proof, or evidence that you've conducted a management review. Same as the case from an internal audit perspective with FDA, you have to demonstrate that you're conducting those. So, like for example, if you were using Greenlight for your quality management system, the evidence and proof would be a signature page with a date and timestamp and all that sort of thing, but from an ISO perspective, yeah. They can and likely will drill into the details of your management review as well as your internal audits. I hope this thinking is gone, but I know like when I started in the industry, there was some people that were just very adamant. Absolute. That no, FDA can't go there. And like it's some sort of witch on or something. I'm like, "Man, that's not really what this is about", but let's be real FDA, although they won't go into the details of your management review or your internal audits, they're going to look at your CAPA system.

Taylor Brown: Yep.

Jon Speer: So...

Taylor Brown: If there's an issue in management review, they're still going to find it. You're presenting the showing of the meeting minutes.

Jon Speer: Though, I just advise people, don't get hung up on that. That is not the thing to worry about. All right. Any other thoughts on management reviews that we haven't covered, or tips that you want to leave the listeners with?

Taylor Brown: I think we talked about it, but just as a final kumbaya moment for your management review, route the meeting minutes, have people sign off on them. Just route them to the attendees to say in solidarity, "Yes. I agree with these meeting minutes," especially if there were decisions made that have people's names on them, make sure that those people are reviewing the meeting minutes, keep your slides accessible and controlled. As we've talked about today, if you need to clean up the slides before you store them, or before you show them in the presentation, the actual management review meeting minutes, keep the slides, the bare minimum, let the conversation guide your discussion obvious and keep the meeting minutes there. But slides should be, as we've talked about the visuals, are we meeting our goals or are we not? I think, again, tying it back to the story at the beginning, I got carried away with explaining why we were here and that's where the auditor spent most of the time in her review of the slides. So, keep it simple, keep it sweet and just make good decisions that come out of it. It's really going to help you in the long run.

Jon Speer: You just said something there that reminded me of another mistake that I made once in management review. There will likely be action items that are identified in your management review. You want to make sure that your minutes reflect those action items and who's responsible and so on and so forth, but here's the gotcha that got me, anyway. The following management review, we didn't speak about the update on those action items from the previous management review. So, if you identify action items on a management review, those, now, become agenda topics for your next management review and you should have a system or some way to track those action items so that you're working on closing the loop and tying up on the end. So that will be another thing to be cognizant of, because it will likely come up. So anyway, Taylor, I think we just made management review like fun and exciting.

Taylor Brown: I think we did. I think it was always fun and exciting. We just didn't want to do it. We just didn't have the time.

Jon Speer: ...we didn't want to do it, because we knew how much time, effort, and energy we were going to spend collecting this data and information. And it was going to suck us away from our normal responsibilities. Right? And I think that's the exciting part about Greenlight Guru, and folks, check it out. Greenlight Guru has a medical device success platform, a software to help you manage your quality system and workflows, including analytics that are going to go a long, long way in making your preparation for management reviews so much simpler. I mean the workflows for CAPAs, and complaints, and nonconformances, and audits, and documents, and records, and change control. And I'm probably missing a lot of things, but these are all workflows and the Greenlight Guru platform and there's analytics and data and charts. And you can pick time periods of what you do want to look at these things. So these are all things that are right now available at your fingertips in your Greenlight system. So check it out, www.greenlight.guru. We'd be thrilled to have a conversation with you to understand your requirements and your needs and see if we have products and solutions that work for you. And then if he needed a little bit of help and guidance we've got med device nerds like Taylor Brown and the rest of our Guru team that they'd be thrilled to help you navigate this process. So, anyway, Taylor, always a pleasure. It's great to catch up and talk about management review. Taylor Brown, senior medical device guru, any parting shots or words or anything like that?

Taylor Brown: Make your management review meetings fun. Order lunch, that's always my last tip and trick. Just order a lunch, make it a meal.

Jon Speer: ...food. If you give food, typically, people show up when there's food, so, yeah, absolutely. All right, folks, thank you so much for listening to the Global Medical Device Podcast, the number one podcast in the medical device industry, and that's because of you. So keep spreading the word. Hopefully, I figured out now that we have video, so if not go check out our YouTube channel, we're pushing the Global Medical Device episodes there for you to watch. You can also watch them on the Greenlight Guru's website. Probably we'll also have links to videos on our LinkedIn page. So you'll find it. I'm sure you've already found it, but if you are watching it on video, on YouTube, you can subscribe to our channel. You can click the bell notification that way you get alerted when there's new content. So, thank you so much. As always, this is your host and founder at Greenlight Guru, Jon Speer, and you have been listening to the Global Medical Device Podcast.

ABOUT THE GLOBAL MEDICAL DEVICE PODCAST:

The Global Medical Device Podcast powered by Greenlight Guru is where today's brightest minds in the medical device industry go to get their most useful and actionable insider knowledge, direct from some of the world's leading medical device experts and companies.

Like this episode? Subscribe today on iTunes or Spotify.