Quality vs Compliance Metrics (and what's next for FDA's Case For Quality)

What is the FDA looking to drive forward in its Case for Quality initiative? 

Here’s what we learned:

What’s happening now?

One of the big focuses of the voluntary pilot is the idea of driving improvement across the industry - both for manufacturers and regulations. Vicenty shares an example of a company that has shared how its assessment results have changed over the course of the pilot. As he emphasizes, the FDA’s role isn’t to hammer on all the areas that aren’t yet green; instead, what they’re interested in seeing is the journey, that there has been consistent improvement over time.

Source: Cisco Vicenty, FDA Case for Quality Program Manager

The particular company whose results are shown was able to drive improvement across at least 7 key areas during the course of the measurement over the pilot period, including seeing the implementation infrastructure area (“red” on 27% in 2016) move up 13% to “yellow” on 40% for 2017. As shown in this diagram, this idea of driving improvements from a systemic point of view paid off for other parts of the system.

The FDA is collecting updated data points across all participants in the voluntary pilot, resulting in much better visibility into what’s working or not working. For instance, it’s much easier to see whether something seems to be a problem for a particular manufacturer, or if there is a pattern of the same problem across many. This transparency gives the FDA a more objective vantage point to extrapolate the data of why certain problems may exist.

The FDA is spending a considerable amount of time delving into its systems and processes to decipher whether there are certain processes that aren’t working as well as originally intended. In turn, manufacturers are able to get better insights from their end to identify any outliers present that may be negatively impacting operations. The FDA has performed these checks across a broad selection of different manufacturer types and sizes throughout the pilot and will carry on with these efforts in order to acquire valuable insight for continuous improvements.

In terms of learning experiences, the FDA is seeing that, from a systemic approach, there are things that are not working as well as they would have intended. Some named examples would be CAPA procedures and systems of measurement.

Free Resource: Want to know which quality domain metrics were endorsed by the FDA? Download here.

What are they changing?

The FDA is looking at how and why it gets the information it does. Is there a better way or a “least burdensome” approach? Importantly, can it achieve the outcome it is looking for (safety, effectiveness) in a faster, more reliable fashion?

Overall the FDA is finding that it can implement more changes in a faster time frame than what it could on an individual review basis. It is also seeing a key mindset shift among participants - from a compliance focus to problem-solving and from hesitancy to engagement.

Collaboration and learning have increased, with ideas being pitched from manufacturers and FDA. It has seen a shift from “assumptions and reacting” to “understanding and insight.”

Compliance vs. Quality Indicators

Another key goal of the pilot is to get to the point of measuring and monitoring what really matters. The pilot is looking to drive a better product, better outcomes for customers, and, as result of that, better outcomes for the patient.

One thing that the FDA did with participants from early on is to ask for what they thought were important measurements to use. At the beginning, they were given a lot of “here’s what you want to hear” compliance indicators. The FDA wanted to know what indicators people would look for that tell them they need to take action. As he says, this took some trust-building. People wanted to know what they were going to do with that information and what it meant.

In more recent times, the indicators given by the companies have shifted. For example, more are focusing on employee health and safety as a big deal, which leads to other indicators being looked at within the organization.

You can see a comparison of early participant versus more recent performance indicators in the chart below. Note how the more recent measures given tend to provide a more robust picture of quality than those earlier suggestions.

Source: Cisco Vicent, FDA Case for Quality Program Manager

The FDA wanted to see a focus on results gained, rather than a company focus on compliance. Areas, such as CAPA and Operational Effectiveness, have shown promising improvements among participants, along with several other areas. The underlying point is that quality is an overall focus, rather than simply being compliant with an item on a checklist.

Of further note is that they’ve received some great suggestions in terms of quality domain metrics from participants. Metrics covering safety, effectiveness, reliability and availability have been extrapolated on to provide a framework that any medical device company can follow.

Why it matters?

Why is there a focus on quality indicators, KPIs and metrics? One of the things the FDA is looking to do with companies is being more objective. It wants to see the result of the process so that the focus isn’t on contain, control and fixation on the process.

Visibility And Innovation

The idea is that once a process has been established and approved for the organization, they are able to really innovate around the process and put more focus on the results that it generates.

This gives the FDA a lot more visibility over what is happening in the organization and gives the organization the ability to align the process with their business. The shift is from a compliance mindset to a quality improvement mindset.

This allows companies to address any issues quickly and effectively as possible. There will be issues that come up, but a quality system is developed with the intention of being able to account for that happening.

Least Burdensome

As Vicenty puts it, the FDA would like those quality and safety outcomes, but it also understands that a medical device company is a business. How can it work with business and quality needs to deliver better outcomes for both?

The slide below from Vicenty highlights key ideas for why this program is important:

Source: Cisco Vicenty, FDA Case for Quality Program Manager

What’s next?

This current pilot has focused on the manufacturing side, but in the future, the FDA would like to expand to other quality inputs, such as design, and supplier management.

The FDA would also like to continue making improvements to enable speed, responsiveness and overall improvement to their 510(k) process. This goes hand-in-hand with a focus on quality inputs.

In terms of systems, the FDA is keen to improve regulatory system elements and bring any back closer to its original intent, where necessary. It would like to increase industry adoption of tools, practices and performance measurement, and establish a full FDA program by 2019.

Free Resource: Want to know which quality domain metrics were endorsed by the FDA? Download here.

Final Thoughts

The FDA’s Case for Quality is a shift in how manufacturers and the FDA has traditionally worked together. The focus on having a collaborative environment where quality is emphasized over mere compliance is a great step toward overall industry improvements.

We're looking forward to seeing more from this program, and the continued establishment set for 2019. Here at Greenlight Guru, helping medical device manufacturers focus on quality is our goal, so the FDA's initiative aligns very closely.