Karen Ginsbury’s Quality Blog #6 (2022/#2)

May 19, 2022

Continuing the theme…How will we achieve an effective #QMS? Is it through Metrics?

Second blog of the year, so better than 2021 – continuous improvement? Time will tell.

The mask situation – no longer required even inside in Israel but still required on plane. As of tomorrow night, no more incoming tests in Israel so will be able to travel to most destinations with no Covid tests which is nice. Are we done with Corona?

The war in Ukraine has sent Covid into the shadows – and increased supply chain complexity. Shortages are aggravated and worldwide we are seeing commodities missing from the shelves of many stores – whole sections empty. To say nothing of fuel prices. How are you managing your supply chains within the framework of qualified suppliers and missing items? No mean feat –considerable ingenuity and flexibility is called for, both by your company and by the regulators, unless we are to experience horrendous shortages – something no one wants.

Since the previous blog, FDA has published the Quality Metrics Program in the Federal Register of March 09. Comments are requested. If you want to submit get them in by 08 June latest. The pilot program was conducted at just 14 sites worldwide. It is not clear how many companies were involved but possibly several sites belonged to a single globally operating, company. Can this be considered a representative sample for determining requirements applicable to thousands of companies around the world?

We need to think about the purpose and place of metrics in an effective QMS as opposed to their use in a GMP environment. In creating and maintaining an effective QMS, we use metrics to measure and follow up on how well we are doing – and to try and influence and change behaviors. This is a voluntary activity, usually management-driven with a view to improvement: efficiency and meeting requirements (achieving quality). In a GMP environment, the use of metrics is a tool for inspectors to write up observations and to rapidly identify risks and pull your company up. You may say “we have nothing to fear” which would seem to be a good attitude. Recent inspection experiences tell a different story, as does FDA’s written reason for collecting the data.

FDA states they are looking for information which implicates companies and will trigger inspections / regulatory action. Consider “Mean Time Between Failures” (MTBF) – an important metric. One which every company should consider implementing. Decreasing time between stops on an automated machine, indicates the equipment being monitored is faulty / failing or ageing. We should all want to know about that ASAP. It is wasteful and it manufactures sub-standard product

during stopping and starting – this is invariably the case. If you discard units during machine stopping and starting (e.g. on a tableting machine), the more stops and starts the greater the waste. If you don’t, you have tablets which are of non-homogeneous friability, hardness and quite likely dissolution. They may or may not comply with the specifications and they may or may not be part of the sample that is QC tested. Therefore, any company with an eye to quality and consistently meeting requirements, as well as environmental awareness (not wasting resources), follows MTBF. But, in a highly regulated and punitive environment (warning letters, forced recalls for “lack of sterility assurance” – not non-sterile product, import alerts and other consequences) would you want to spoon-feed the information OUT OF CONTEXT, for HQ (headquarters) staff to decide you are out of compliance? Remember in the movies “I plead the fifth” – the right to remain silent if you may incriminate yourself. FDA and the EU have a policy of not asking to review or copy records from internal audits “to encourage frank and open audits.” By the way, this is one of the privileges you can lose if you get into a situation of “for-cause” or “violative” inspection. We are seeing a move away from this policy in general. With scrutiny during on-site inspections of management review meetings, companies are becoming more guarded in the documentation of these minutes. This makes the management reviews LESS effective which is the unintended consequence of the regulatory environment. The metrics program is asking for data relating to management of the quality system and continual improvement. There will be unintended consequences. Take “right first time data” i.e. how many batches have any kind of deviation, incident or non-conformance associated with them. Companies will not incriminate themselves and will find workarounds. Some of the companies participating in the pilot already have those little tricks in place. For example, instead of calling every non-desirable, unwanted or unusual event a failure or a non-conformance and tracking them in the QMS (as one must if the system is to be effective), we see “minor incidents” reported in a batch record as a “NTF” – note to file or something similar. QA reviews the batch record and “decides” (presumably using King Solomon’s special QA wisdom) if the incident meets the threshold for opening a deviation or reportable / “countable” incident in the QMS. Therefore, e.g. a missing signature or date or time, an unfilled field, an illegal crossing out might be an NTF, a machine stoppage, may or may not be an NTF etc. Would you blame a company for taking such an approach if FDA is to compare my MBTF with Merck’s or Novartis or a small start-up company’s. There is a lack of context associated with the data: the how, why, where, when, the process and the complexity of the equipment etc. Similarly, the data is “dry” – we aren’t given a chance to defend / explain it which we do during an on-site inspection.

In conclusion, you are urged to read the proposed program and answer FDA’s questions. Submit comments. Explain to FDA why your company has an effective QMS and why the proposed metrics are going to mess with it. Tell them they can see whatever they want on site – but reviewing it dry and off-site, creates a skewed picture which can work only to your disadvantage. Or worse, they will encourage companies to submit skewed data which is in no-one’s interest. The good companies will get better anyway and the bad ones, sadly, will get worse. Regulators need to work to catch the real baddies (yes – politically incorrect but that actually is their job), help the companies trying to be good, to fix the issues and reward the good companies as they have been promising for years without doing so, with fewer inspections.

Looking forward to seeing you again soon – September no longer very far off is it!

Karen