Conducting Data Integrity Audits: A Quick Guide

Data integrity issues continue to impact drug and device companies in increasingly complex ways, as demonstrated through a number of recent warnings from regulators.

The sophistication of modern data environments has made data integrity auditing as critical as it is difficult. These assessments demand a lot from auditors, and the breadth of systems in need of assessment can present daunting risks when auditors fail to assess all the areas in need of review, leaving their organization vulnerable to serious compliance problems.

Whether you're establishing a data integrity auditing program for the first time or simply need to improve your current audit strategy, we've summarized some key concepts to consider each step of the way.

1. Set Audit Goals and Prepare Accordingly

At its heart, the goal of any effective data integrity audit is to identify any existing data or metadata going unnoticed. This included deleted data, reprocessed data, data being misused as test samples, or data that isn't being reviewed during final batch disposition.

If you're struggling to know where to begin, start by defining the opportunities and incentives for breaches and misuse of data. Once these problem-prone areas are highlighted, target your audit activities to prioritize your assessment around high-risk areas, making the smartest use of your time and resources.

To identify the key opportunities that lend themselves to compliance problems, review the system controls and requirements currently in place such as access control, management control, file structure, and data management across the entire data lifecycle.

Next, look for possible incentives that may motivate data integrity breaches. These come in many forms, not all of which may be apparent or easy to define.

Company cultures that reward and encourage speedy results, for instance, can create perverse incentives for those handling data, both from management and fellow colleagues. Since something as broad and amorphous as a company culture can be difficult to point to in practical terms, narrow your focus to a particular section of your company protocol to hone in on exactly how a motivation for data misuse—intentional or not—could reasonably emerge from it.

Grab our free white paper for a comprehensive guide to data integrity.

Ensuring Enterprise-Wide Data Integrity in FDA-Regulated Industries

Similarly difficult to identify are incentive structures rooted in the methods and processes that may be deeply ingrained into your organization. These can include issues related to development and method transfer. These issues continue to appear in FDA 483s and Warning Letters.

When hunting for high-risk targets, start your search where most issues are found: OOS and stability systems.

Here are some related tips for guiding your audit prep:

• Identify any products with high OOS rates, especially those that undergo Phase II investigations and products that don't conclude in definitive lab error.
• Pay close attention to any stability specifications that are highly restrictive and/or may prove to be a challenge for the established expiry period.
• Once you've highlighted high-risk opportunities, select a broad data set over a defined timeframe and trace back all data generated from the beginning to final reporting. If a data integrity problem does exist, there's a very good chance this process will likely reveal an indicator of something to investigate further.

2. Conduct the Audit

On audit day, take care to create a comfortable environment for those being assessed. Clearly communicate your goals and offer a summary of what the audit will entail to set expectations and establish friendly, professional rapport from the outset.

After this brief introduction, use the following guide as a template from which to build out your own data integrity audit activities:

• Identify the area's authority structure and note front line personnel. Confer with area or department management to understand the staffing hierarchy and identify the front line employees your audit should focus on. While management should absolutely be included in the audit, keep in mind they may be somewhat removed from the specific process being reviewed. Persistent attempts by managers to speak on behalf of their staff should be regarded as potential red flags worthy of further investigation.
• Take a risk-based approach to prioritizing audit activities. Given a limited timeframe, auditors need a way to gather a lot of data across multiple systems in relatively little time. One of the more practical approaches to prioritizing your time is to focus on the processes that have the greatest impact on products and results. Reviewing process flowcharts, compiling and listing out testing methods, and reviewing SOPs are all effective ways to do this efficiently. Be sure to build flexibility into your audit plan as well. Problems—both major or minor—may come to your attention during your very first walk-though. Make sure you have the ability to pursue these problems if and when they arise.

• Don't rely on summary reports. While useful for getting a general sense of how a certain operation is performed, never rely on summary reports alone when determining if a system is or isn't acceptable and compliant. The most important components of GMP can only be assessed by examining the raw data itself. Use summary reports as starting points to trace results back to their data sources, such as a specific sample set.

• Be tactful when handling discrepancies. If a major data discrepancy is revealed, avoid conveying criticism or judgment onto staff. Instead, invest your energy in investigating and gathering as much information as possible. Keep the goal fixed on finding solutions as well as illuminating the problem and possible root causes. Keep in mind that the outcome of any investigation into an issue is directly related to the quality of evidence collected. Photos, document copies, electronic media copies, and verbal evidence are all essential to capture immediately after discovering an issue.

3. Wrap Up with Documentation

During the final phase of the audit, document all problematic and/or questionable conditions discovered throughout data integrity controls, oversight, and results.

Never definitively conclude that data integrity breaches don't exist just because nothing was discovered. If specific conditions suggest opportunities for problems exist due to inadequate controls or inappropriate incentives, they should be documented and followed up on as well.

If any "orphan" data is discovered, note these findings and indicate the type and conditions that allowed it to exist outside of where it should have been.

4. Remediate Issues Through CAPA

A few immediate actions may be necessary after completing audit activities:

• If orphan data is discovered, it must be addressed as a requirement for evaluating OOS results.
• You may need to report through the Field Alert Reporting system, Biological Process Deviation Report, or recall assessment based on your findings.
• If the conditions you found could allow for other orphan data, a subsequent assessment of those data sets may be necessary.

Following any immediate post-assessment activities, remediation should be conducted through corrective and preventive action (CAPA) planning. CAPA is particularly useful in situations requiring more substantial long-term technological solutions by enabling you to bridge the gap through interim controls. These might include routine reviews of at-risk data sets, broader oversight of systems that are shown to have audit trail problems, and manual review of reprocessing and integration when parameters aren't under proper control.

After carefully crafting CAPAs, these plans should be verified to ensure their contents are effective following implementation.

In addition to ongoing CAPA remediation, routine review of practices performed by quality personnel on the manufacturing floor can be a simple, yet highly effective measure for detecting and resolving data-related issues before they develop into something far more serious. A quality expert with knowledge of the operations and practices being reviewed is best suited to perform these reviews; however a designated supervisor or foreman can be trained for oversight as well.

With a "second round" of review to double-check data, issues that may have evaded the verifier can be identified while the producer is still on-site. When an error is found, the designated quality reviewer can initiate questioning and either work to solve the issue on-site or immediately escalate the problem to higher authority if technical or procedural problems are discovered.

Going Further With the Help of a Third Party Expert

Third party data integrity, validation, and quality experts can perform comprehensive computer systems and data assessments to ensure your system requirements are fully met and adequately documented. In addition to eliminating the risks born out of internal bias, an experienced data integrity professional is uniquely equipped to implement solve to problems they've addressed many times before.

Learn more about how we help life science organizations with data integrity assessments and remediation here.

“When establishing management controls, executives should seek the expertise of outside consultants who can provide data integrity expertise along with valuable external perspectives on the company’s dynamics— something that can be difficult, if not impossible, to see from the inside."
Chinmoy Roy, Data Integrity, CSV and Process Automation Professional

Data governance and management practices are evaluated using risk-based validation strategies to protect the integrity of your data and strengthen your quality system in the process.

Compliance gaps identified during the assessment are addressed through comprehensive remediation, which we've outlined in our accompanying post here as well as the white paper offered below.

Grab our free white paper, Ensuring Enterprise-Wide Data Integrity in FDA-Regulated Industries, and learn everything you need to know to protect your data in an increasingly complex regulatory environment.