By Dr. Patrick Howson
In previous posts, we discussed how a lack of repeatability and robustness in preclinical studies is a contributing factor to the declining success rates of novel therapeutics. and how quality systems such as Enhancing Quality in Preclinical Data (EQIPD) can help minimize biases in data generation. We also took a deeper look into some aspects of preclinical efficacy studies that EQIPD addresses, such as research integrity, experimental design, and data validity.
In this post, we explore other critical areas covered by EQIPD, in particular data handling, data analysis, and transparent reporting. We also highlight how Atuka has addressed these areas and has continued to align our quality systems with EQIPD.
Data handling
Proper data handling is vital in ensuring the integrity, reproducibility, and reliability of biomedical research. The EQIPD quality system relies on proper data management to produce trustworthy research outcomes, which is particularly critical in preclinical studies that inform later, more expensive studies such as toxicology and clinical studies. EQIPD is designed to be fully compatible with the ALCOA+ principles for data integrity that were initially developed by the US Food and Drug Administration (FDA). ALCOA is an acronym for Attributable, Legible, Contemporaneous, Original, and Accurate, with the ‘+’ adding four more attributes; Complete, Consistent, Enduring, and Available. These principles are essential for ensuring data is trustworthy.
- Attributable: Records must show who did what and when.
- Legible: Data must be permanent and understandable throughout its retention period.
- Contemporaneous: Records must be made at the time the activity happens, not later.
- Original: The first capture of the data or a certified true copy must be kept.
- Accurate: Information must be a true reflection of what occurred, free from error or falsification.
- Complete: All data related to a process or activity must be included; selective recording is not acceptable.
- Consistent: Records should follow a logical sequence and be free of unexplained gaps.
- Enduring: Data must be stored in a permanent, secure medium for the required retention period.
- Available: Information must be easily accessible when needed by inspectors or reviewers.
Abiding by these principles helps prevent data integrity issues like inaccuracy, incompleteness, and inconsistencies. At Atuka, all new employees who work on studies are provided with training to ensure that they understand the principles of ALCOA+, and why they are important.
Data Analysis
Data analysis is important because it transforms raw data into valuable insights that allows organizations to make evidence-based decisions. EQIPD defines a systemic approach to data analysis that helps ensure the reliability of data. This approach:
- Prevents “fishing expeditions”: Data analysis, based on a well-defined hypothesis established during the study planning phase, minimizes speculative data analysis to find positive results. Post-hoc analysis of data is still valuable, but this approach prevents post hoc analysis being presented as formal, knowledge gathering research and thus helps prevent P-hacking (data are analysed in many ways to achieve a desired result) and HARKing (Hypothesizing After the Results are Known—a hypothesis based on the interpretation of the data is presented as having existed before the data were obtained).
- Mitigates bias: The EQIPD Quality System includes core requirements to minimize bias during analysis, such as blinding researchers to the treatment groups. This prevents conscious or unconscious bias from skewing the results.
- Enabling meta-analysis: By standardizing data collection and analysis methods, EQIPD makes it easier for researchers to combine and compare data from multiple studies through meta-analysis. This provides more robust conclusions than single studies can offer.
At Atuka, our protocols define inclusion and exclusion criteria, contain a statistical analysis section, and pre-specifies what endpoints will be assessed. When post-hoc analyses is performed, or when new endpoints are added to a study, these are clearly identified in our reports and verifiable by cross referencing them to our dated and signed protocols.
Transparent reporting
Inconsistent and incomplete reporting of methods contributes to the low reproducibility rates seen in preclinical research. Transparent reporting ensures that other researchers can replicate experiments, and if findings cannot be replicated, it helps determine why. The EQIPD quality system addresses this through requirements such as:
- Traceability: Data must be documented and stored in a way that allows reported outcomes to be traced back to the original experimental data.
- Documentation of methods: Study protocols and experimental methods must be clear, unambiguous, and accessible.
- Recording of all studies: Researchers must disclose all repetitions of a study, regardless of the outcome, to prevent publication bias.
At Atuka, we have always incorporated transparency and traceability into our studies and thus are EQIPD compliant. However, we have incorporated additional staff training to ensure our team fully understands why transparent reporting is an important factor in generating robust and reliable data.
Summary
Atuka has now fully aligned its quality systems with EQIPD and we have now begun the formal accreditation process that will ultimately result in Atuka being the first Canadian company to receive EQIPD accreditation. However, Atuka’s focus on quality will not stop with EQIPD accreditation. Although EQIPD accreditation will be an important milestone, we view this as part of a journey of continuous improvement that will ensure that Atuka continues to provide our collaborators with reliable data upon which key drug development decisions can be made.