By Dr. Patrick Howson
In our previous post we discussed how a lack of repeatability and robustness in preclinical studies is a contributing factor to the declining success rates of novel therapeutics. We also highlighted the ways in which having quality systems in place that minimize bias and ensure a high degree of standardization across procedures can potentially improve success rates. We looked at one quality system, Enhancing Quality in Preclinical Data (EQIPD), that has been designed for just this purpose.
In this post, we take a deeper dive into some of the key, critical components of EQIPD—such as research integrity, experimental design, validity—and look at how they form part of the training on EQIPD that staff receive at Atuka, which has been developed from resources on the EQIPD E-Learning website. This training is part of our ongoing efforts to align Atuka’s quality system with EQIPD.
Research integrity
Research integrity means absolute honesty, transparency, and accountability in the conduct and reporting of research. Traditionally, research misconduct is defined as fabrication, falsification, or plagiarism in proposing, performing, reviewing, or reporting research results.
- Fabrication is making up results and recording them as if they were real.
- Falsification is manipulating research materials, equipment or processes or changing, omitting or suppressing data or results without justification.
- Plagiarism is using other people’s work and ideas without giving proper credit to the original source, thus violating the rights of the original author(s) to their intellectual outputs
Over the years, there have been many examples of research misconduct, such as the alleged link between the MMR vaccine and autism, some stem cell research, and research on Simufilam. In addition to direct violations, there are several other types of unacceptable practices such as withholding or misrepresenting research findings, encouraging others to violate research integrity, self-plagiarism, and hampering the work of other researchers. Ultimately, researchers rely on the trustworthiness of other researchers to make scientific progress and, by providing a flexible framework that addresses the root causes of irreproducibility and questionable research practices, EQIPD helps maintain research integrity and demonstrates this integrity to others.
Experimental design
Unlike other commonly used quality systems such as GLP, EQIPD has been designed to boost innovation. As such, it is a flexible framework that allows for both exploratory research, where higher levels of uncertainty are tolerated, and knowledge-generating research where maximal rigour is required. What EQIPD does require is that investigators must assert in advance whether the study informs a formal knowledge claim. The contrast between exploratory and knowledge-generating research in terms of EQIPD requirements is shown below.
| Exploratory research | Formal knowledge-generating research | |
|---|---|---|
| Type of research | hypothesis generating research providing support that an emerging hypothesis is valid initial screening of compounds | experimental studies to scrutinize preclinical findings through replication of results research that enables decisions associated with significant resource and time costs labour-, resource- and/or time-intensive studies that cannot be easily repeated |
| Study protocol | should be defined and documented before starting the experiments | must be defined and documented before starting the experiments |
| Study hypothesis | advised to define | must be pre-specified |
| Blinding | advised to implement | should be implemented, exceptions must be justified and documented |
| Randomisation | advised to implement | should be implemented, exceptions must be justified and documented |
| Sample size and power analysis | advised to define and document before starting the experiments | must be defined and documented before starting the experiments (e.g. included in the study protocol) |
| Data analysis | advised to define and document before starting the experiments | must be defined and documented before starting the experiments (e.g. as a formal statistical analysis plan and/or included in the study protocol) |
| Inclusion and exclusion criteria | advised to define and document before starting the experiments | must be defined and documented before starting the experiments (e.g. included in the study protocol) |
| Deviations from protocol | advised to document | must be documented |
| Preregistration/ dated protocols | no | should be implemented |
At Atuka, much of our research is knowledge generating and adheres to the EQIPD recommendations. However, we also occasionally perform exploratory research, and on these occasions, we ensure that our collaborators are fully informed of the strengths and limitations of this type of research, to minimise the risk of the data being misrepresented.
Validity
Validity refers to how correct the results of an experiment are, and the confidence that the observed effects are caused by the manipulated variable. EQIPD enhances validity by implementing measures that reduce bias and confounding factors. The two most important tools to reduce bias are:
- blinding, e.g., ensuring that researchers do not know what animals are receiving which treatments, and
- randomization, e.g., ensuring that animals are randomly assigned to treatment groups.
At Atuka, we have measures in place to ensure that these processes are in place for all knowledge-generating studies. On occasions where full blinding is not possible, for example if the treatment causes a phenotypic change which would be hard to mask, then blinding is still maintained as far as possible, for example for post-mortem endpoints. In these cases, the impact on study validity is discussed with our collaborators before the study is initiated and additional measures to maintain study validity may be implemented.
Summary
Atuka continues to align its quality systems with EQIPD. In our next post on quality at Atuka, we will focus on how Atuka handles, analyses, and reports data within the EQIPD framework and explain why this is a critical process in generating robust, transparent, and reliable data upon which important decisions can be taken.