IPSIndian Journal of Pharmacology
Home  IPS  Feedback Subscribe Top cited articles Login 
Users Online : 2419 
Small font sizeDefault font sizeIncrease font size
Navigate Here
  Search
 
  
Resource Links
 »  Similar in PUBMED
 »  Search Pubmed for
 »  Search in Google Scholar for
 »Related articles
 »  Article in PDF (559 KB)
 »  Citation Manager
 »  Access Statistics
 »  Reader Comments
 »  Email Alert *
 »  Add to My List *
* Registration required (free)

 
In This Article
 »  Abstract
 » Introduction
 »  Materials and Me...
 » Results
 » Discussion
 » Conclusion
 »  References
 »  Article Figures
 »  Article Tables

 Article Access Statistics
    Viewed619    
    Printed25    
    Emailed0    
    PDF Downloaded132    
    Comments [Add]    

Recommend this journal

 


 
 Table of Contents    
RESEARCH ARTICLE
Year : 2019  |  Volume : 51  |  Issue : 1  |  Page : 11-16
 

A study to assess the methodological quality of in vivo animal experiments published in Indian journal of pharmacology: A retrospective, cross-sectional, observational study


Department of Pharmacology, Heritage Institute of Medical Sciences, Varanasi, Uttar Pradesh, India

Date of Submission13-Oct-2018
Date of Acceptance01-Feb-2019
Date of Web Publication19-Mar-2019

Correspondence Address:
Dr. Sandeep Kumar Gupta
Department of Pharmacology, Heritage Institute of Medical Sciences, Varanasi - 221 311, Uttar Pradesh
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/ijp.IJP_536_18

Rights and Permissions

 » Abstract 


BACKGROUND: Good reporting of experimental research is a vital part of research process. Although the reporting guidelines such as Animal research: Reporting in vivo experiments (ARRIVE) require the authors to stick to a standard format, they do not guarantee study reports' validity. For assessing the study reports validity, critical appraisal tools are required.
OBJECTIVE: The evaluation of the methodological quality of in vivo animal studies of Indian Journal of Pharmacology (IJP) published between 2011 and 2017 through critical appraisal was the primary objective. The secondary objective was to evaluate the adherence of selected studies to the ARRIVE guidelines.
MATERIALS AND METHODS: All in vivo animal studies published as full-text articles in IJP from January 2011 to December 2017 and satisfying the inclusion norms were included. A checklist based on the underlying principles of ARRIVE statement was applied to in vivo animal research (AR) published in IJP. For critical appraisal of reports, risk-of-bias domains were also applied on studies from in vivo AR.
RESULTS: One hundred and sixty-one studies satisfied the inclusion criteria. Seventy-three articles (45.34%, 95% confidence interval [CI]: 0.37–0.53) discussed about randomization procedure. Eighteen articles (11.18%, 95% CI: 0.07–0.16) gave details about blinding when assessing results. None of the studies provided details of sample size calculation. Eight articles (4.97%, 95% CI: 0.02–0.09) commented on the study limitations.
CONCLUSION: It was found that adherence to only some criteria of ARRIVE guidelines was subpar. There is a need for optimal reporting of random distribution of animals to experimental groups, concealment of allocation, blinded outcome evaluation, computation of sample size, and attrition of animals for improving the validity of AR.


Keywords: Animal research: reporting in vivo experiments guidelines, critical appraisal, in vivo animal research, methodological quality, risk-of-bias tool


How to cite this article:
Gupta SK. A study to assess the methodological quality of in vivo animal experiments published in Indian journal of pharmacology: A retrospective, cross-sectional, observational study. Indian J Pharmacol 2019;51:11-6

How to cite this URL:
Gupta SK. A study to assess the methodological quality of in vivo animal experiments published in Indian journal of pharmacology: A retrospective, cross-sectional, observational study. Indian J Pharmacol [serial online] 2019 [cited 2019 Apr 25];51:11-6. Available from: http://www.ijp-online.com/text.asp?2019/51/1/11/254587





 » Introduction Top


Replication of preclinical studies' results in humans has been disappointing. There may be two likely explanations behind this. To begin with, animals are intricate living creatures. Second, the methodological nature of animal research (AR) may be deficient, causing ambiguous outcomes.[1] There is considerable corroboration that reporting of experimental research is less than optimal.[2] Moreover, the low replication of ARs can be ascribed to insufficient reporting. Thus, optimal reporting of experimental research is an essential part of any research.[3] In view of these confirmations, there have been demands for improved reporting in AR.[4] The Animal research: Reporting in vivo experiments (ARRIVE) statement in 2010 epitomized a key step to ameliorate the reporting standard of AR.[5],[6] The ARRIVE guidelines recommend addition of the methodological factors that have been found to be inadequately reported in animal experiments. Application of ARRIVE may result in enhanced reporting quality of AR (though there is a report contrary to that).[7]

Although the reporting guidelines require the authors to stick to a standard format, they do not guarantee the study validity.[5] The poor reporting of animal experiments, due to compromised internal and external validity, has been implicated in the divergences between the results of preclinical and clinical studies.[8] Validity of the report can be evaluated by critical appraisal tools.[9] It is imperative to appraise the internal as well as external validity to judge whether outcomes of the study represent the source and target population. Internal as well as external validity can be assessed through critical appraisal.[10]

Indian Journal of Pharmacology (IJP) is a globally recognized and highly respected journal. Majority of studies published in IJP consist of in vivo AR. It was thus sensible to evaluate in vivo animal studies adherence to ARRIVE guidelines and to evaluate the methodological quality of the in vivo animal studies through critical appraisal.[11],[12]

Objective

The evaluation of the methodological quality of in vivo animal studies published in IJP from 2011 to 2017 through critical appraisal was the primary objective. The secondary objective of was to evaluate the adherence of the published in vivo animal studies to the ARRIVE guidelines.


 » Materials and Methods Top


Study design

This was a retrospective, cross-sectional, observational study to evaluate the methodological quality of in vivo animal studies through critical appraisal and to examine their adherence to ARRIVE guideline. The ethical committee sanction was not obligatory because this study completely relies on the information accessible in the public domain and is a quality audit not incorporating sensitive information retrieval.

Data extraction

All in vivo animal studies published as full-text articles in IJP from January 2011 to December 2017 and fulfilling the eligibility criteria were eligible for inclusion. Eligible articles were identified by screening of titles, abstracts, keywords, and methodology.

Eligibility criteria

Inclusion criteria

(1) Animal studies with treatment intervention, (2) only original research articles, and (3) studies published between January 2011 and December 2017.

Exclusion criteria

(1) In vitro experiments, (2) studies on animals that were dead before inception of the study, (3) studies where no intervention was performed, (4) studies with no control group, and (5) review articles, commentaries, communications or correspondences, and short communication.

Assessment of adequacy of reporting and critical appraisal of evidences

The assessment of reporting of the in vivo animal experiments was done by guiding principles on the ARRIVE statement. A checklist was prepared dependent on the ARRIVE statement [Table 1], [Table 2], [Table 3], [Table 4], [Table 5].[6],[13] Some items on the checklists were added based on a study by Muhlhausler et al., 2013.[5] The checklist was applied to in vivo AR published in IJP. For critical appraisal of reports, risk-of-bias domains were also applied on studies from in vivo AR.[10],[14] The in-depth appraisal of AR's methodology, management, and analysis was done to determine its validity.
Table 1: ARRIVE checklist for title (item number 1), abstract (item number 2), introduction (item number 3), objectives (item number 4), and methods (item number 5 and 6)

Click here to view
Table 2: ARRIVE checklist for methods (item number 7 and 8)

Click here to view
Table 3: ARRIVE checklist for methods (item number 9, 10, and 11)

Click here to view
Table 4: ARRIVE checklist for methods (item number 12 and 13), results (item number 14)

Click here to view
Table 5: ARRIVE checklist for results (item number 15, 16, 17), discussion (item number 18, 19 and 20)

Click here to view


Data analysis

An exploratory and descriptive statistical analysis was done. Descriptive statistics was applied to illustrate the reporting of studies, as well as for the key methodological items. The SPSS statistical software package (SPSS for Windows, Version 16.0. Chicago, SPSS Inc.) was utilized for statistical analysis. Criteria's were demarcated as “yes” if they were unambiguously reported or “no” if they were indistinct or not stated at all. The lower and upper limits of the 95% confidence interval (CI) for the proportions was calculated.


 » Results Top


Overall, 1185 articles from IJP between January 2011 and December 2017 were screened for suitability. [Figure 1] explains the retrieval and selection of articles in the present study. Out of the total 1185 articles, the screening of title and abstracts excluded 637 articles. These 637 articles included Drug watch (190); Letters to editor or correspondences (144); Short communications (118); Education forum and review articles (54); Editorials (46); and others (n = 85), i.e., Abstracts (29), Book reviews (18), Case reports (13), Obituaries (11), Errata (7), Announcements (4), Special article (2), and Journal club (1). Now, 548 articles were left and these articles were further evaluated.
Figure 1: Flow diagram of retrieval and selection of articles

Click here to view


One hundred and sixty-nine articles were omitted from 548 articles after full-text review. These 169 articles included randomized control trials (30); cross-sectional studies (25); pharmacokinetic/pharmacodynamics studies (22); drug utilization studies (20); questionnaire studies (19); observational studies (16); nonrandomized clinical trials (10); randomized uncontrolled clinical trials (10); systematic review/meta-analysis (6); case–control/cohort studies (5); drug safety studies (4); and pharmacoeconomic studies (2). Now, 379 preclinical articles were left, and these articles were further evaluated. A total of 218in vitro and other types of preclinical studies were omitted after evaluation of complete text. Only rest of 161 studies satisfied the inclusion prerequisites and thus incorporated in the study [Figure 1].

Out of 161 included articles, 33 articles (20.50%, 95% CI: 0.14–0.27) explained in what way the animal type being utilized can justify the methodical objectives. All the included articles clearly described objectives of the study. One hundred and forty-five articles (90.06%, 95% CI: 0.84–0.93) mentioned about the ethical permissions. Seventy-three articles (45.34%, 95% CI: 0.37–0.53) discussed about randomization procedure. Eighteen articles (11.18%, 95% CI: 0.07–0.16) described any type of blinding [Table 1]. One hundred and fifty-three articles (95.03%, 95% CI: 0.90–0.97) provided specifics of the animals used [Table 2]. None of the studies provided specifics about computation of sample size [Table 3]. All the articles provided details of statistical methods used [Table 4]. Eight articles (4.97%, 95% CI: 0.02–0.09) commented on the study limitations. Fifty-five articles (34.16%, 95% CI: 0.27–0.41) listed all funding sources [Table 5].


 » Discussion Top


In critical appraisal, the most common characteristic contemplated is internal validity.[15] If the outcomes derived from study population are indicative of the source population, then the study is assumed to be valid internally. The major hazard to internal validity is the risk of bias.[10],[14] In in vivo AR, selection bias arises when nonrandom factors impact the distribution of animals to study interventions. Selection bias happens if detected variances between the groups in the study are attributable to the confounder rather than interventions. Randomization is the optimum tool for distribution of confounders across the intervention groups in equal proportion.[10] In this study, 45.34% of articles gave details about randomization. In a study by Kilkenny et al. 2009, 12% of all studies reported randomization of animals to experimental groups.[3] In a review by Hess 2011, 28% studies reported about randomization of animals.[16] In a methodological study of AR by Bara and Joffe 2014, 61% of all studies gave details about randomization.[1]

Performance bias happens when there are logical differences between experimental groups in susceptibility to determinant apart from the study intervention.[10] Allocation concealment can be used to prevent performance bias.[17] Random housing of the animal can also be used to prevent performance bias.[14] In this study, none of the studies reported allocation concealment mechanism. In a study by Bara and Joffe 2014, 6% of the studies reported allocation concealment.[1] Whereas in a study by Hirst et al. 2014, 15% of studies conveyed it.[18]

Detection bias can ensue if there are variances in outcome assessment methods between treatment groups.[10],[17] It can be avoided by blinding of the persons evaluating the outcome.[10] In this study, 11.18% of the study reported blinding. In a study by Bara and Joffe 2014, 40% of all studies stated about blinding.[1] Macleod et al. 2015 reported about blinding in 29.5% of animal studies.[19] In a study by Kilkenny et al. 2009, only 14% of all included study reported about blinding.[3]

Attrition bias is a problem in clinical research but the likelihood for discretionary attrition bias in in vivo AR is practically nil. However, nondiscretionary attrition can unintentionally introduce bias.[10] The commonly used tool to prevent attrition bias is reporting dropouts. In this study, animal number in each experimental group included in the analysis was given by all studies. None of the article in this study gave details about any dropout or explain why they were not included in the analysis. In the report by Hotwani et al. 2017, 91% of the study reported the animal number in each experimental group included in the analysis.[12]

External validity defines whether outcomes of the study represent the target population. O'Connor and Sargeant stated that if the study results are internally valid, then it will represent the target population.[10] There are no guidelines for assessing external validity of AR.[10] This study does not offer any insight into the external validity of the studies analyzed. This study had several limitations. The included studies were diverse with respect to the study outcomes, animals used, and disease types. Some studies may have failed to report some methodological details when in fact these were used. This study only assesses the reporting quality of assessed studies and did not aim to evaluate the quality of conduct.


 » Conclusion Top


Overall, this study demonstrated good adherence of in vivo AR studies published in IJP with respect to the ARRIVE statement with few exceptions. It was found that adherence to only some criteria of ARRIVE guidelines was subpar. There is a necessity for optimal reporting of random distribution of animals to experimental groups, concealment of allocation, blinded outcome evaluation, computation of sample size, and reporting of attrition of animals for improving validity of AR.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
 » References Top

1.
Bara M, Joffe AR. The methodological quality of animal research in critical care: The public face of science. Ann Intensive Care 2014;4:26.  Back to cited text no. 1
    
2.
Steward O, Balice-Gordon R. Rigor or mortis: Best practices for preclinical research in neuroscience. Neuron 2014;84:572-81.  Back to cited text no. 2
    
3.
Kilkenny C, Parsons N, Kadyszewski E, Festing MF, Cuthill IC, Fry D, et al. Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS One 2009;4:e7824.  Back to cited text no. 3
    
4.
Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW, et al. Acall for transparent reporting to optimize the predictive value of preclinical research. Nature 2012;490:187-91.  Back to cited text no. 4
    
5.
Muhlhausler BS, Bloomfield FH, Gillman MW. Whole animal experiments should be more like human randomized controlled trials. PLoS Biol 2013;11:e1001481.  Back to cited text no. 5
    
6.
Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biol 2010;8:e1000412.  Back to cited text no. 6
    
7.
Leung V, Rousseau-Blass F, Beauchamp G, Pang DS. ARRIVE has not ARRIVEd: Support for the ARRIVE (Animal research: Reporting of in vivo experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia. PLoS One 2018;13:e0197882.  Back to cited text no. 7
    
8.
Sena ES, Currie GL, McCann SK, Macleod MR, Howells DW. Systematic reviews and meta-analysis of preclinical studies: Why perform them and how to appraise them critically. J Cereb Blood Flow Metab 2014;34:737-42.  Back to cited text no. 8
    
9.
van der Worp HB, Macleod MR. Preclinical studies of human disease: Time to take methodological quality seriously. J Mol Cell Cardiol 2011;51:449-50.  Back to cited text no. 9
    
10.
O'Connor AM, Sargeant JM. Critical appraisal of studies using laboratory animal models. ILAR J 2014;55:405-17.  Back to cited text no. 10
    
11.
Aiman U, Rahman SZ. Standard of reporting animal-based experimental research in Indian journal of pharmacology. Indian J Pharmacol 2015;47:514-7.  Back to cited text no. 11
[PUBMED]  [Full text]  
12.
Hotwani JH, Deshmukh KR, Mehta MN. Adherence to ARRIVE guidelines in animal research articles published in a National and International Pharmacology Journal. Int J Basic Clin Pharmacol 2017;6:1696-700.  Back to cited text no. 12
    
13.
Liu Y, Zhao X, Mai Y, Li X, Wang J, Chen L, et al. Adherence to ARRIVE guidelines in Chinese journal reports on neoplasms in animals. PLoS One 2016;11:e0154657.  Back to cited text no. 13
    
14.
Hooijmans CR, Rovers MM, de Vries RB, Leenaars M, Ritskes-Hoitinga M, Langendam MW, et al. SYRCLE's risk of bias tool for animal studies. BMC Med Res Methodol 2014;14:43.  Back to cited text no. 14
    
15.
Bailoo JD, Reichlin TS, Würbel H. Refinement of experimental design and conduct in laboratory animal research. ILAR J 2014;55:383-91.  Back to cited text no. 15
    
16.
Hess KR. Statistical design considerations in animal studies published recently in cancer research. Cancer Res 2011;71:625.  Back to cited text no. 16
    
17.
Higgins JP, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, et al. The cochrane collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928.  Back to cited text no. 17
    
18.
Hirst JA, Howick J, Aronson JK, Roberts N, Perera R, Koshiaris C, et al. The need for randomization in animal trials: An overview of systematic reviews. PLoS One 2014;9:e98856.  Back to cited text no. 18
    
19.
Macleod MR, Lawson McLean A, Kyriakopoulou A, Serghiou S, de Wilde A, Sherratt N, et al. Risk of bias in reports of in vivo research: A Focus for improvement. PLoS Biol 2015;13:e1002273.  Back to cited text no. 19
    


    Figures

  [Figure 1]
 
 
    Tables

  [Table 1], [Table 2], [Table 3], [Table 4], [Table 5]



 

Top
Print this article  Email this article
 

    

Site Map | Home | Contact Us | Feedback | Copyright and Disclaimer
Online since 20th July '04
Published by Wolters Kluwer - Medknow