Improving Transparency in Dental Research by Making the Raw Data Available

October 11, 2011

Fraud and misconduct in medical science research are an unfortunate reality. We have seen several cases in the last few years, giving rise to the perception that such misbehaviour is increasing.1 In one extreme example, Dr. Scott Reuben, a once-prominent researcher in pain management, fabricated data on painkillers and reported these results in more than 20 journal articles.2 Many people were placed at risk of potentially serious side effects, including increased cardiovascular complications.3 Furthermore, Dr. Reuben’s fraudulent findings compromised the results of any secondary research conducted with his data, such as systematic reviews and meta-analyses.4 Clinical decision-making based on the results of these fraudulent studies may very well be flawed.

It would therefore seem logical to make every possible effort to reduce the chances of false or unreliable data being published in the scientific literature. The challenge is that identifying such fraud is not an easy task. The peer-review process can assess the research methods used by an author, but it can’t determine whether the procedures themselves were actually conducted. Yet manuscripts which are peer-reviewed and get published in the scientific literature are generally assumed to be high-quality articles.

Publishing the original data (known as the “raw data”) used by authors in preparing a manuscript could be an effective means of deterring or reducing scientific misconduct. The raw data would allow other research groups and interested readers to reproduce or verify the analyses used in an article.5 Registering the trial protocol in a public clinical trial registry is another way to monitor deviations between what is reported in the trial methodology and the final published paper.

To determine how frequently these 2 measures are used in current editorial policies, I examined whether dental publications suggest or require the publication of raw data and the registration of the clinical study protocol in the public domain. The results of my online review of such policies may generate debate among all parties involved in dental clinical research, including journal editors, researchers, patients and JCDA readers.

Not All Editorial Policies Are Created Equal

I reviewed the guidelines to authors of the 64 dental journals that had an impact factor in the 2009 Journal Citation Reports, published by Thomson Scientific, to determine whether the journals had a policy suggesting or requiring publication of the original data from a randomized controlled trial (RCT). I also examined if the journals suggested or required pre-registration of RCT study protocols in a clinical trial registry. Similar editorial policy information was obtained from the 10 highest ranked general and internal medical journals, along with a random sample of 40 other medical journals, for comparison purposes. I chose to assess these lower ranked medical journals to allow for a more equal comparison between dental and medical publications with relatively similar impact factors.

The publication policies of the 10 highest ranked medical journals are more explicit than the dental journals and lower ranked medical journals. No dental journal suggests or requires the publication of the raw data with manuscript submission (Table 1). However, some dental journals allow the submission of supplementary material, such as data sets or additional figures or tables. Almost one-third of the dental journals (29%) require or suggest the pre-registration of the trial protocol in a public entity.

Although some of the high-ranked medical journals suggest that the raw data be published, most do not make it a requirement. (Table 2). However, most high-ranked medical journals require the pre-registration of the trial protocol in a public entity. None of the lower ranked medical journals suggested or required the publication of the raw data with manuscript submission (Table 3). Similar to the dental journals, less than one-third of these journals (24%) require the pre-registration of the trial protocol in a public entity.

 

Why Transparency in the Publication Process Is Important

The results of my review demonstrate that the editorial policies of dental journals could be improved. With no policy even suggesting or requiring the publication of the original data, monitoring the reliability of the data by editors and reviewers is a daunting challenge. A recent paper6 suggests measures that could be taken to reduce the chance of scientific misconduct in dentistry. Although this article recommends other interesting measures to improve transparency in the editorial process, such as software to control plagiarism and detect image fakery, the publication of raw data and the registration of the study protocol are not discussed.

Sharing the original data can be beneficial in many ways, most notably by enabling other researchers to replicate the study findings. Replication is one of the cornerstones of experimental studies,7 and the repetition of a study under similar conditions should increase the reliability of the results. Publishing raw data also opens up pivotal parts of the research, like detailed statistical assessments and complete results of the primary research.5,8,9 Such elements allow researchers to thoroughly interpret the findings. Publishing the research protocol could reduce cases of selective outcome reporting (where researchers report only some of the analyzed outcomes) and generate less biased meta-analytic estimates.10 In fact, there is some evidence that trial outcomes can sometimes be inconsistent with protocols.11 This would justify having journals adopt a policy of demanding the research protocol of a clinical trial with the final manuscript.

If other journals adopted some of the editorial procedures of the high-ranked medical journals, transparency in research would improve. For example, for industry-sponsored trials, these medical journals ask that the complete data set (protocol, raw data and paper) be assessed by an independent statistician to check the reliability of the data. This is requested because industry-funded trials are more likely to be associated with statistically significant pro-industry findings.12,13Authors could also be asked to report details of any possible previous or duplicate publication in a covering letter, which would reduce the chance of publishing duplicate data and would optimize the secondary research process in systematic reviews and meta-analyses.

This article provides suggestions for changes to editorial policies that would improve transparency in the sharing of data in dental publications. Having authors publish the study protocol in the public domain would be an improvement of the editorial process. But the dental community should now decide if it is time to go further and start requesting that authors share the raw data from clinical trials. I hope this article will stimulate constructive discussion of this topic.

THE AUTHOR

 

Dr. Faggion is a research fellow in the department of prosthodontics, University of Heidelberg Dental School, Heidelberg, Germany.

Acknowledgements: The author was partially funded by a postdoctoral fellowship from the medical faculty of the University of Heidelberg.

Correspondence to: Dr. Clovis Mariano Faggion Jr., Department of prosthodontics, Dental School, University of Heidelberg, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany. Email: clovisfaggion@yahoo.com

The views expressed are those of the author and do not necessarily reflect the opinions or official policies of the Canadian Dental Association.

This article has been peer reviewed.

References

  1. Steen RG. Retractions in the scientific literature: is the incidence of research fraud increasing? J Med Ethics. 2011;37(4):249-53. Epub 2010 Dec 24.
  2. Shafer SL. Tattered threads. Anesth Analg. 2009;108(5):1361-3.
  3. Faunce T, Townsend R, McEwan A. The Vioxx pharmaceutical scandal: Peterson v Merke Sharpe & Dohme (Aust) Pty Ltd (2010) 184 FCR 1. J Law Med. 2010;18(1):38-49.
  4. White PF, Kehlet H, Liu S. Perioperative analgesia: what do we still know? Anesth Analg. 2009;108(5):1364-7.
  5. Vickers AJ. Whose data set is it anyway? Sharing raw data from randomized trials. Trials. 2006;7:15.
  6. Luther F. Scientific misconduct: tip of an iceberg or the elephant in the room? J Dent Res. 2010;89(12):1364-7. Epub 2010 Oct 12.
  7. Hackam DG, Redelmeier DA. Translation of research evidence from animals to humans. JAMA. 2006;296(14):1731–2.
  8. Altman DG, Cates C. Authors should make their data available. BMJ. 2001;323(7320):1069-70.
  9. Hutchon DJ. Publishing raw data and real time statistical analysis on e-journals. BMJ. 2001;322(7285):530.
  10. Higgins JPT, Altman DG, Sterne JAC (editors). Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.
  11. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291(20):2457-65
  12. Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? JAMA 2003;290(7):921-8.
  13. Bhandari M, Busse JW, Jackowski D, Montori VM, Schünemann H, Sprague S et al. Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials. CMAJ. 2004;170(4):477-80.

 

 

Back to article

 

 

Table 1 Editorial policy of dental journals regarding data sharing, as explicitly stated in the guidelines to authors section

Journal 2009 impact factor Management of raw dataa Research protocol in public domainb
Journal of Clinical Periodontology 3.549 No Suggestion
Journal of Dental Research 3.458 No Requirement
Oral Oncology 3.123 No Requirement
Periodontology 2000c 3.027
Journal of Endodontics 2.953 No No
Clinical Oral Implants Research 2.920 No Suggestion
Dental Materials 2.882 No No
Caries Research 2.462 No No
Clinical Implant Dentistry and Related Research 2.452 No No
Community Dentistry and Oral Epidemiology 2.418 No Suggestion
Oral Microbiology and Immunology 2.336 No No
Clinical Oral Investigations 2.233 No No
International Endodontic Journal 2.223 No Suggestion
Journal of Periodontology 2.192 No No
Journal of Oral Pathology and Medicine 2.144 No Suggestion
Journal of Dentistry 2.000 No Requirement
The International Journal of Oral and Maxillofacial Implants 1.978 No No
Journal of Periodontal Research 1.966 No Suggestion
European Journal of Oral Sciences 1.956 No Suggestion
Oral Diseases 1.922 No Requirement
The Journal of the American Dental Association 1.726 No No
The International Journal of Periodontics and Restorative Dentistry 1.702 No No
Operative Dentistry 1.683 No No
Archives of Oral Biology 1.649 No No
The Journal of Adhesive Dentistry 1.638 No No
Orthodontics and Craniofacial Research 1.607 No Suggestion
Journal of Oral and Maxillofacial Surgery 1.580 No No
Implant Dentistry 1.505 No No
Oral Surgery Oral Medicine Oral Pathology Oral Radiology and Endodontology 1.499 No No
Journal of Oral Rehabilitation 1.483 No No
International Journal of Oral and Maxillofacial Surgery 1.444 No No
Acta Odontologica Scandinavica 1.412 No No
American Journal of Orthodontics and Dentofacial Orthopedics 1.327 No No
British Journal of Oral and Maxillofacial Surgery 1.327 No Requirement
Dental Traumatology 1.316 No Requirement
American Journal of Dentistry 1.314 No No
Journal of Orofacial Pain 1.263 No No
Journal of Cranio-Maxillo-Facial Surgery 1.252 No Requirement
Dento Maxillo Facial Radiology 1.229 No No
The International Journal of Prosthodontics 1.227 No No
Australian Dental Journal 1.220 No No
The Journal of Prosthetic Dentistry 1.215 No No
International Journal of Paediatric Dentistry 1.141 No Suggestion
British Dental Journal 1.089 No No
Journal of Dental Education 1.087 No No
European Journal of Dental Education 1.024 No No
Gerodontology 1.014 No No
European Journal of Orthodontics 0.975 No No
Community Dental Health 0.969 No No
Journal of Public Health Dentistry 0.961 No Suggestion/ requirementd
Journal of the Canadian Dental Association 0.959 No No
Angle Orthodontist 0.937 No No
Dental Materials Journal 0.929 No No
Swedish Dental Journal 0.927 No No
Journal of Orofacial Orthopedics 0.890 No No
Cleft Palate-Craniofacial Journal 0.884 No No
Journal of Esthetic and Restorative Dentistry 0.797 No No
Odontology 0.650 No No
Quintessence International 0.635 No No
International Dental Journal 0.632 No No
Journal of Cranio-Mandibular Practice 0.412 No No
Journal of Applied Oral Science 0.386 No Requirement
Revue de Stomatologie et de Chirurgie Maxillo-Faciale 0.349 No No
Journal of Dental Sciences 0.107 No No

aManagement of raw data: Does the journal suggest or require submission of the raw data with the manuscript?
bResearch protocol in public domain: Does the journal suggest or require pre-registration of the trial in the public domain? It was considered a suggestion when the guideline used the word “encourages,” but did not explicitly demand the registration.
cThe journal only publishes papers by invitation (reviews).
dMandatory for industry-sponsored trials.

 

 

Back to article

 

 

Table 2  Editorial policy of the 10 highest ranked general and internal medicine journals regarding data sharing, as explicitly stated in the guidelines to authors section

Journal 2009 impact factor Management of raw dataa Research protocol in public domainb
New England Journal of Medicine 47.050 No Requirement
Lancetc 30.758 Suggestion
Journal of American Medical Associationd 28.899 Requirement
Annals of Internal Medicine 16.225 Suggestion Requirement
British Medical Journal 13.660 Suggestion Requirement
PLOS Medicine 13.050 Suggestion Requirement
Annual Review of Medicinee 9.940
Archives of Internal Medicine 9.813 No Requirement
Canadian Medical Association Journal 7.271 No Requirement
Journal of Internal Medicine 5.942 No No

a Management of raw data: Does the journal suggest or require submission of the raw data with the manuscript?
b Research protocol in public domain: Does the journal suggest or require pre-registration of the trial in the public domain? It was considered a suggestion when the guideline used the word “encourages,” but did not explicitly demand the registration.
c Authors may be asked to provide the raw data for research papers when they are under review and up to 10 years after publication in the journal.
d The journal will publish raw data as a supplementary online file if such information is important for readers wishing to interpret a study or for others to replicate the study.
e The journal publishes only reviews.

 

 

Back to article

 

 

Table 3 Editorial policy of 40 randomly selected general and internal medicine journals regarding data sharing, as explicitly stated in the guidelines to authors section

Journal 2009 impact factor Management of raw dataa Research protocol in public domainb
The Medical Journal of Australia 2.894 No Requirement
European Journal of Clinical Investigationc 2.643 No Requirement
QJM: An International Journal of Medicine 2.627 No No
British Journal of General Practice 2.442 No No
Pain Medicine 2.393 No No
Journal of Urban Health 2.205 No No
Scandinavian Journal of Primary Health Care 2.205 No No
Cleveland Clinic Journal of Medicined 2.149
Translational Research 2.062 No Requirement
Internal Medicine Journal 1.786 No Requirement
Swiss Medical Weekly 1.681 No Requirement
Disease-a-Monthe 1.571
Indian Journal of Medical Research 1.516 No No
Journal of Hospital Medicine 1.496 No Requirement
The Journal of Family Practice 1.426 No No
Panminerva Medica 1.426 No No
Croatian Medical Journal 1.373 No Requirement
The Tohoku Journal of Experimental Medicine 1.347 No No
Family Medicine 1.331 No No
Medicina Clínica 1.231 No No
Archives of Medical Sciencef 1.012 No
Wiener klinische Wochenschrift 0.955 No No
Southern Medical Journal 0.924 No No
Archives Of Iranian Medicine 0.874 No Requirement
Primary Care 0.809 No No
São Paulo Medical Journal/Evidence for Health Care 0.746 No Requirement
Upsala Journal of Medical Sciences 0.733 No No
Journal of the Formosan Medical Association 0.661 No No
Medical Problems of Performing Artists 0.617 No No
Revista da Associacao Médica Brasileira 0.589 No No
Saudi Medical Journal 0.510 No No
Scottish Medical Journal 0.507 No No
Revista de Investigación Clínica 0.505 No No
Revista médica de Chile 0.487 No No
La Presse Médicale 0.416 No No
West Indian Medical Journal 0.338 No No
International Journal of Osteopathic Medicine 0.206 No No
Acta Clinica Croatica 0.188 No No
Medicinski Glasnik 0.136 No No
International Medical Journal 0.115 No No

a Management of raw data: Does the journal suggest or require submission of the raw data with the manuscript?
b Research protocol in public domain: Does the journal suggest or require pre-registration of the trial in the public domain? It was considered a suggestion when the guideline used the word “encourages,” but did not explicitly demand the registration.
c The journal suggests submitting the study protocol together with the paper.
d The journal does not publish clinical trials.
e The journal publishes reviews only.
f Authors may be asked to provide the raw data.

Add new comment