Improving Transparency in Dental Research by Making the Raw Data Available

Fraud and misconduct in medical science research are an unfortunate reality. We have seen several cases in the last few years, giving rise to the perception that such misbehaviour is increasing.¹ In one extreme example, Dr. Scott Reuben, a once-prominent researcher in pain management, fabricated data on painkillers and reported these results in more than 20 journal articles.² Many people were placed at risk of potentially serious side effects, including increased cardiovascular complications.³ Furthermore, Dr. Reuben’s fraudulent findings compromised the results of any secondary research conducted with his data, such as systematic reviews and meta-analyses.⁴ Clinical decision-making based on the results of these fraudulent studies may very well be flawed.

It would therefore seem logical to make every possible effort to reduce the chances of false or unreliable data being published in the scientific literature. The challenge is that identifying such fraud is not an easy task. The peer-review process can assess the research methods used by an author, but it can’t determine whether the procedures themselves were actually conducted. Yet manuscripts which are peer-reviewed and get published in the scientific literature are generally assumed to be high-quality articles.

Publishing the original data (known as the “raw data”) used by authors in preparing a manuscript could be an effective means of deterring or reducing scientific misconduct. The raw data would allow other research groups and interested readers to reproduce or verify the analyses used in an article.⁵ Registering the trial protocol in a public clinical trial registry is another way to monitor deviations between what is reported in the trial methodology and the final published paper.

To determine how frequently these 2 measures are used in current editorial policies, I examined whether dental publications suggest or require the publication of raw data and the registration of the clinical study protocol in the public domain. The results of my online review of such policies may generate debate among all parties involved in dental clinical research, including journal editors, researchers, patients and JCDA readers.

Not All Editorial Policies Are Created Equal

I reviewed the guidelines to authors of the 64 dental journals that had an impact factor in the 2009 Journal Citation Reports, published by Thomson Scientific, to determine whether the journals had a policy suggesting or requiring publication of the original data from a randomized controlled trial (RCT). I also examined if the journals suggested or required pre-registration of RCT study protocols in a clinical trial registry. Similar editorial policy information was obtained from the 10 highest ranked general and internal medical journals, along with a random sample of 40 other medical journals, for comparison purposes. I chose to assess these lower ranked medical journals to allow for a more equal comparison between dental and medical publications with relatively similar impact factors.

The publication policies of the 10 highest ranked medical journals are more explicit than the dental journals and lower ranked medical journals. No dental journal suggests or requires the publication of the raw data with manuscript submission (Table 1). However, some dental journals allow the submission of supplementary material, such as data sets or additional figures or tables. Almost one-third of the dental journals (29%) require or suggest the pre-registration of the trial protocol in a public entity.

Although some of the high-ranked medical journals suggest that the raw data be published, most do not make it a requirement. (Table 2). However, most high-ranked medical journals require the pre-registration of the trial protocol in a public entity. None of the lower ranked medical journals suggested or required the publication of the raw data with manuscript submission (Table 3). Similar to the dental journals, less than one-third of these journals (24%) require the pre-registration of the trial protocol in a public entity.

Why Transparency in the Publication Process Is Important

The results of my review demonstrate that the editorial policies of dental journals could be improved. With no policy even suggesting or requiring the publication of the original data, monitoring the reliability of the data by editors and reviewers is a daunting challenge. A recent paper⁶ suggests measures that could be taken to reduce the chance of scientific misconduct in dentistry. Although this article recommends other interesting measures to improve transparency in the editorial process, such as software to control plagiarism and detect image fakery, the publication of raw data and the registration of the study protocol are not discussed.

Sharing the original data can be beneficial in many ways, most notably by enabling other researchers to replicate the study findings. Replication is one of the cornerstones of experimental studies,⁷ and the repetition of a study under similar conditions should increase the reliability of the results. Publishing raw data also opens up pivotal parts of the research, like detailed statistical assessments and complete results of the primary research.^5,8,9 Such elements allow researchers to thoroughly interpret the findings. Publishing the research protocol could reduce cases of selective outcome reporting (where researchers report only some of the analyzed outcomes) and generate less biased meta-analytic estimates.¹⁰ In fact, there is some evidence that trial outcomes can sometimes be inconsistent with protocols.¹¹ This would justify having journals adopt a policy of demanding the research protocol of a clinical trial with the final manuscript.

If other journals adopted some of the editorial procedures of the high-ranked medical journals, transparency in research would improve. For example, for industry-sponsored trials, these medical journals ask that the complete data set (protocol, raw data and paper) be assessed by an independent statistician to check the reliability of the data. This is requested because industry-funded trials are more likely to be associated with statistically significant pro-industry findings.^12,13Authors could also be asked to report details of any possible previous or duplicate publication in a covering letter, which would reduce the chance of publishing duplicate data and would optimize the secondary research process in systematic reviews and meta-analyses.

This article provides suggestions for changes to editorial policies that would improve transparency in the sharing of data in dental publications. Having authors publish the study protocol in the public domain would be an improvement of the editorial process. But the dental community should now decide if it is time to go further and start requesting that authors share the raw data from clinical trials. I hope this article will stimulate constructive discussion of this topic.

THE AUTHOR

Dr. Faggion is a research fellow in the department of prosthodontics, University of Heidelberg Dental School, Heidelberg, Germany.

Acknowledgements: The author was partially funded by a postdoctoral fellowship from the medical faculty of the University of Heidelberg.

Correspondence to: Dr. Clovis Mariano Faggion Jr., Department of prosthodontics, Dental School, University of Heidelberg, Im Neuenheimer Feld 400, 69120 Heidelberg, Germany. Email: clovisfaggion@yahoo.com

The views expressed are those of the author and do not necessarily reflect the opinions or official policies of the Canadian Dental Association.

This article has been peer reviewed.