In the medical sciences roughly one in three articles contains an inconsistent p-value ( Garcia-Berthou & Alcaraz, 2004), and in psychiatry about one in ten articles ( Berle & Starcevic, 2007). Several large-scale studies estimated that roughly half of psychology articles using NHST contain at least one p-value that is inconsistent with the reported test statistic and degrees of freedom, while around one in eight such articles contain a gross inconsistency, in which the reported p-value was significant and the computed p-value was not, or vice versa ( Bakker & Wicherts, 2011 Caperos & Pardo, 2013 Nuijten, Hartgerink, Van Assen, Epskamp, & Wicherts, 2016 Veldkamp, Nuijten, Dominguez-Alvarez, van Assen, & Wicherts, 2014). However, NHST results are often misreported. The results of NHST underlie substantive conclusions and serve as the input in meta-analyses, which makes it important that they are reported correctly. Most psychological researchers use Null Hypothesis Significance Testing (NHST) to evaluate their hypotheses ( Cumming et al., 2007 Hubbard & Ryan, 2000 Sterling, 1959 Sterling, Rosenbaum, & Weinkam, 1995). We argue that open data is essential in improving the quality of psychological science, and we discuss ways to detect and reduce reporting inconsistencies in the literature. We did find that journal policies on data sharing seem extremely effective in promoting data sharing. Overall, we found no relationship between data sharing and reporting inconsistencies. In Study 3, we looked at papers published in the journal Psychological Science to check whether papers with or without an Open Practice Badge differed in the prevalence of reporting errors. In Study 2, we compared reporting inconsistencies in psychology articles published in PLOS journals (with a data sharing policy) and Frontiers in Psychology (without a stipulated data sharing policy). In Study 1, we compared the prevalence of reporting inconsistencies in two similar journals on decision making with different data sharing policies. We therefore hypothesized that journal policies about data sharing and data sharing itself would reduce these inconsistencies. Previous research found that reluctance to share data was related to a higher prevalence of statistical errors, often in the direction of statistical significance ( Wicherts, Bakker, & Molenaar, 2011). In this paper, we present three retrospective observational studies that investigate the relation between data sharing and statistical reporting inconsistencies.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |