ISSN: 2578-5095
Authors: Trafimow D*, Rodriguez A, Myuz HA, Wang C and Wang T
Background: Given widespread agreement about the importance of evidence-based medicine, it is crucial to know the precision of the data in top medical journals. Such research has been performed in psychology with pessimistic findings; but not in medicine. Methods: 30 articles in each of three top medical journals (90 totals) were randomly selected from all articles featuring sample means, using between-participants analyses, and published in 2017. The journals were The New England Journal of Medicine (NEJM), The Journal of the American Medical Association (JAMA), and The Lancet (L). Though not a criterion for selection, most of the articles included an Experiment 1 and an Experiment 2. Results: Article-wise sample means analyses indicate superior precision for medical journals relative to previously published findings from psychology journals (Md < .1 for all medical journals but Md > .2 for all psychology journals). However, NEJM outperformed JAMA and L for more focused experiment-wise analyses involving sample means in Experiment 1 (Md = .12, .17, and .20, respectively) and Experiment 2 (Md = .12, .17, and .21, respectively). Similar results occurred for differences between experimental and control group means (as opposed to the means themselves) in Experiment 1 (Md = .06, 21, and 23, respectively) and Experiment 2 (Md = .06, 21, and 27, respectively). Conclusion: Top medical journals perform well with respect to precision relative to top psychology journals. But there is room for improvement in medical journals too, particularly in JAMA and L.
Keywords: NEJM; JAMA; Lancet; Evidence-Based Medicine; Medical Science; Human Condition
