Duke TIP Study Reveals Low Replication Rate in Psychology Research.—November 30, 2012
For Immediate Release
DURHAM, NC—A recent Duke TIP study has revealed a staggeringly small percentage of replication studies in psychology research, which are crucial to the efficacy of the field.
Following recent notorious fraud cases and failed attempts to replicate controversial and classic findings, discussion of the role of replication in research has resurged. Replication studies can help identify and diagnose concerns about the integrity and reproducibility of research.
The research team, led by Duke TIP Gifted Education Specialist Matthew Makel, used the complete publication history since 1900 of 100 top psychological journals for their data. Their analysis revealed that roughly 1.6 percent of psychology studies use the term “replication” in their text. The team randomly selected 500 of the replication studies and found that only 68 percent of all articles using the term “replication” were actual replications of studies previously done, resulting in an even smaller overall replication rate of 1.07 percent.
“We sought to inform the current conversation about replication and its place in the field of psychology,” says Makel. “The replication rate we found is staggeringly low but not surprising—researchers aren’t eager to replicate studies when the deck is stacked against them being published. The rewards go to the original research and researcher, not the team confirming first place. Replication, however, is crucial to the field for scientific credibility. It reinforces the reliability of research, and serves as a safety check for fraudulent findings.”
Makel’s team also looked at whether the number of replications was changing over time and whether the original findings were successfully being replicated. They found that, despite numerous systemic biases, the rate at which replications are being published has increased in recent decades. Unlike previous findings in other fields, the vast majority of replications in psychology report similar findings to their original studies, and are therefore considered successful. These replications, however, were far less likely to be successful when there was no overlap in authorship between the original and replicating articles: the replication rate of studies involving original authors was 92 percent, while studies successfully replicated by an independent group were 65 percent.
“We hope the field of psychology comes out of this time of scrutiny and turmoil with stronger and more rigorous methods of research that ultimately lead to deeper understanding of the human mind. We are trained as researchers to ask questions—of our own research and of others’. I hope that we’ll take seriously the need for these questions to include assessing the work of our peers and not just our own,” says Makel.