The Reproducibility of Research Results
Recently, Dr. David Litchfield, chair, Department of Biochemistry shared with me an article from the Economist entitled, “Unreliable research Trouble at the Lab” (October 19th 2013 edition). I spent some time thinking about the questions it posed and felt it would be a great focus for my message this month.
This article focuses on the reproducibility of research results, or the lack thereof. We should all be aware that a cornerstone of the advancement of knowledge in all fields is “good science". To what does “good science” refer?
I believe, as scientists, we pursue answers to worthwhile research questions; pose worthwhile hypotheses; conduct research using proper research design, methodologies and controls; apply proper methods to analyze outcomes; and, make reasonable conclusions based on those outcomes. And our research and results must pass “peer-review”.
If an experiment is repeated in the same manner by another capable group of researchers using the exact same methods and approaches as we did, good science should report reproducible results.
The Economist article states “when scientists at Amgen tried to replicate 53 studies that they considered landmarks in the basic science of cancer, they were able to reproduce the original results in just six.” The article states, “the governments of the OECD spent $59 billion on biomedical research in 2012.” If much of the work being produced is not reproducible are these governments and the public getting value for their investment?
After reading this article, I was left asking the questions: Why is this happening? What is causing all this non-reproducibility in research results? Are we incompetent? Have we lost our integrity? Have we lost our ability to do “good science”?
I believe that for the most part we all are practising “good science” and are applying proper scientific methods and rigour to our research. We are acting with integrity and following the principles of research.
But then why is it so hard to reproduce an experimental outcome? Even the simplest experiment contains many variables that are virtually impossible to control equally all the time. It is in fact very difficult to precisely repeat even the simplest experiment exactly every time. Remember a 95 per cent confidence interval means your outcome would not occur five per cent of the time.
It’s important to remind ourselves often how challenging it is to uncover and substantiate new information on any subject. It is a privilege to be able to pursue research and a public trust to be allowed to do it. It must be conducted with the highest standards possible and even when that happens, 'the truth' can be very evasive indeed.
I want to commend all of you for your ongoing work and your good research practices. I would be interested in hearing about your perspectives on the article, which you can access through here.
Please send your comments to gradstudies@schulich.uwo.ca
Andy
