I am of the opinion that the peer review system, as it currently stands, is unsustainable. I say this as an author, a reviewer, and an exasperated editor. Maybe it worked at one time, but not anymore. What changed? First of all, the enterprise of peer reviewed research has grown exponentially over the last few decades, especially with entry of non-Anglo players. Second, there is a fundamental mismatch between the extreme pressures to publish and the expectation that we should all participate in peer review. Everyone wants to get published. No one wants to spend time reviewing because that’s time not spent getting published. But if no one reviews, then no one gets published. It’s like a catch-22.
“Everyone wants to get published. No one wants to spend time reviewing because that’s time not spent getting published. But if no one reviews, then no one gets published.”
Those of us who play the research game know that it’s just that: a game. There is a lot of random chance involved. Did your manuscript land with the right editor? Did s/he send it to the right reviewers? Did the right reviewers drop out along the way? Doing excellent science is necessary to get published, but not sufficient. Sometimes you just have to hope that the cards fall your way.
Another drawback of peer review is that it rewards conformity and filters out controversial topics or approaches. At least in the social sciences, we all know that there are certain research questions that simply will not get published in the top journals, no matter how well answered.
But are there any alternatives to the current system? Thankfully, yes. Here are just three of them.
Option 1: Credentialized publication. Quite simply, everyone with a Ph.D. can publish whatever they want, whenever they want. Other scientists can then “rate” research output, such that the best output rises to the top. This follows the logic of the “marketplace of ideas,” where only good ideas will gain traction in the long term, and bad ideas will just be relegated to the dustbins of publication history. If there are problems with a piece of research, then someone can write a new paper pointing out the problems and link it to the original paper. This of this as a constant stream of comments and replies.
Option 2: Limited peer review. This is similar to the PLoS model of peer review. Under this system, only empirical papers will be reviewed, and where they are reviewed, the focus should be exclusively on the empirics and methodology. “But is the paper interesting?” Who cares? That’s subjective anyway. “Does it cite the right literature?” Who cares? That’s subjective anyway. “Does it use the right theory?” Who cares? That’s subjective anyway. All that matters in this system is that the methodology is of a high standard. If it is, then the paper gets accepted. This shift allows the research community to then decide for themselves on whether the paper is interesting, rather than a few reviewers deciding for us.
Option 3: Professionalization of peer review. Under this system, we create a division of labor between scientists and reviewers. Scientists do science, but do not spend time reviewing. Reviewers spend time reviewing, but not doing science. Obviously, under the current system, reviewers do not get paid. In a professionalized system, peer review will become paid employment. This is financially possible because of the huge profit margins of many publishers. In addition to giving scientists more time to focus on research, it creates new employment opportunities for the huge number of PhDs who are otherwise under-employed. Moreover, paid reviews will be much higher quality reviews, because low quality returns will render a reviewer unemployable.
These are just three ideas that have been floating around my head recently. There may be other innovative solutions as well. Do you have any great ideas on how to change the broken peer review system? We’re really going to have to think outside the box to make it sustainable moving forward.
Professor Andrew R. Timming
This article is published under a Creative Commons 4.0 License.