The peer review system is so broken, it’s hard to even know where to start. Maybe it’s similar to Churchill’s views on democracy: peer review is the worst system of evaluating the quality of a manuscript, except for every other system of evaluation. But maybe—in fact, certainly—it has room for improvement.
Revolutionary improvement would require imagination and audacity. It would mean overthrowing the current model and replacing it with a new system based on wholly different fundamentals. I’m not, as yet, certain what this would look like, but I’m trying real hard to imagine it.
Then there is the possibility of incremental improvement. This would involve changes to the current system that target its vulnerabilities, of which there are many. It might mean re-working the funding model of scientific publication, replacing perverse incentives with “healthy” ones, and changes to the process.
But, as with any change process, alterations can make things better, but they can also make things worse. Ultimately, it is an empirical question. If we want to improve the peer review system, we’re going to have to try out novel approaches to evaluation and assessment for while and see if they work. If they don’t, at least we’ll know, right?
One of the key problems I see in peer review is that it mimics, or even compounds, the dark side of social media. I’m talking about anonymity. It’s bad enough that social media brings out the worst in named accounts; it’s even worse when it comes to anonymous accounts. Anonymity affords a protective veil that enables toxicity. Because peer review is anonymous, reviewer comments often descend into soul-destroying incivility and/or unreasonable demands.
“Anonymity affords a protective veil that enables toxicity. Because peer review is anonymous, reviewer comments often descend into soul-destroying incivility and/or unreasonable demands.”
Yes, editors have an obligation to step in when there is evidence of bullying, but they are far less likely to intervene over incivility or unreasonableness. These have become normative in the process.
What if we had to sign our reviews? I know that this happens on occasion in the natural sciences, but it is virtually unheard of in the humanities and social sciences.
Let’s start with the disadvantages of de-anonymized reviews. Obviously, retaliation is a concern, especially for early career researchers. As the old saying goes, “an eye for an eye makes the whole world blind,” or rather, “unpublished.” We could end up with never-ending vendettas that shut down publications. But isn’t that why we have editors? To arbitrate? The opposite may also be a problem. We might see a “I scratch your back if you scratch mine” dynamic evolve. Of course, both of these problems could be dramatically reduced if signed reviews were published with the final article. My signature on a review would guarantee that I will stand by my comments through hell and high water.
This leads us to the advantages. Signed reviews could dramatically reduce incivility, especially if they were published alongside the final article. They could also tone down the “nit-pickiness” of anonymous peer reviews. Let’s face it. Most peer reviews of fundamentally sound manuscripts say something to the effect of: “This paper still sucks because you didn’t write it the way I would have.” My guess is that we’d have less of that nonsense with signed reviews. For my part, I would only highlight what I consider to be objectively unsound elements of the manuscript, thereby placing my “preferences” aside. I would also feel less of an obligation to go in search of problems that don’t exist.
What do you think? Should we all start signing our reviews? Would you be willing to do so? I don’t know about you, but I would.
Professor Andrew R. Timming
This article is published under a Creative Commons 4.0 License.
To submit an article to Dire Ed, visit http://dire.ed.com/submissions/