How to be a good reviewer (a view from an editor)
I’m on the editorial board of two journals and over the years have seen many reviews come in. What I have been impressed with is the overall quality of the reviews. Many of you are excellent and constructive reviewers. I have consistently learned from you on how to be a better reviewer (and editor) myself. This post is aimed more for the smaller fraction of reviews that I have seen – as well as a reminder for the rest of us.
Here, I thought I would summarize my frustrations with some of the reviews that I have seen. While still relatively rare, unfortunately, these seem to be increasingly more common. This list is by no means complete and there are likely some items I’ve forgotten. Further, there are several good guides out there on how to be a good reviewer (see here, here, and here) including a good one over at Dynamic Ecology.
Remember, the goal of the reviewer is to: (i) offer a clear path for for the authors for how the science could be improved; (ii) help educate the editor on why the paper is potentially important and where it stands/fits in with the literature; (iii) provide thoughtful, scholarly, and constructive advice for how the science could (if needed) be improved. As an editor I rely on my reviewers to offer a clear perspective on the novelty and importance of the paper. Further, I want to know how the paper fits into the bigger picture of the field – a scholarly view of the contribution. I look to you as a reviewer to provide this sagely view. . . .
My reviewer pet peeves (or a guide to providing constructive scholarly reviews)
- I know you as a reviewer are busy but it does not give you an excuse to be cranky and not provide any constructive comments. It may be the worst paper in the world – that does not mean you have liberty to trash talk authors, question, approach etc. The goal of a reviewer is to be constructive no matter how much you dislike the paper. Our science will only succeed if we help out our fellow researchers. Its your duty.
- PLEASE organize your review! You would be surprised with the ‘stream of unedited consciousness’ that sometimes I get back from reviewers. If I have to take time to reread your review over and over again so that I can understand your rational argument it probably means you have issues that go beyond the scope of the paper you are reviewing. Reviews should provide a clear roadmap for the authors and the editor for the importance of the contribution and how the science could be improved. I want to see a clear rational argument why a paper should be rejected or modified. A laundry list that is unorganized only makes my job as an editor more difficult and says that you really don’t care. See here for a good bit of advice for how to organize your review .
- Don’t say ‘this is well known in the literature’, ‘we already knew this’, ‘ the results have already been shown’ etc. without providing citations or justification. Anyone can type these words – its more difficult to backup your rational. As an editor I am looking to you to provide clear logic. You would be surprised how many reviewers do not back up their assertions/claims without citations! Explain just why ‘it is well known’ and show me who actually ‘showed these same results’ with providing a citation and/or quantitative reasoning.
- Please do not use ‘The paper is not novel enough for this journal’ as your main critique. Do not use ‘not novel enough’ as an excuse for rejection I see this comment all the time. It is really just code or short hand – its a sneaky way to shoot down a paper that you don’t like for whatever reason without providing solid justification. Its a cheap shot. Don’t do it. If you use the *not novel enough* card you should at least be able to point to very clear rational reasons (note plural) why as well as provide citations. You would be surprised how many reviewers resort to such a gameplan for shooting down papers.
- Don’t move the goalpost in your reviews. Yes, new issues may come up in the review but let’s be reasonable here. If you keep changing the rules of the game it just slows everyone down. Unless the issues are fatal to the central conclusions moving the goalpost comes across as another tactic for rejecting papers you don’t like.
- Don’t get overly bogged down in the weeds of statistical minutia, writing style, figure clarity etc. etc. As an editor (and as an author) I want to hear if the goals of the paper and the science are good. Step back and focus on the Science not the weeds. . . Focus on the conclusions of the paper. Do your suggested changes or the issues you may have really impact the conclusions of the paper? Is your issue a fatal flaw in the conclusions or are you just recommend how you would do the analyses? I want to know if the conclusions will change not how you would have done it.
- Bad organization is not necessarily a reason to reject a paper. Sure the writing may be crap or you may organize the paper differently but its your job to see the forest for the trees.
- I know you think the author should cite your work (or your favorite papers) but that is not an excuse to reject the paper.
- Please no personal negative comments, hearsay, hyperbole etc. A review is no place for water-cooler talk, gossip or for cheap shots. While I thankfully don’t see many reviews like this I have seen enough reviews that refer to “discussions I have had with others that did not like this work” or “the opinions of my colleagues and lab group” or “things I have heard about so and so” etc.
- If the manuscript is clearly from an undergraduate or graduate student try to go out of your way to be extra constructive. There is nothing more demoralizing to a young scientist than a terse, sloppy, bad review. One of the best experiences that I recently observed was when one of my newer graduate students received back a set of helpful and nice (yes, they were actually nice) reviews. You could see the light go off -‘ ah! this is how Science works! It is a peer self improving endeavor’. Those helpful reviews did more than I ever could do as an advisor.
- Be on time . . . mea culpa! This is one that I constantly struggle with as a reviewer myself. I certainly have come in late on reviews. Good science relies on all of us to be timely and we should all remind ourselves that publishing papers is one side of the publishing coin. We need to assist our colleagues with providing our reviews and doing so in a timely fashion is part of the science contract we all abide to.