Brian Derby writes a post with the title Refereeing Chores at his Nature Network blog. Brian has refereed hundreds of papers in his scientific career, so he’s as experienced as it gets at the process. Even so, he writes about how it took half a day to reach his decision – to recommend rejection of the submitted paper – and had yet to write his report in suitably critical yet constructive tones for the author and journal.
The process required a first read to form an initial impression, a second read, to use Brian’s words, “required that I check up on some of the references and make sure that the article is consistent, which it was but I did not agree on the applicability of the model proposed. More reading of the bibliography. I discover at least one self-reference that appears to be there for no reason other than to (perhaps) improve citation score (I am now grumpy). I discover another reference that is on a completely different subject that must be in by mistake…. A discreet phone call to a colleague allows me to discuss the mechanism and we agree that it is flawed. Now follows the difficult bit, I have to write a carefully worded letter justifying my concerns to the editor.”
Or to put it another way, here is how the Nature journals recommend a review is written:
“The primary purpose of the review is to provide the editors with the information needed to reach a decision. The review should also instruct the authors on how they can strengthen their paper to the point where it may be acceptable. As far as possible, a negative review should explain to the authors the weaknesses of their manuscript, so that rejected authors can understand the basis for the decision and see in broad terms what needs to be done to improve the manuscript for publication elsewhere. This is secondary to the other functions, however, and referees should not feel obliged to provide detailed, constructive advice to authors of papers that do not meet the criteria for the journal (as outlined in the letter from the editor when asking for the review). If the reviewer believes that a manuscript would not be suitable for publication, his/her report to the author should be as brief as is consistent with enabling the author to understand the reason for the decision.
Confidential comments to the editor are welcome, but it is helpful if the main points are stated in the comments for transmission to the authors. The ideal review should answer the following questions.” (More than a dozen follow, and this list is followed by a second list of “further questions to consider”.)