Quality in peer review

Some thoughts on how to achieve quality in peer review from Communications Biology Editorial Board Members.

Christina Karlsson Rosenthal

Peer Review week is here and the editors at Communications Biology have taken the opportunity to reflect on the topic in a series of posts on the Nature Research community sites.  Our Editorial Board Members have experience of peer review from the perspective of authors, reviewers and editors. I have asked four of them to share their thoughts about how to achieve high quality in peer review. 

Editorial Board Members who contributed to this post:

Natalie Elia, Ben Gurion University, Israel
Tiago Dantas, Institute for Research and Innovation in Health, Portugal
Marco Fritzsche, University of Oxford, United Kingdom
Ngan Huang, Stanford University, USA

In your view, what are the characteristics of a high quality referee report?

Natalie Elia: A constructive document that focus on the main issues with the manuscript. And include (if possible) suggested experiments. 

Tiago Dantas: It contains detailed analyses of the experimental design, data and interpretation of results, contextualization of the findings reported with the literature and helpful criticisms with suggestions of feasible experiments to address them.

Marco Fritzsche: I think that a high quality peer review report presents itself as well-structured, unbiased, and educated assessment of opinion. Experienced reviewers differentiate themselves through a concise brief synopsis outlining in their view the scientific focus, novelty, and significance of the work. The quality and depth of the review becomes clear through a well-justified list of suggestions for revisions and their implications for becoming suitable for publication in the journal. To this end, as an editor, I also really appreciate a clearly stated recommendation for or against publication.

Ngan Huang: In a high quality referee report, the introductory paragraph shows strong understanding of the methodology and salient findings of the manuscript.  High quality referee reports provide constructive comments to improve the quality of the results and to strengthen the conclusions.  High quality reports also consider the state of the field and how the manuscript in question contributes new knowledge to the field.

[A high quality report is] A constructive document that focus on the main issues with the manuscript. (Natalie Elia)

What does a really low quality one look like?

Natalie Elia: A report that focuses on minor point / technical points or a negative report that is vague and does not specify the exact concerns of the reviewer. 

Tiago Dantas:  Short and superficial analysis, complete lack of constructive criticisms, asking for experiments that are completely out of the scope of the manuscript, or that are simply unfeasible. It’s also full of spelling and grammar mistakes.

Marco Fritzsche: At Communications Biology we don’t often receive a "very" poor peer review where the report does not match any of the above outlined criteria of a high quality review report in both a positive or negative sense. For example, I don't find it very helpful if the referee recommends an article for publication without any further explanations. Similarly, strong criticism without explanations or references to support their criticism is not fair to the authors, editors, or even the peer reviewers themselves because additional revision cycles are required extending unnecessarily the duration of the peer review process. However, we receive frequently average quality reports, which are too often unstructured and/or only partly match the criteria of a high quality peer review. These low quality peer reviews are sometimes also driven by emotional criticism.

Ngan Huang: In my opinion, a low quality report is one in which the report contains critical adjectives to deride the quality of the manuscript, without the provision of an approach to address the issues being critiqued (ie, "Do not publish this!").  Regardless of the quality of the manuscript, the language of the report should reflect level-minded and fair use of adjectives that should not personally attack the authors.  

...a low quality report is one in which the report contains critical adjectives to deride the quality of the manuscript, without the provision of an approach to address the issues being critiqued.(Ngan Huang)

Do you think there is anything the author can do to facilitate a good peer review process? If so, what? 

Natalie Elia: To set the goals of the work and the conclusions drawn as accurately as possible in the title and abstract.  

Tiago Dantas: Yes: The author can try to write their manuscript in the clearest way possible, highlighting: (a) the question addressed; (b) the advantages of the model system used; (c) the findings of their study.  They can also have their manuscript proofread by several colleagues. They should try to be as detailed as possible in the description of the experimental design/ methodologies and all conclusions in the study should be supported by its data. 

Marco Fritzsche: Yes, I believe that the authors should put themselves into the position of the referees and editor prior to submission. I recommend to review your own article, carefully prepare the presentation of their work especially the figures and manuscript text, and ideally ask your colleagues to pre-review the paper too. I find that authors too often don't make an effort to present their findings in well-structured Figures and writing. As an editor, reviewer, and author I have learned that judging the quality and impact of a paper is very challenging if the presentation is poor and lacking logic.

Ngan Huang: Besides recommending or excluding specific referees, the authors can consider having colleagues read through the manuscript prior to submission.  This way, the colleagues who are reading the manuscript with fresh eyes can more quickly provide feedback to improve the flow, clarity, and scientific merits of the manuscript, such that the manuscript would be easily understood by referees.

I believe that the authors should put themselves into the position of the referees and editor prior to submission (Marco Fritzsche).

What top tips do you have for an early career researcher reviewing their first paper?

Natalie Elia: To focus on the main issue and ask themselves whether the experiments they suggest are required and whether they will advance our knowledge. 

Tiago Dantas: Try to be as constructive as you would like another reviewer to be with your manuscripts, and, if possible, peer review your first manuscripts with your mentor while you are still a postdoc to gain experience. It’s a good idea to print figures, if they are at the end of the manuscript, so that you can analyse them as you read the manuscript.

Marco Fritzsche: The greatest mistake I made with my early peer reviews was that I had only read the article 1-2 times before putting my arguments together, which can make you unnecessarily critical. I recommend to carefully read an article, have a think from the perspective of the authors, and then finally putting your peer review together. Make sure that you provide a clear structure and recommendation with references to the relevant literature and criticism.  Generally, I recommend to review a paper with a positive attitude towards publication. I wish to recommend any referee to write the peer review to the best of their abilities putting themselves into the position of the authors and editors.

Ngan Huang: Before reviewing their first manuscript, early career researchers can get practice by co-reviewing manuscripts with their research mentors.  Also, as most manuscripts can benefit from the addition of more experiments and analysis, it may be difficult for early career researchers to determine what new experiments are necessary to be included in a revision manuscript and what experiments would be nice to have.  Co-reviewing manuscripts with research mentors can provide the opportunity to discuss how the addition of new experiments would improve the quality and impact of a manuscript.   

Try to be as constructive as you would like another reviewer to be with your manuscripts, and, if possible, peer review your first manuscripts with your mentor while you are still a postdoc to gain experience. (Tiago Dantas)