Reviewing Guidelines for Literature Reviews

1. Declining a review request – Referees who need to decline a review request (for whatever reason, e.g. lack of time or lack of the specific expertise required) should as soon as possible notify the editor that they recuse themselves.

2. Conflict of interest – Referees should, as soon as possible, notify the editor that they recuse themselves from reviewing the manuscript if there is a conflict of interest. There is a conflict of interest where impartial and objective review is compromised for any reasons which could introduce bias or could be reasonably perceived by anyone as a possible source of bias.

3. Anonymity – The review process is double-blind. Referees shall abstain from any action that could breach their anonymity vis-a-vis the author. 

4. Tasks – A referee is expected to:    

4.1. Read the Guidelines for authors.    

4.2. Scrutinise the paper received for evaluation – This includes directly inserting in the paper detailed comments and suggested changes as necessary (using the track changes mode). The wording should be kept neutral and courteous. Once you have finished your review, please make the reviewed file (including your comments) anonymous before sending it back to the editor: File / Info / Check for issues / Inspect document / Deselect all options except “Document properties and personal information” / Inspect / Remove all / Close / Save.

4.3. Draft a concise evaluation report addressed to both the editor and the author – In this report, the referee will (a) explain the major strengths and weaknesses of the paper (both content-wise and with respect to the Guidelines for authors), and (b) make a recommendation in terms of publishing, revising or declining.

5. Criteria – Among others, the criteria to be used for the evaluation especially include:

5.1. Relevance of title, abstract, and keywords  – Do the title, abstract and keywords accurately reflect the major points of the paper? – Does the abstract comply with the journal’s requirements (i.e. “An abstract presenting in not more than 300 words the research question, the objectives of the paper, the method used for the literature review, the main results obtained, and blindspots identified as worth research consideration in future”)?

5.2. Quality of the introduction  – Does the introduction comply with the journal’s requirements (i.e. “An introduction setting up the research question including key definitions, and spelling out the stakes, objectives, and scope of the literature review. The last paragraph of the introduction shall list the major remaining sections.”)? – Could the introduction be improved? If yes, how?

5.3. Clarity and relevance of the method adopted for the literature review  – Is the method clearly described? – Is the method relevant? – Does the author discuss the respective quality of sources?

5.4. Clarity and relevance of the results obtained  – Are the results clear? – Are the results in relation to the research question?

5.5. Discussion  – Does the discussion take on board the approaches and results from existing research literature?  – Does the article identify blind spots that are worth research consideration in the future?

5.6. Conclusion  – Are the conclusions based on the review results? – Does the author draw all the conclusions that derive from the results? – Are the conclusions logical given the review results?

5.7. Scope and up-to-dateness  – Does the article embrace at least a significantly large part of the expected scope? – Aren’t major pieces of the existing research literature on the topic missed? – Does the literature review take on board the most recent research literature on the topic?

5.8. Overall added value – Has the article added value as compared to existing literature reviews on the topic?

5.9. Overall quality of writing and text structure  – Is the writing easy to follow and catchy? – Is the main text structured in a coherent manner?

5.10. Ethical issues – Does the manuscript seem to pose ethical issues (e.g. plagiarism, duplicate submission, redundant publication, etc.)?

6. Constructiveness and supportiveness – In the evaluation report, referees shall make sure to: (a) Explain the reasoning behind their criticisms, providing arguments and examples as necessary (in addition, it should be kept in mind that in case of rejection, authors may appeal based on contesting referees’ comments and looking for evidence that these comments were wrong); (b) Provide suggestions that could help the author improve the manuscript; and (c) Keep to neutral and courteous language.   

7. Recommendations – Referees shall conclude their evaluation report with one of the following five possible recommendations: (a) Acceptance (i.e., no revision is required); (b) or (c) Acceptance conditional to minor or substantial revision, along with a point by point response (i.e., upon receipt, the revised article will be checked by the editor, with the assistance of the referees if needed); (d) Invitation to resubmit the paper after in-depth revision, along with a point by point response (the revised paper will be sent back to the referees for reassessment before a final decision is made); or (e) Rejection.   

8. Private report to the editor – The referee may also draft an additional separate comment specifically addressed to the editor, which will not be disclosed to the author.

9. Confidentiality – The review process is confidential. Therefore, referees should not share the materials they receive with anyone, whether during the review process or after it has ended. In particular, they should not refer the manuscript to any colleague or student without the permission of the editor. Referees should also not make use of, or quote from, the manuscript before it is published. In addition, referees should also abstain from uploading manuscripts into tools that do not guarantee confidentiality, are accessible by the public, or may re-use the uploaded content (for example, generative AI tools like ChatGPT).

10. Ethics – Referees should be impartial and comply with standards of ethical conduct, including the ethical guidelines for peer reviewers set by the Committee on Publication Ethics (COPE), and the ethical standards “17. Responsibilities of Reviewers” of the Code of Ethics of the American Educational Research Association (AERA).