What do Editors want?

The editor of the journal is the first person who will see your finished paper and the one whose opinion will matter the most. We seldom personally know the editor of the journal we are submitting to. We could as well be exchanging messages with a robot. But behind these emails, although many of them are automatic, there are real humans - researchers like me or you. Who are these mysterious editors and what do they want?

This week I interview two next-door robots, my good friends and colleagues, Jeroen van den Bergh, editor-in-chief of the journal Environmental Innovation and Societal Transitions, and Esteve Corbera, editor of the journal Geoforum. I discuss with them their roles as editors, the ways they make their decisions, and happy, or not so happy, moments in their experience as editors.

GK: Jeroen, you have already written for my blog what a good article looks like. When you receive a new submission for EIST, what is the first thing you look at?

JvdB: I am interested to see whether the research seems original, the abstract stimulating, and that the study (when empirical) has sufficient observations or interviews, and clear non-trivial conclusions. I also check if the title is succinct, informative and catchy. Many titles are too long, boring, misguiding or contain repetition of terms or notions. Sometimes I ask authors to adapt minor things, such as title, abstract, highlights, section structure or conclusions, before inviting reviewers.

GK: And you, Esteve? 

EC: I first look at the title and abstract, to first make sure that my editors-in-chief have not sent me something which is far away from my expertise. Sometimes this is inevitable given the amount of submissions Geoforum receives every month. I then read the introduction, the methods and the conclusion before sending a paper out for review. I want to be sure that the paper is both theoretically and methodologically novel and rich, and that it has a solid argument and appealing narrative. It’s not only me, all Geoforum editors look for the same thing.

GK: I am sure they do, and that’s what stresses us authors… The worst fear of an author is desk rejection. I don’t know about you, but I imagine these desks that reject papers to be dark brown and made out of heavy mahogany. Esteve, when does your desk reject a paper without sending it out for review?

EC: Haha, not it’s not my desk rejecting the paper, it is always me believe me. I reject a paper often when it is out of scope or when it is weak, which I can detect through a first reading. However, since the journal’s scope is wide, I also consult with my editors-in-chief if I am unsure. In Geoforum, we are receiving an increasing number of submissions and our desk-rejection rate is increasing. This is also happening across other reputable journals. Sometimes, we get articles that have not even been formatted to fit the journal’s style and audience, which probably means that they had been submitted, and probably rejected, elsewhere. Pay attention to this authors, no one likes to be the second choice!

JvdB: I reject a paper when it is clearly not in the scope of the journal, or has a very weak empirical foundation, or deals with a too narrow issue that is likely to interest few readers. I also look at the text and if it is not well written or seems sloppy otherwise, then out! If I am unsure, I will consult one or more associate editors for a second opinion. We tend to quickly come to an agreement.

GK: I’ve been an editor to a couple of special issues and one of the nightmarish tasks is finding reviewers. How do you two choose reviewers?

JvdB: I first check the reviewers recommended by the author(s), then I search through the publisher’s system (EVISE) for further reviewers, focusing on people with many hits in terms of relevant keywords. I also look through the submitted paper for core references or much cited papers and may select some of the associated authors. Out of the resulting set I try to find four or five reviewers. I invite two or three of these immediately and keep the others in stock in case one declines the invitation. Sometimes I look wider, such as in Scopus, for authors of similar studies.

EC: Choosing reviewers is my worst nightmare, you are right. I do what Jeroen does. We use the older EES system, which is similar to EVISE and I look for reviewers based on keywords and topic, across scientific databases, or simply look for relevant reviewers who might appear in the article’s reference list. Many reviewers do not respond because some of them might have left academia (e.g. former PhD students), or simply ignore or delete the journal’s invitation to review. The EES system warns us when a potential reviewer has published with the authors in the past. On average, I invite 15-20 people before I get 2-3 reviewers! Occasionally, I send an email directly to authors to request a list of 5-10 potential reviewers with whom they do not have a conflict of interest. They often respond with a list of people that I already invited!

GK: Ok, the reviews are back. Do you become robots again and forward them to the authors with an automatic message or do you make a synthesis and explain what you want them to do?

JvdB: If the reviewers have similar advice (minor revision, major revision, reject) it is fairly easy, I just follow their advice. I first check if the reviewers have made any confidential comments to me as the editor. If they haven’t, then I glance through their comments to the author to see what their main concerns are. Based on what they write, in the decision letter to the author I will stress particular issues that need serious attention. If the reviewers have inconsistent advice, I look in more detail at their reviews to see if one is clearly of better quality. If not, I may add a third reviewer or I look through the paper myself to arrive at a fair decision.

EC: When reviews are aligned and not contradictory, I tend to forward the comments as they stand without adding my own judgement. As an editor, I prefer to let the reviewers indicate how the paper should be improved, rather than act as an additional or steering reviewer. When two reviewers reject, I reject. When reviews are contradictory (e.g. major and minor revisions), I do offer suggestions to authors on how to proceed. When one of the reviews rejects a paper and the others recommend revisions it is trickier. I first read the arguments of the rejection and compare them with those of major/minor revisions. I often look for a third or fourth reviewer, often another Geoforum editor or member of our international board. Then I make a decision. Often reviewers who reject a paper are in fact suggesting a very major revision doable with some weeks of additional work. In these cases, and when other reviewers are positive, I give the authors a second chance. The problem is that sometimes when the article is re-submitted the reviewer who rejected it can be surprised, or even angry that we didn’t follow her or his suggestion, and then does not want to review the paper again.

GK: Next step. The authors work on a revision and resubmit. Do you read the full revised paper or focus only on the response letter?

JvdB: Both. I like a short letter from the author to me, followed by a detailed response to the reviewers. I glance through this response and the paper to see if I notice some clear errors or problems. If so, I ask the authors to make further changes. If not, I send the paper with the response to the previous reviewers.

EC: Yes, the response letter is super important. It needs to state clearly how each of the comments have been addressed, or if not why. This can be done using a table, or a numbered list for each of the reviewers’ comments. I read the letter of response and how the authors claim they have addressed the comments, looking then at key parts of the manuscript. I then send the new version of the article to the original reviewers. If one of two of these are not available and the original submission deserved a “revise and re-submit” or a “major review” I look for additional reviewers.

GK: I know you both well, and you are good people. Are you also good when you are editors or do you feel like Roman emperors in the Coliseum, waving your thumb up or down to the poor gladiators? Do you want to help all authors improve their paper and get published, or only get the best papers published?

JvdB: I don’t think the two are incongruent. I tend to give the benefit of the doubt to the authors. Hence, if reviewers disagree, I would rather invite an additional one. I always check if reviewers are fair and offer good-quality reviews. Sometimes I press reviewers to reconsider if I feel they are too quick or stern in their judgment. My experience, also as an author, is that editors of other journals unfortunately often assume the reviewers are always right. But we are all peers with imperfections and biased judgements.

GK: Ok, I see, a soft-hearted Emperor. And you Emperor Esteve?

EC: Your metaphors are getting gory, Giorgos. I agree with Jeroen that these are not opposite categories. As I noted earlier, we are increasing our desk-rejection rate at Geoforum, but we tend to be very rigorous in selecting reviewers, who are ideally three for each article. I believe, and my fellow editors do too, that the more constructive comments an author receives, the better the published article will be too. In my life as an author, I have always found reviews very helpful and instructive, and in my life as an Editor, I’m hoping that Geoforum authors feel the same too. We are not really Emperors and this is not a fight. We are peers helping other peers improve their work.

GK: I agree. My gladiator metaphor was too much, though sometimes the interaction with journals can seem like a fight. As I’m growing older, I get more confident in contesting editors’ decisions if I think they are unfair. Are you willing to be convinced by an author who fights back after a rejection?

EC: I have been an editor of Geoforum for almost two years, and I have not yet faced such a situation. But if this ever happened to me, I would consult with my editors-in-chief and respond collegially and politely to the author.

GK: Yes, assuming they were collegial and polite to you. And you, Jeroen?

JvdB: I haven’t received either many of these requests, perhaps because we do try our best to make fair and well-motivated decisions. My response I guess would depend on the type of reviews. If reviewers suggest something between reject and major revision, and the study clearly is important and innovative, then I may decide to accept a new submission of a much-reworked paper. But I don’t want to overrule reviewers, as they are the backbone of our journal. So, I always try to get the explicit approval of reviewers before making any final accept decision.

GK: Any funny story from your experience as editors?

EC: I can’t think of any funny anecdote. Being an editor is not funny. It is a rather poorly paid, and very time-consuming contribution to the academic community. You spend a lot of time reading the submissions, writing emails, corresponding with other fellow editors, with authors, screening and evaluating special issue proposals, attending meetings with the publisher, organising journal events and activities. We would be more understanding with journal editors, with reviewers, and be more sympathetic to the peer-review process if every doctor in academia worked as an editor for some time! We would also learn more about the publishing industry and understand its major flaws, challenges and advantages.

GK: Ok Esteve, this is heart-wrenching, definitely not funny. You just won the award for worst stand-up comedy of 2018. Jeroen, I know you can do better.

JvdB: One author submitted the same paper three times, with slightly distinct titles. Each time it got rejected – the first time after reviews, the other times as desk-rejection. This seemed like a test of the memory of the editor (me). He was unlucky that my memory still functions fairly well. This author is now blacklisted.

GK: Thank you both for this interview and for your work as editors, which is a laudable service to the academic community.