One of the things I am trying to come to terms in my new role as a TACL editor is desk rejection, that is when an editor rejects a paper without sending it to reviewers. In my career I have had some papers unfairly desk rejected (including at TACL), so I initially was pretty dubious about this. Even if a paper is not very good, surely the authors deserve some feedback after working on it for many months. But I am now coming around to the opinion that desk rejection is necessary, unfortunately, because of the rising number of submissions, including submissions which are either LLM-written or written by humans who have little interest in science and are just trying to improve their CVs. There is no obligation to provide feedback on such papers.

Papers can be desk-rejected for a number of reasons, including wrong format, hallucinated citations, LLM-written, inappropriate topic for venue, and weak content. At TACL, in some months 2/3 of submissions get desk rejected (the number fluctuates quite a bit). Desk rejection can be done by either editors-in-chief (like me) or action editors. Except for format desk reject, all desk reject decisions are checked by a second editor.

Format desk reject

Principle: papers are desk-rejected if they do not conform to a venue’s formatting rules, such as max/min page length.

This is the least controversial type of desk reject, and the most common type of desk reject at TACL; papers are desk-rejected because they do not follow TACL format. Sometimes the violations are extreme, eg a 40-page single-column Microsoft Word document which is not even about NLP (I assume the author used an AI agent to auto-submit the manuscript to a large number of venues). In other cases, the violations are smaller, such as papers which are slightly too long.

I personally am keen to strictly enforce rules that figures are readable at 100% size, and appendices follow TACL rules. I am not going to ask our reviewers to look at papers with unreadable figures (blog) or bloated appendices!

In principle, submissions can also be desk rejected if they violate multiple submission rules, eg are simultaneously submitted to both TACL and ARR. The challenge here is getting information on submissions to other venues.

Hallucinated citations

Principle: papers are desk-rejected if they contain citations to references that do not exist.

The ARR reviewing system for xACL conferences has recently started using tools to detect hallucinated citations; meta-reviewers are asked to manually check suspect submissions. Any paper with hallucinated citations is desk-rejected, and further penalties may be applied (eg, other submissions from the same authors are also desk rejected).

I support this type of desk rejection, and we are exploring ways to do this in TACL. Fabricated citations are completely unacceptable, and indeed a violation of research integrity principles.

Papers should also be desk rejected if they use prompt injection; ie, include prompts (usually invisible to human readers) which tell LLMs to rate the paper highly, if LLMs are used to review the paper.

Papers written by LLMs

Principle: papers are desk-rejected if they written by LLMs.

Formally speaking, TACL follows ACL rules on allowable usage of generative AI when writing papers; roughly speaking, these allow AI to be used for writing assistance, finding related work, and brainstorming. More informally, I strongly believe that TACL is a venue for papers written by *people*, not LLMs! We got quite a few LLM-written “AI slop” submissions, in some cases multiple such submissions from the same “author” in a month.

Detecting LLM-written papers is not straightforward, and is an evolving process. I will say that all such cases are manually checked and verified, it is bad practice to desk-reject a paper as LLM-written purely on the basis of a detector such as pangram.

Inappropriate topic

Principle: Papers are desk rejected if their topic is out of scope for the venue.

This is easy to say, but not always straightforward to do. At least for me, I also look at whether there is an obvious alternative venue. So I may reject a hard-core ML paper because I think it would fit much better into JMLR or TMLR, but I try to be tolerant of interdisciplinary papers which do not have an obvious alternative venue (this is partially because of unfortunate experiences which my students and I have had in the past).

Content desk rejection

Principle: Papers are desk rejected if editors think acceptance is very unlikely because of weak content.

This is the most controversial type of desk rejection. Essentially an editor looks at a paper for 10-15 minutes, and desk rejects it if he/she judges that acceptance is very unlikely. The rationale is that we do not want to waste reviewers time on low-quality submissions.

One problem with content desk rejection is that unusual papers (radically different approach, interdisciplinary, etc) are more likely to be desk-rejected for content than incremental papers (I have seen with my own submissions). But such unusual papers may be more useful than incremental papers! In part because of this, ideally content desk rejection would be very limited. Unfortunately, venues across science are seeing an explosion of low-quality submissions, largely because of the combination of AI-for-science tools and pressure to publish large numbers of papers, so content desk rejection is necessary.

Final comment/rant: Incentivising quantity is really bad for science

One of the things which really angers me is that incentives, especially for beginning and early career researchers, emphasise quantity instead of quality. People are told that they need N papers to start a PhD, finish a PhD, get a good job in industry, etc. Because of these incentives, some people will send out large numbers of LLM spam submissions; most will get rejected, but the submitter may get lucky (eg, have “reviewers” who use LLMs for reviews, since LLM reviewers tend to like LLM-written papers), and just a few successes will open the gate to good jobs, PhD positions, etc.

Even people who do not resort to LLM spam may decide to focus on churning out large numbers of so-so papers, instead of a small number of good papers, because of above incentives.

In science, quality is much more important than quantity- a good paper (eg, one which wins an Outstanding paper award) is 100x more useful than a so-so paper (eg, typical xACL paper), and a great paper (eg, one which will win a Test of Time award) is 100x more useful than a good paper. So it would be great if “AI for Science” tools were used to improve quality, since a single good paper is much more scientifically valuable than ten so-so papers. Unfortunately, incentives usually focus on quantity, which makes ten so-so papers are more valuable than a single good paper from a CV perspective. Therefore we are seeing a flood of low-quality submissions instead of a rising number of high-quality submissions. Which is bad for science, and increases the need for “unfortunate necessities” such as desk rejection.