The ARR reviewing system for major NLP and ACL conferences is under enormous strain, because of skyrocketing submission numbers. I personally think the system will collapse if it is not changed. I also think that changes will be much more effective if they are based on a clear understanding of the purpose of these conferences. I wrote some thoughts about this on LinkedIn, which I have tried to pull together and expand in this blog.

As background, the Association of Computational Linguistics (ACL) runs a number of major conferences in NLP, which use a common reviewing platform (ARR). Every paper is reviewed by three reviewers and a meta-reviewer, and authors can respond to and rebut reviews. The reviewing process takes around 3 months, with another 3 months post-review before the conference (the post-review period includes creating the final “camera-ready” version of the paper). However, the ARR reviewing system is struggling to cope with an exponentially growing number of submissions (17K in current cycle, compared to 8K in equivalent cycle in 2025).

ARR needs to change, but we need to first agree on the purpose of ACL’s conferences.

Purpose 1: Researchers meet and discuss

The purpose of most scientific conferences is to allow researchers to meet and discuss what they are doing, share preliminary results, explain their hopes for the future, etc. If this is the main goal of ACL conferences, then we should (eventually) become like conferences in other academic fields, where people submit 2-page abstracts which are lightly reviewed. Serious papers are published in journals, not conferences.

I say “eventually” because such a change would have to happen in stages, not least because we do not have the journal infrastructure to handle huge numbers of submissions. But a first step could be to simplify the current review process; perhaps just two human reviews (and some kind of limited LLM analysis?), with no author rebuttal process. And at the same time start building up journal capacity (conflict-of-interest note: I co-edit one of the ACL’s journals, Transactions of ACL).

Purpose 2: CV enhancement

Alternatively, perhaps the main goal of ACL is to allow researchers to enhance their CVs. Lots of people are told they need N papers in good venues (such as ACL conferences) in order to start a PhD, finish a PhD, get a job, get promoted, etc. I personally suspect that much of the increase in ARR submissions comes from people trying to publish papers in order to meet such goals. Anyways, if the main purpose of ACL conferences is to allow researchers to show employers (etc) that they have lots of papers at ACL conferences, then I dont see why we need a high-quality review process.

To put this another way, I dont see why I should volunteer my (unpaid) time to review papers whose main purpose is to burnish someone else’s CV. I suspect a lot of reviewers feel likewise.

Purpose 3: Identify high-quality science

Perhaps the main goal of ACL conferences is to identify high-quality scientific papers. In this context we need to distinguish between recall and precision. Lets assume the goal is precision (any paper which is accepted is high quality science). I suspect this comes closest to the current view of ACL conferences, but in all honesty I think this is becoming impossible. The ACL conferences publish a huge number of papers and are not very selective (around a third of the papers in the ACL Anthology repository for 2025 come from ACL conferences and the closely-affiliated EMNLP conference). I also do not believe it is possible to do high-quality reviewing of 17K papers in 3 months. I suspect most of the papers published at ACL conferences are scientifically mediocre (this was certainly my perception when I attended ACL in 2025), and I do not see this changing.

If we truly want venues which publish papers which are guaranteed to be of high scientific quality, we need to look elsewhere. Perhaps the best bet is to focus more on highly selective journals, which is what other scientific fields do.

Purpose 4: Home for exciting science

Perhaps the goal of ACL conferences is high quality science, but recall instead of precision. Ie, we want to provide venues where important and innovative science can appear, and do not care if this means lots of rubbish also gets published. The best indicator of truly important papers is the ACL Test-of-Time award (for papers published 10 and 25 years ago which have had a major impact). Test-of-Time papers are often not recognised as ground breaking when they are published (none of them won a Best Paper award at the time of publication), indeed one of the 2026 Test-of-Time papers was published by an MSc student in a Student Research Workshop (paper). So we know that the papers that matter and have long-term impact are often not recognised as important when they are published. Perhaps the goal of ACL conferences is to provide a venue which welcomes large numbers of papers from many contexts and on all sorts of topics, so that the innovative “gems” have a home even if they do not fit the community’s current fixations.

If this is our goal, then again I would argue for a simpler and cheaper reviewing process, with a higher acceptance rate (standard advice if recall is more important than precision).

Do we need high-quality reviewing for ACL conferences?

Its not up to me to decide which of the above purposes are most important, thats up to the community at large. But I will say that purpose 1 (meeting and discussion), 2 (CV enhancement), and 4 (exciting science) all suggest a weaker, simpler and cheaper reviewing process. I personally would start by

• Erasing the distinction between main conference and “Findings” papers at ACL conferences
• Getting rid of author rebuttals (or maybe just allowing a very short one-off response)
• Reducing number of human reviewers from 3 to 2
• Using LLMs where they add value, such as in finding hallucinated references

The above changes might lower the prestige of ACL conferences. I personally think this would be a good thing, because it would encourage the large number of people trying to publish N papers in A* venues to submit their AI slop elsewhere, and leave us alone.

I think a high-quality reviewing process only makes sense for purpose 3 (identifying high-quality science). But the current ARR system does not do this well, and I dont think this is ARR’s fault, I think its impossible to do this well at the submission volume and timescales ARR is working at.

Final thoughts

The reviewing system for ACL conferences is struggling and may collapse, and needs to change. Change should be guided by a clear understanding of the purpose of these conferences. My preference is for conference reviewing to become weaker, simpler, and cheaper, but the decision is up to the community, not to me.

I also in no way mean to criticise the people in charge of reviewing for ACL and ARR! They are doing the best they can in difficult circumstances, and need guidance from the community as to what conferences are trying to achieve.