A few weeks ago I was chatting to one of our PhD students, who had presented a paper at ACL 2018 (which he was justifiably very proud of).  He asked me about future publications, eg should he publish in ACL again or go for other high-prestige conferences such as NAACL or EMNLP.  I told him he should consider publishing in a journal such as TACL, Computational Lingiustics, or Natural Language Engineering, and he was very surprised; he had never previously really thought about publishing in a journal instead of a conference.

Advantages of Publishing in Journals

I suspect a lot of students (and indeed other researchers) in NLP have similar attitudes; they are focused on conference publication, and don’t consider journals.  Which is a real shame, because I think it is much *better* to publish in journals, for the following reasons:

Better reviewing process: Journal authors can properly interact with their reviewers, and improve their paper based on reviewer’s suggestions and concerns.  Reviewing has absolutely improved many of my papers (see previous blog), and I like to think that my comments have improved many of the papers I have reviewed.   This can happen with conferences as well, since usually authors can submit revised versions after seeing reviews, but in my experience its much less common.  Partially because there isnt much time for this (usually people only have a few weeks to prepare final versions), but also because many conference paper authors don’t in fact significantly update their papers, which in turn discourages reviewers like me from making suggestions.  Certainly when I review a journal paper, my mindset is “how can this paper be improved”; whereas when I review a conference paper, my mindset is more “accept or reject”.

Flexibility over timing and size: Conference papers are submitted on a rigid schedule and need to follow rigid size constraints.  Journal papers, in contrast, can be submitted at any time, and often have some flexibility about size as well.  So if I am publishing in a journal, I dont need to wait for a suitable conference deadline, I can submit when I’m ready.  And if I need a few more pages, its often (not always) possible to arrange this.

No travel: If you are presenting at a conference, you need to physically go to the conference.  While for many young researchers this is a perk, for someone like me (with a many family and professional responsibilities) it can be a real hassle.  The need to travel is also unfair to researchers who dont have much funding, such as most researchers from poorer countries, and more generally people who are not very good at playing the “funding game”.

Disadvantages of Publishing in Journals

Are there advantages of publishing in conferences?  In the past, one factor was that journals were slow in publishing papers.  I think this is less true in 2018 than it was in the past, though.  In particular, TACL usually gets papers out in 6-9 months (from first submission to formal publication), which is not much longer than conferences (and of course TACL papers can be submitted at any time, you dont need to wait for a conference deadline).  Computational Linguistics is getting better as well.   My most recent paper there (A Structured Review of the Validity of BLEU) (which benefitted immensely from reviewers comments and suggestions!) only took about a year from initial submission to formal publication (and it was available 9 months after submission under the  journal’s Early Access tab).

Another issue (which the above-mentioned student told me) is the perception that conference papers are better from a CV perspective than journal papers.    I dont think this is true; certainly when I look at CV’s I put more value on a good journal paper than a good conference paper.  Having said this, I realise that TACL in particular does not yet have a journal impact factor, which causes problems in some CV-related contexts.  However, if you’re worried about this, there are plenty of NLP and AI journals (including Computational Linguistics) which do have impact factors.

Incidentally, a week before my above-mentioned discussion with the NLP student, I attended a faculty meeting about publications, and people there from other disciplines were astounded that in Computer Science, conference papers are regarded as serious publications! CS is very much the outlier in this regard, almost all other fields expect serious research results to be published in journals.

In all honesty, I suspect many people prefer conferences because its easier to publish low-quality research in conferences.   There are a lot of really bad papers in ACL conferences (I wrote a blog about one, which unfortunately was not atypical).  And its not just ACL.  I recently asked my MSc students to choose a paper from IJCAI 2018 and evaluate its quality, and I was pretty shocked by what they found; most of the papers they looked at had major problems, and some were scientifically worthless.  You get weak papers in journals as well, but they are less common, and its very rare to get a paper in a good journal which is completely worthless.

And this should not be surprising to anyone who is familiar with the reviewing process.  Conference reviewing is essentially becoming a lottery, where outcome may bear little relationship to quality.  In part because the skyrocketing number of submissions to AI and NLP venues means that reviewing is done by people who are (A) under a lot of time pressure (B) to review many papers (C) many of which are in areas they know little about.  Whereas journal reviewing is still (mostly) of high quality, at least for good journals.

Conclusion

I think conferences and workshops are great venues for presenting speculative ideas, work in progress, etc.  But journals are the best way to publish solid research results!