I’ve had several discussions recently about “LLM-written” papers which went nowhere founder because people interpreted “LLM-written” in very different ways. Below I outline 4 types of LLM-written papers which I see fairly often. Of course these are just a few points in a large space, but hopefully distinguishing between these broad types will lead to more useful discussions about appropriate usage of LLMs in creating papers.

Type A: LLM-assisted paper which is checked by authors

Authors of these papers use LLMs to do things such as:

• Brainstorm new ideas: I dont do this myself, but I know other researchers who find this to be very valuable.
• Literature search: The AI literature is huge and expanding exponentially, and LLMs can be very useful for finding related work (I often use LLMs for this).
• Advice on experimental design, such as best statistical test: I sometimes do this.
• Using LLMs to improve writing. I do not do this myself, but I know many authors who are not native English speakers who find this very valuable, and also many readers who benefit from improved writing.

Crucially, in Type A papers, authors check the above. For example, if an LLM suggests a related paper to me, I read the paper, I dont just rely on an LLM summary of the paper.

I think most people regard Type A papers as acceptable. The ACL guidance on usage of generative AI allows Type A papers.

Type B: LLM-assisted paper which is not checked by authors

Type B papers use LLMs for the same purposes as Type A papers, but authors do not check LLM output. For example, if an LLM suggests a related paper, authors will include this paper in their literature review, using an LLM summary of the paper; they will not read the paper themselves.

This is dangerous, and poor practice. Authors are responsible for papers and should check them, in the same way that doctors are responsible for medical decisions and should check any AI advice they receive. From a practical perspective, if a Type B paper has a serious problem such as a hallucinated citation, then this could lead to the paper being rejected and in some contexts could potentially lead to authors losing reputation or being blacklisted.

Unfortunately, Type B papers seem to be common, and indeed I think some students now see this as the norm. This largely reflects a research culture which prioritises quantity over quality and indeed does not seem to place much value on scientific rigour.

Type C: LLM-written papers based on author ideas

Type C papers are completely written by LLMs, but are based on high-level research questions and experimental design from a human author. In other words, a person writes some research questions and ideas for experimental design (which may be just half a page of bullet points) and asks an LLM to flesh out the design, run the experiment, analyse the data, and write the paper.

At TACL, I see many papers which claim to be of this type. In other words, the paper is clearly written by an LLM, but the authors claim that the LLM was simply investigating their idea.

Nihar Shah of TMLR did an eye-opening experiment where he contacted ten authors of papers which were going to be desk rejected, and asked the authors to explain their paper to him (blog). Three of the authors could answer high-level questions about their paper but had difficulty when asked further questions on technical details. In my mind, this suggests a Type C paper.

I am very sceptical about such papers. Experimental design and execution is difficult, and must be guided by deep knowledge of research questions, datasets, baselines, etc. LLMs can certainly give advice on this, but I would not trust an LLM to do this on its own. Also, LLM-written papers are often incoherent – the individuals sentences make sense, but the paragraphs do not link together into a cohesive narrative. Indeed, for this reason TMLR is increasing the importance of clear presentation in its review criteria (blog).

Type D: Completely LLM-written papers

Type D papers are completely written by LLMs, with no human input other than a prompt asking the LLM to write an ARR (etc) paper. They seem to be common; in Shah’s experiment at TMLR, 3 of the 10 authors were unable to answer basic questions about the paper, and another three refused to talk to Shah. I suspect these were Type D papers. Other venues, including Arxiv, have also reported papers written by “authors” who did not understand “their” papers. I have also attended conference talks where authors could not answer basic questions about their paper.

It goes against all scientific principles for an “author” to submit a paper which he does not understand and did not write. In science, authors are responsible for their papers, including responding to questions and concerns. If they cannot do this because they are clueless about “their” paper, then the paper is not a valid scientific contribution.

Unfortunately, large numbers of “authors” have zero interest in science and just want papers in order to achieve a career goal such as getting a good job with a company or getting into a good university. Such people may see nothing wrong with asking Claude to write 10 papers for them and submitting the papers to various venues; but this is not science!

Discussing whether LLM-written papers are OK

Going back to where I started from, discussions of LLM-written papers need to take above into account. For example, I recently complained about Type D papers, and several people responded that Type A papers are fine, so why am I complaining?

I think most people believe that Type A papers are OK, although there are exceptions, especially if we look beyond computer science (humanities editorial). Likewise I think most scientists (excluding AI gurus and corporate PR teams…) think that Type D papers are not acceptable. Opinions differ about Type B and Type C; I do not like them, but other people accept them, and we can have useful debates about whether they are acceptable. But such debates need to be clear about what kind of paper they are talking about; debates about LLM-written papers in general, which ignore the above distinctions, usually lead nowhere, at least in my experience.