I am often asked how AI will impact Computer Science teaching. The biggest challenge is adapting what we teach so that it is relevant to a world where AI assistants are heavily used in software development. We should also use AI tutors to help teach. Least important is making assessments more resistant to AI cheating.

Quantitative comparisons of different LLMs are not very interesting in research papers, because the LLMs in question will probably be out of date by the time the paper is published. However looking for behaviour which is shared by several LLMs is definitely interesting and worthwhile.

ACL/ARR have rules and guidelines for how papers are written. Unfortunately many authors (and reviewers) ignore these, which makes their papers harder to read and less useful. Please follow the rules!

The most exciting and rewarding moments of my research career were when I discovered something new and exciting about NLG, language, etc. I describe a few of these “Eureka” moments. I hope my readers also have them; when I reflect back on my career, these moments and insights are what I remember best, much more so than getting papers or proposals accepted.

Research culture is very important but also very hard to change. I suspect this is one reason why it is so difficult to get people to do more rigorous and meaningful experiments.

Ive seen a number of diagrams recently which are too complicated and difficult to understand. I explain some of the problems I see and give advice.

The academic world has changed in many ways since I got my PhD in 1990. One of the worst changes is that researchers in 2025 usually need to pay thousands of pounds to publish their work. This is unfair to researchers with limited funding, and not good for science.

We have a really nice NLP research group in University of Aberdeen, with a dozen researchers who work on topics such as evaluation, interpretability, health applications, cognitive aspects, and cross-temporal research. We regularly publish and win awards in top venues. Its exciting!

There were lots of interesting papers in 2024. I describe a few of them, and also list others I have mentioned in previous blogs; all are about evaluation, experimental rigour, real-world utility, and/or healthcare applications.

A lot of experiments in NLP are neither rigorous nor replicable, and some NLP researchers don’t seem to care. Which is depressing… But I do see interest in incrementally improving experimental rigour and replicability, I strongly encourage people to try to do a bit better, even if they cant fix everything.