I think there is enormous potential in using AI personal health assistants to improve health, including things like helping patients manage chronic illness, live more healthily, make informed decisions, and communicate with clinicians. There are huge challenges (technical and non-technical), but if this could be done well, it could radically improve health and enable healthcare systems to cope with increasingly elderly populations.
LLM benchmarks for coding are closer to real-world use than other LLM benchmarks, but they still do not measure real-world utility. I explain this by contrasting what is measured by SWE-bench with what is measured by a recent study of real-world utility in software development.
Current benchmark (suites) for evaluating LLMs are disappointing. I describe the properties that I think good benchmarks and benchmark suites should have, but often do not, such as being correct, challenging, diverse, and real-world.
I was very disappointed to realise that the evaluation suite for Amazon Nova (and I assume for other LLMs) has poor coverage of NLG tasks. Which is surprising since LLMs are largely used to generate texts; shouldnt they be evaluated, at least in part, on their ability to do this well?
There were lots of interesting papers in 2024. I describe a few of them, and also list others I have mentioned in previous blogs; all are about evaluation, experimental rigour, real-world utility, and/or healthcare applications.
I am very happy to see that the MT community is adopting the annotation-based MQM protocol as a gold-standard evalution technique. Having such a gold standard both strengthens evaluation and also supports exciting new research in evaluation.
A lot of experiments in NLP are neither rigorous nor replicable, and some NLP researchers don’t seem to care. Which is depressing… But I do see interest in incrementally improving experimental rigour and replicability, I strongly encourage people to try to do a bit better, even if they cant fix everything.
I’ve written a new book on NLG which was published in late October 2024. It tries to present a broad perspective on NLG, including requirements, evaluation, and use cases as well as technology. Perspective is similar to my blog in many ways, and I hope blog readers find it useful.
Deployed software systems need to be maintained as bugs emerge and the domain and user needs evolve; this perhaps is especially challenging for systems based on LLMs. Unfortunately little is known about maintaining NLG systems,
In NLG we focus on quantitative evaluation, but qualitative techniques can also be used. Quantatitive hypothesis testing is essential, but its also really useful to ask people what they think of an NLG system in an open-ended way.
Ehud Reiter's Blog 