When I asked participants what they most liked at the recent INLG conference, people highlighted events and sessions which focused on discussion and interaction, not technical research papers. Perhaps there is a lesson here that conferences should focus more on interaction and community, and not simply be regarded as venues for presenting research papers.
One of the highlights of INLG for me was the panel on “What users want from real world NLG”. I summarise a *few* of the really interesting points made about trust, authoring, configurability, human-in-loop, and other key issues for real-world NLG users.
Anya Belz and I are looking for a research fellow to work on a new project on reproducibility of human evaluations of NLP systems. This is a great opportunity for a researcher who wants to improve the scientific quality of human evaluations in NLP!
We’ve just completed a shared task on evaluating accuracy of NLG texts. This was really interesting, and amongst other things showed that current neural data-to-text systems struggle to learn how to use some words which have clear but relatively complex definitions.
I encourage students to have “exercises” where they critically read an academic paper, looking for problems in evaluations. This will help develop skills for writing as well as reading papers. So give it a go!
In 2016, I was shocked by the poor scientific quality of research in neural NLG. Fortunately, the situation is better in 2021! However, progress has been less than I had hoped, I think in part because the “leaderboard” culture does not encourage good science.
I’m looking for a PhD student to work on using AI and NLG to help cancer patients who are managing their condition at home. The student will be jointly supervised by people at Aberdeen’s Medical School. I think this is a very exciting PhD, and a chance to work on ideas that could make a real difference to people’s lives!
Why would anyone use a Bayesian model instead of a neural model in clinical decision support? Perhaps because the Bayesian model is much easier to justify and adapt to a changing world. Explaining Bayesian models is also a really interesting research challenge, and one of my colleagues has funding for a PhD student in this area.
I am really excited by the potential of NLG in healthcare. However, if we want to build real-world NLG systems which improve health, we need to surmount the “pain points” of data, evaluation, and safety.
If you are considering using a new dataset from a repository such as Kaggle, you should first check that the data in the dataset is of high quality and appropriate for your needs. A bit of “due diligence” at the beginning can stop you wasting lots of time and effort on an unsuitable data set.
Ehud Reiter's Blog 