I was one of the main organiser of the (online) INLG2021 conference in late September. After the event, I asked several attendees what part of INLG they most liked. This was informal, based on people I happened to end up chatting with, it wasnt a formal survey. The responses were varied, and included our panel, the invited talks, generation challenges (NLG shared tasks), and our “Birds of a Feather” discussion sessions.

What surprised me was that no one mentioned technical research papers, oral sessions, or poster sessions as their favourite part of the conference. People told me that they liked and learned from the papers and posters, but none of the people I talked to said this was the highlight of the conference for them.

Maybe this shouldnt be a surprise. After all, you dont need to go to a conference to read its papers, you can read them in the ACL Anthology or indeed (for many papers) Arxiv. So it makes sense that the real “value-added” of a conference is in the other events, not the papers. Indeed, in most scientific fields, research results are published in journals, and conferences focus on discussion, interaction, work-in-progress, and so forth.

Birds of a Feather Discussion Session

When I asked people what they most liked at INLG, the most common response was our “Birds of a Feather” discussion session. This was a session where we had six Zoom breakout rooms, each discussing a different NLG topic. Attendees simply joined whichever breakout room they were most interested in, and proceeded to discuss the topic. No presentations or formal structure, although each topic/room had a chair. The topics (neural NLG, rule-based NLG, evaluation, resources, NLG in healthcare, explainability and trustworthiness) were chosen based on the results of a survey we ran a few months before INLG.

So a very simple structure, but seems to have worked very well for a lot of people. I suspect it was especially useful for attendees who were not presenting research papers/posters, since it gave them a chance to discuss their interests and work.

This session was partially inspired by the “Birds of a Feather” sessions at ACL conferences. However I think it ended up being quite different in practice because INLG is a much smaller conference (generally 100-200 attendees), compared to the thousands of attendees at the large ACL conferences. Anyways, I would definitely recommend such discussion sessions for other small conferences!

Panel on What users want from real-world NLG

The second-most common response to my “what did you like best” question was our panel on “What users want from real-world NLG”. Four people from commercial NLG or dialogue companies (all of whom were practitioners building real-world systems) gave their perspective on this question. This was one of the highlights of INLG for me personally, and indeed I wrote a blog about it, so I wont go into detail here. I should say that other people really liked the panel as well, it wasnt just me!

I think one of the reasons people liked the panel was because it addressed the big picture. Technical papers by their nature tend to be focused on specific research contributions, most of which (in all honesty) are incremental. The panel, in contrast, addressed the fundamental question of what users are looking for and what challenges the NLG community needs to address to make the technology useful in solving real-world problems.

Needless to say, opinions differ on this question, and I’m sure some of the participants disagreed with what some of the panelists said. But the important thing was that the panel was a chance to talk about fundamental “big picture” issues, and I think attendees really appreciated this.

Invited Talks

The third most common response to “what did you like best” was the invited talks. We had two invited talks: a talk by Natalie Schluter on automatic summarisation, and a talk by Tim Bickmore on health counselling dialogue systems. Bickmore’s talk in particular got some very positive comments, perhaps because it was a bit different. Bickmore is primarily a researcher in HCI, virtual agents, and health informatics. He is not an NLP researcher, but knows enough about NLP and NLG to present his work in an interesting and appropriate fashion to NLG researchers, and I think many attendees really enjoyed learning about research in these related areas.

Generation Challenges

Lastly, one person I talked to highlighted Generation Challenges. Generation Challenges is a set of shared tasks in NLG. This year, we had a session presenting results of shared tasks proposed at INLG 2020 (ie, last year), posters about individual submissions to these tasks, and proposals for new challenges to be presented next year at INLG 2022. I suspect the person who was keen on this may have been more interested in the shared tasks than in the submitted solutions. I actually helped to organise both of the shared tasks proposed at INLG 2020 and presented at INLG 2021, and I would absolutely encourage interested people to do further work on these tasks!

New look conference?

Should conferences focus more on above (discussion sessions, panels, invited talks, shared tasks) and less on presentation of technical research papers? At INLG 2021, we aimed for a roughly 50-50 split between oral and poster sessions presenting research papers, and the other events mentioned above. This was a change from previous INLG conferences (and indeed most NLP conferences), where most of the time is devoted to presentation of research papers. But maybe we should have gone further, and only devoted one-third (or one-quarter) of the conference schedule to research papers, and used the “freed up” time for more discussion sessions?

One way to achieve this is to drop oral presentations of research papers, and have all such papers presented as posters. As well as saving time, this would also encourage discussion and interaction. We actually discussed this option for INLG, but decided it was too radical. But in retrospect I think this is an option that conferences should seriously consider.

Final thoughts

In CS, AI, and NLP, we often think of conferences primarily as a venue for publishing research papers. But conferences should also be places where attendees can meet, interact with, and learn from other researchers. The best way of supporting this of course depends on the conference (eg, size and physical/virtual), but I think the above mix of events and sessions worked reasonably well for INLG.