In early 2021 I wrote a blog about my “vision” of using NLG to humanise data and AI. I recently looked at this again, and I think I missed something very important in this blog, which is that a system which communicates and humanises data should probably be interactive and conversational. The user should be able to tell the system what he/she is interested in, ask questions, add context, and otherwise interact with the system. In other words, we need conversational data-to-text.

Over the years I’ve worked on some projects where people interacted with NLG systems using a mouse. Eg users could click on a word and choose a “followup question” option, or select part of a graph and see a text summary of the data in the selected area. But until very recently I have never been involved with the most obvious interaction type for NLG, which is a chatbot where the user interacts via writing (or speaking).

This started to change in 2020, when I got a bit involved with a chatbot which Arria was developing to answer questions about business intelligence (BI) data, Arria Answers. As usual, I cant say much about Arria products here because of commercial confidentiality, but I did write an Arria blog about Answers which is public. Anyways, interactivity is a key part of most BI data visualisation systems, and so we’d like BI NLG to also be interactive! The technology forecasting firm Gartner has identified “Conversational Chatbots for Analytics” as a new technology in its “hype cycle”, and many BI vendors now include some kind of chat and question-answer interface in their product.

In 2021 I got involved with conversational data-to-text in my university work, when two of my PhD students, Simone Balloccu and Allmin Susaiyah (both part of PhilHumans) started working with chatbots to communicate medical information in e-health contexts (example paper). In Simone’s case this was after he had built and evaluated a non-interactive NLG system to communicate health information; he decided that switching to an chatbot would make his system more effective, and I agreed with him.

Whats important in conversational data-to-text?

A conversational data-to-text system must of course do a good job on core data-to-text functionality, that is it must produce easily readable texts which accurately communicate key insights about the data.

I think its also important that such systems should be able to use graphs as well as words when appropriate. Doctors who are involved in our e-health work have made this point several times, and Simone Balloccu is working in this.

I am also really keen on is user adaptation. Especially if someone regularly uses an interactive system, it should be possible to build up a good profile of the persons interests, linguistic competence, domain knowledge, and (more generally) what he/she needs help with. Using this information would allow the NLG system to be much more useful and effective! Allmin Susaiyah is working on user adaptation in our e-health work (example paper).

One thing I’m not sure about is whether conversational data-to-text systems should go beyond explaining data and engage in social interaction, show empathy, and more generally try to build a “relationship” with users? This is of course a major theme in dialogue research, and we have done some work in empathy in PhilHumans. However, my own experience is that whether this is desirable depends on context and use case. Sometimes users want to build relations with empathetic chatbots, but in other cases they just want the information presented clearly, and indeed can get annoyed if a chatbot starts pretending to be a person.

Last but not least, a commercial system must be robust! For example, it must work with different data sets, user queries, etc. The Arria Answers team has put a lot of effort into robustness.

Final Thoughts

I think conversational data-to-text is going to be very important, this is best way of explaining complex data to people in a lot of contexts. On a personal level, I’m also excited because this is something of a new direction for me. I’ve dabbled in dialogue and related HCI issues in the past, but its always been a sideline for me, so I’m looking forward to learning more about research (and commercial practice) in these areas!