Most of my blogs are from a “university” perspective,so I thought I’d write one which is more from an “Arria” perspective.   Not surprisingly, there is a lot of discussion at Arria about the best way to use machine learning (ML) in the sort of NLG systems that Arria’s clients want to build.   In my Chief Scientist role, I often end up giving advice and comments on the many ideas put forward by Arria staff, and I think the two of the most common comments I make are “this requires unrealistic training data and corpora” and “our customers dont want this automated”.

I’ve talked about corpus issues elsewhere in my blog, so I wont say much about this here.  Machine leaning, especially deep learning, requires large corpora for training, and such corpora have rarely been available in the NLG projects I have worked on (at the university as well as at Arria).    This of course is not a new observation, many people have made similar points!  I personally liked the way Regina Barzilay described, in her NAACL 2016 keynote (slides and video), her surprise at learning that she couldnt use the sort of ML techniques which are popular at ACL in a real-world medical NLP project.

I’ll instead focus on the other issue, which is what customers want.  If you look at Arria’s home page, the first thing you see is

Sophistication Meets Simplicity

The only NLG solution with advanced analytics + advanced linguistics in
one easy-to-use platform—customizable, configurable, controlled by you

The italicised phrases are key; Arria’s clients want NLG systems which they can customise, configure, and control.  Most of Arria’s clients do not want systems which are “black boxes” which they cannot customise, configure, and control.

So from an ML perspective, ML must be used in ways which make it simpler and easier to build NLG systems, but do not limit the ability of clients to customise, configure, and control their systems.   To take two extreme points, building a classifier to decide whether to use “a” or “an” is much appreciated, because this automates a routine grammatical task, and therefore simplifies project creation without limiting the ability of clients to control the things they care about, such as content and terminology.   But building an end-to-end deep learning “black box”  from a corpus is not appropriate, because it means that customers cannot customise, configure, and control the content and terminology of the texts produced by their NLG systems.

Of course there are many intermediate points, and the challenge is to understand how users want to customise (etc) NLG systems, and try to automate (via ML or otherwise) everything else, in order to simplify the authoring task.

I cannot give any more details about what Arria is doing, because of commercial confidentiality.   But I do want to encourage NLG researchers who are working on ML approaches to think about how these approaches impact the ability of NLG developers to customise, configure, and control NLG systems.    ML techniques which do not seriously limit developers in this way are more likely to be used in real-world applications.