Accuracy, Fluency, and Utility
Texts produced by NLG systems can be evaluated in terms of accuracy (content is correct), fluency (text is readable), and utility (text is useful). I discuss these three “dimensions” of NLG evaluation.
Author: ehudreiter
Generated Texts Must Be Accurate!
I’ve been shocked by the fact that many neural NLG researchers dont seem to care that their systems produce texts which contain many factual mistakes and hallucinations. NLG users expect accurate texts, and will not use systems which produce inaccurate texts, not matter how well the texts are written,
The Brexit Mess
I am getting so fed up with UK politics that I will break my “no politics” rule in this blog and express my frustration with the Brexit mess and the way politicians have handled it.
I’m Impressed by Capetown Uni’s Diversity
I’m impressed by the diversity of NLG researchers at Capetown, from a gender, race, and disability perspective. An inspiration the rest of us!
ML is Used More if it Does Not Limit Control
When we try to use ML in commercial NLG contexts, one of the challenges is that NLG developers want to be able to customise, configure, and control their systems. So we need ML approaches which do not stop devs from configuring things they are likely to want to change.
Do We Encourage Researchers to Use Inappropriate Data Sets?
I’m beginning to think that in some ways the NLP community *encourages* researchers to use poor-quality or otherwise inappropriate data sets. Which is a truly depressing thought…
NLG and Explainable AI
Some thoughts on key NLG challenges in explainable AI: evaluation, conceptual alignment, narrative. Comments are welcome!
Why isnt Research Software such as BabyTalk Used?
Most research software does not enter everyday operational use. In part because research projects usually do not worry about issues such as maintainability, regulatory approval, and change management, which are essential to the long-term success of commercial software.
What Makes a Good Narrative?
Some thoughts on the properties texts need to have in order to be good non-fictional narratives, and speculations on how we might generate such texts.
Cycling Through Wales, England, and my Wife’s Family History
A travelogue about my recent cycling holiday, in Wales and England, where I saw many places related to my wife’s family history,
Ehud Reiter's Blog 