Mistakes in Evaluating ML
Unfortunately, I see many students (and indeed other people) make some basic mistakes when evaluating machine learning, for classifiers as well as NLG.
Author: ehudreiter
Grand Challenge: Helping the General Public to Make Data-Based Decisions
15 years ago, I siad a grand challenge for CS/AI./NLG was to help the general public effectively understand and use data. Progress on this has been less than I hoped, but this remains a worthwhile and important challenge!
Farewell to Richard Kittredge, pioneer in applied NLG
Farewell to Richard Kittredge, who died in early April 2019. Richard was a pioneer in applied NLG, and also an inspiration to me personally.
Skills Required to Use Different NLG Technologies
An important difference between different approaches to building NLG systems is the skills needed to use these approaches to build systems. Machine learning requires the most skills, smart templating the least, and simplenlg-type programmatic approaches are in the middle.
Bad Data Means Bad Output
Perhaps the most common reason for bad NLG output texts is low-quality input data. Ie, Garbage In, Garbage Out is true regardless of our technology.
Keep Good Records of Your Experiments
Someone recently asked me for details of an experiment I did 12 years ago, and it was not easy to get this information, because I had not properly archived it. Lesson: properly archive detailed information about experimental design, material, results, etc.
Evaluation Grand Challenge: Is NLP System Good Enough for a Use Case?
I was recently asked by someone if it was possible to easily determine whether an NLP system was good enough for a specific use case. Currently this is very hard. Making it easy could be a “grand challenge” for evaluation!
My Vision for SIGGEN
I am now chair of ACL SIGGEN. I hope SIGGEN can help the NLG community by encouraging high-quality scientific research, strengthening interaction with the non-NLP world, and providing trusted unbiased information about NLG.
Google: Please Stop Telling Lies About Me
People who search for me in Google see a Google-generated box which incorrectly says that I am Israeli. Google has ignored my complaints about this; they dont seem to care about the accuracy of their content-production algorithm. Which is ethically pretty dubious!
Does Deep Learning Prefer Readability over Accuracy?
In both NLG and MT contexts, deep learning approaches can result in texts which are fluent and readable but also incorrect and misleading. This is problematical if accuracy is more important than readability, as is the case in most NLG contexts.
Ehud Reiter's Blog 