Skip to content

Ehud Reiter's Blog

Ehud's thoughts and observations about Natural Language Generation

  • Home
  • Blog Index
  • About
  • What is NLG
  • Publications
  • Resources
  • University
  • Contact
Uncategorized

How can I tell if a paper is scientifically solid?

Apr 6, 2020 ehudreiterLeave a comment

A PhD student recently complained that to me that he was wasting a lot of time reading scientifically dubious papers. I give some suggestions on indicators of poor scientific quality in research papers.

Uncategorized

Lets Use ML for Insights!

Mar 20, 2020 ehudreiterLeave a comment

I’d love to see more people using machine learning to provide insights about NLG problems and related linguistic issues. I personally think this is much more useful than tweaking models to show a 1% increase in state-of-art in a very artificial context.

Uncategorized

Why do we still use 18-year old BLEU?

Mar 2, 2020Mar 2, 2020 ehudreiter2 Comments

NLP technology has changed and advanced over the past two decades, but it often seems that NLG evaluation has not. Why is the 18-year old BLEU metric still so dominant?

Uncategorized

Shared Task on Evaluating Accuracy?

Feb 18, 2020Jun 23, 2020 ehudreiter1 Comment

We’re thinking of organising a shared task on evaluating the accuracy of texts produced by NLG systems. Comments welcome, also let me know if you might participate.

Uncategorized

Do people “cheat” by overfitting test data

Feb 6, 2020Feb 11, 2020 ehudreiter1 Comment

NLP in 2020 is dominated by papers which report small improvements in state-of-art. I suspect that a lot of these improvements are due to overfitting test data, not to genuine scientific advances.

Uncategorized

AI professionals also focus on change management

Jan 27, 2020Jan 27, 2020 ehudreiter1 Comment

If we want to deploy AI in the real world, we need to think about “change management” issues. Eg if users think that AI threatens their jobs or adds extra hassle, then uptake will be slow. This has been a problem for AI and statistical algorithms since the 1950s.

Uncategorized

Amateurs focus on models; professionals focus on data

Jan 14, 2020 ehudreiter1 Comment

There is a military saying that “amateurs discuss tactics, professionals discuss logistics”. Similarly I think AI professionals should focus on data more than models. I suggest four simple initial questions to ask about your data if you want to build an ML system.

Uncategorized

Lessons from 25 Years of Information Extraction

Jan 2, 2020Jan 2, 2020 ehudreiter1 Comment

I really liked Grishman’s recent paper on 25 years of research in information extraction, and summarise a few of the key insights here, about relative progress in different areas of NLP, reluctance of researchers to use complex evaluation techniques, and corpus creation vs rule-writing.

Uncategorized

Election results: Lessons from a real-world NLG system

Dec 23, 2019Dec 23, 2019 ehudreiter2 Comments

The BBC used Arria NLG to generate stories about the recent UK election. In this application, texts communicated a meaning, there was no corpus, accuracy was paramount, and domain experts wanted to control the system. Most applied NLG systems I have worked on have had similar constraints.

Uncategorized

Apologies to my students for limited feedback!

Dec 11, 2019 ehudreiterLeave a comment

I’ve spent much of the past few weeks marking, but nonetheless was unable to give my students detailed feedback and critiques, My apologies!

Posts navigation

Older Posts
Newer posts
  • LinkedIn
  • Twitter

Top Posts & Pages

  • "Will I Pass my PhD Viva"
  • What is NLG
  • Best Papers I Read in 2020
  • Comparing Human Evaluations
  • How do I Build an NLG System: Tools?
  • How do I Learn about NLG?
  • Blog Index
  • Hallucination in Neural NLG
  • Small differences in BLEU are meaningless
  • Language Grounding and Context
Blog at WordPress.com.
Ehud Reiter's Blog
Blog at WordPress.com.
Cancel