Skip to content

Ehud Reiter's Blog

Ehud's thoughts about Natural Language Generation. Also see my book on NLG.

  • Home
  • Blog Index
  • About
  • What is NLG
  • Publications
  • Resources
  • University
  • Book
  • Contact

Category: evaluation

evaluation

Evaluating Accuracy

Nov 17, 2020Dec 3, 2020 ehudreiter11 Comments

Craig Thomson and I will present a paper at INLG on a methodology for evaluating the accuracy of generated texts, based on asking human annotators to mark up factual errors in a text. This is not cheap, but I think it is the most robust and reliable approach to measuring accuracy.

Posts navigation

Newer posts
  • LinkedIn
  • Twitter

News: I am likely to retire in summer 2026. Looking for interesting things to do afterwards.

Top Posts & Pages

  • Resources and Links
  • Retirement Plans: Travel and some academics
  • Do a sanity check on your experiments
  • How do I Learn about NLG?
  • Contact
  • What LLMs cannot do
  • Exercise: Find Problems in an Evaluation
  • Lets use error annotations to evaluate systems!
  • Learning does not require evaluation metrics
  • Blog Index
Blog at WordPress.com.
  • Subscribe Subscribed
    • Ehud Reiter's Blog
    • Join 102 other subscribers.
    • Already have a WordPress.com account? Log in now.
    • Ehud Reiter's Blog
    • Subscribe Subscribed
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar