When we use NLG in real-world applications, we need to ensure that the system will behave acceptably in unusual cases, and can be updated as its usage changes. This chapter looks at several issues related to this topic:
- Safety: Can we ensure that NLG systems do not harm users?
- Testing: What are the special challenges of software quality assurance for NLG systems?
- Maintenance: How do NLG systems need to change as the world and user requirements evolve?
There is an extensive general literature on all of these topics, the book focuses on specific issues that arise in NLG contexts.
Resources: Selected Blogs
- Could NLG systems injure or even kill people?
- How do I Build an NLG System: Testing and Quality Assurance
- Maintaining NLG Systems
Resources: Surveys and books
- International Scientific Report on the Safety of Advanced AI (final version, book just references interim version)
- Understanding and Mitigating Risks of Generative AI in Financial Services (shows that safety issues are domain-dependent, generic safety solutions may not work in specific domains)
Ehud Reiter's Blog 