Natural-language generation

Natural-language generation (NLG) is a software process that transforms structured data into natural language. It can be used to produce long form content for organizations to automate custom reports, as well as produce custom content for a web or mobile application. It can also be used to generate short blurbs of text in interactive conversations (a chatbot) which might even be read out by a text-to-speech system.

Automated NLG can be compared to the process humans use when they turn ideas into writing or speech. Psycholinguists prefer the term language production for this process, which can also be described in mathematical terms, or modeled in a computer for psychological research. NLG systems can also be compared to translators of artificial computer languages, such as decompilers or transpilers, which also produce human-readable code generated from an intermediate representation. Human languages tend to be considerably more complex and allow for much more ambiguity and variety of expression than programming languages, which makes NLG more challenging.

NLG may be viewed as the opposite of natural-language understanding (NLU): whereas in natural-language understanding, the system needs to disambiguate the input sentence to produce the machine representation language, in NLG the system needs to make decisions about how to put a concept into words. The practical considerations in building NLU vs. NLG systems are not symmetrical. NLU needs to deal with ambiguous or erroneous user input, whereas the ideas the system wants to express through NLG are generally known precisely. NLG needs to choose a specific, self-consistent textual representation from many potential representations, whereas NLU generally tries to produce a single, normalized representation of the idea expressed.[1]

NLG has existed since ELIZA was developed in the mid 1960s, but commercial NLG technology has only recently become widely available. NLG techniques range from simple template-based systems like a mail merge that generates form letters, to systems that have a complex understanding of human grammar. NLG can also be accomplished by training a statistical model using machine learning, typically on a large corpus of human-written texts.[2]

Example

The Pollen Forecast for Scotland system[3] is a simple example of a simple NLG system that could essentially be a template. This system takes as input six numbers, which give predicted pollen levels in different parts of Scotland. From these numbers, the system generates a short textual summary of pollen levels as its output.

For example, using the historical data for July 1, 2005, the software produces:

Grass pollen levels for Friday have increased from the moderate to high levels of yesterday with values of around 6 to 7 across most parts of the country. However, in Northern areas, pollen levels will be moderate with values of 4.

In contrast, the actual forecast (written by a human meteorologist) from this data was:

Pollen counts are expected to remain high at level 6 over most of Scotland, and even level 7 in the south east. The only relief is in the Northern Isles and far northeast of mainland Scotland with medium levels of pollen count.

Comparing these two illustrates some of the choices that NLG systems must make; these are further discussed below.

Stages

The process to generate text can be as simple as keeping a list of canned text that is copied and pasted, possibly linked with some glue text. The results may be satisfactory in simple domains such as horoscope machines or generators of personalised business letters. However, a sophisticated NLG system needs to include stages of planning and merging of information to enable the generation of text that looks natural and does not become repetitive. The typical stages of natural-language generation, as proposed by Dale and Reiter,[1] are:

Content determination: Deciding what information to mention in the text. For instance, in the pollen example above, deciding whether to explicitly mention that pollen level is 7 in the south east.

Document structuring: Overall organisation of the information to convey. For example, deciding to describe the areas with high pollen levels first, instead of the areas with low pollen levels.

Aggregation: Merging of similar sentences to improve readability and naturalness. For instance, merging the two following sentences:

  • Grass pollen levels for Friday have increased from the moderate to high levels of yesterday and
  • Grass pollen levels will be around 6 to 7 across most parts of the country

into the following single sentence:

  • Grass pollen levels for Friday have increased from the moderate to high levels of yesterday with values of around 6 to 7 across most parts of the country.

Lexical choice: Putting words to the concepts. For example, deciding whether medium or moderate should be used when describing a pollen level of 4.

Referring expression generation: Creating referring expressions that identify objects and regions. For example, deciding to use in the Northern Isles and far northeast of mainland Scotland to refer to a certain region in Scotland. This task also includes making decisions about pronouns and other types of anaphora.

Realization: Creating the actual text, which should be correct according to the rules of syntax, morphology, and orthography. For example, using will be for the future tense of to be.

An alternative approach to NLG is to use "end-to-end" machine learning to build a system, without having separate stages as above.[4] In other words, we build an NLG system by training a machine learning algorithm (often an LSTM) on a large data set of input data and corresponding (human-written) output texts. The end-to-end approach has perhaps been most successful in image captioning,[5] that is automatically generating a textual caption for an image.

Applications

The popular media has paid the most attention to NLG systems which generate jokes (see computational humor), but from a commercial perspective, the most successful NLG applications have been data-to-text systems which generate textual summaries of databases and data sets; these systems usually perform data analysis as well as text generation. Research has shown that textual summaries can be more effective than graphs and other visuals for decision support,[6][7][8] and that computer-generated texts can be superior (from the reader's perspective) to human-written texts.[9]

The first commercial data-to-text systems produced weather forecasts from weather data. The earliest such system to be deployed was FoG,[10] which was used by Environment Canada to generate weather forecasts in French and English in the early 1990s. The success of FoG triggered other work, both research and commercial. Recent applications include the UK Met Office's text-enhanced forecast.[11]

Currently there is considerable commercial interest in using NLG to summarise financial and business data. Indeed, Gartner has said that NLG will become a standard feature of 90% of modern BI and analytics platforms.[12] NLG is also being used commercially in automated journalism, chatbots, generating product descriptions for e-commerce sites, summarising medical records,[13][14] and enhancing accessibility (for example by describing graphs and data sets to blind people[15]).

An example of an interactive use of NLG is the WYSIWYM framework. It stands for What you see is what you meant and allows users to see and manipulate the continuously rendered view (NLG output) of an underlying formal language document (NLG input), thereby editing the formal language without learning it.

Content generation systems assist human writers and makes writing process more efficient and effective. A content generation tool based on web mining using search engines APIs has been built.[16] The tool imitates the cut-and-paste writing scenario where a writer forms its content from various search results. Relevance verification is essential to filter out irrelevant search results; it is based on matching the parse tree of a query with the parse trees of candidate answers.[17] In an alternative approach, a high-level structure of human-authored text is used to automatically build a template for a new topic for automatically written Wikipedia article.[18]

Several companies have been started since 2009 which build systems that transform data into narrative using NLG and AI techniques. These include Narrative Science,[19] Phrasetech,[20] Arria NLG, Automated Insights, Adzis NLG,[21] Retresco, Narrativa,[22] Visual NLG,[23] Yseop and United Robots.[24] Open-source NLG solutions exist as well, for instance RosaeNLG,[25] SimpleNLG [26] and The RiTa Toolkit.[27]

Evaluation

As in other scientific fields, NLG researchers need to test how well their systems, modules, and algorithms work. This is called evaluation. There are three basic techniques for evaluating NLG systems:

  • Task-based (extrinsic) evaluation: give the generated text to a person, and assess how well it helps him perform a task (or otherwise achieves its communicative goal). For example, a system which generates summaries of medical data can be evaluated by giving these summaries to doctors, and assessing whether the summaries helps doctors make better decisions.[14]
  • Human ratings: give the generated text to a person, and ask him or her to rate the quality and usefulness of the text.
  • Metrics: compare generated texts to texts written by people from the same input data, using an automatic metric such as BLEU.

An ultimate goal is how useful NLG systems are at helping people, which is the first of the above techniques. However, task-based evaluations are time-consuming and expensive, and can be difficult to carry out (especially if they require subjects with specialised expertise, such as doctors). Hence (as in other areas of NLP) task-based evaluations are the exception, not the norm.

Recently researchers are assessing how well human-ratings and metrics correlate with (predict) task-based evaluations. Work is being conducted in the context of Generation Challenges[28] shared-task events. Initial results suggest that human ratings are much better than metrics in this regard. In other words, human ratings usually do predict task-effectiveness at least to some degree (although there are exceptions), while ratings produced by metrics often do not predict task-effectiveness well. These results are preliminary. In any case, human ratings are the most popular evaluation technique in NLG; this is contrast to machine translation, where metrics are widely used.

gollark: JS will actually let you get away with this, too:```javascriptif (window.containsBees) console.log("bee contamination detected") console.log("initiating quarantine")```
gollark: They are optional.
gollark: Yes, this is much better:```javascriptfunction produceApioform() {let apioform = "apio"apioform += Math.random() > 0.5 ? "hazard" : "form"return apioform}```
gollark: I did make a thing for Fibonacci indentation.
gollark: 2 spaces <:bees:724389994663247974><:dodecahedron:724893894822854697>

See also

References
  1. Dale, Robert; Reiter, Ehud (2000). Building natural language generation systems. Cambridge, U.K.: Cambridge University Press. ISBN 978-0-521-02451-8.
  2. Perera R, Nand P (2017). "Recent Advances in Natural Language Generation: A Survey and Classification of the Empirical Literature". Computing and Informatics. 36 (1): 1–32. doi:10.4149/cai_2017_1_1.
  3. R Turner, S Sripada, E Reiter, I Davy (2006). Generating Spatio-Temporal Descriptions in Pollen Forecasts. Proceedings of EACL06
  4. "E2E NLG Challenge".
  5. "DataLabCup: Image Caption".
  6. Law A, Freer Y, Hunter J, Logie R, McIntosh N, Quinn J (2005). "A Comparison of Graphical and Textual Presentations of Time Series Data to Support Medical Decision Making in the Neonatal Intensive Care Unit". Journal of Clinical Monitoring and Computing. 19 (3): 183–94. doi:10.1007/s10877-005-0879-3. PMID 16244840.
  7. Gkatzia D, Lemon O, Reiser V (2017). "Data-to-Text Generation Improves Decision-Making Under Uncertainty" (PDF). IEEE Computational Intelligence Magazine. 12 (3): 10–17. doi:10.1109/MCI.2017.2708998.
  8. "Text or Graphics?". 2016-12-26.
  9. Reiter E, Sripada S, Hunter J, Yu J, Davy I (2005). "Choosing Words in Computer-Generated Weather Forecasts". Artificial Intelligence. 167 (1–2): 137–69. doi:10.1016/j.artint.2005.06.006.
  10. Goldberg E, Driedger N, Kittredge R (1994). "Using Natural-Language Processing to Produce Weather Forecasts". IEEE Expert. 9 (2): 45–53. doi:10.1109/64.294135.
  11. S Sripada, N Burnett, R Turner, J Mastin, D Evans(2014). Generating A Case Study: NLG meeting Weather Industry Demand for Quality and Quantity of Textual Weather Forecasts. Proceedings of INLG 2014
  12. "Neural Networks and Modern BI Platforms Will Evolve Data and Analytics".
  13. Harris MD (2008). "Building a Large-Scale Commercial NLG System for an EMR" (PDF). Proceedings of the Fifth International Natural Language Generation Conference. pp. 157–60.
  14. Portet F, Reiter E, Gatt A, Hunter J, Sripada S, Freer Y, Sykes C (2009). "Automatic Generation of Textual Summaries from Neonatal Intensive Care Data" (PDF). Artificial Intelligence. 173 (7–8): 789–816. doi:10.1016/j.artint.2008.12.002.
  15. http://www.inf.udec.cl/~leo/iGraph.html
  16. Galitsky, Boris (2013). A Web Mining Tool for Assistance with Creative Writing. Advances in Information Retrieval. Lecture Notes in Computer Science. Lecture Notes in Computer Science. 7814. pp. 828–831. doi:10.1007/978-3-642-36973-5_95. ISBN 978-3-642-36972-8.
  17. Galitsky B, de la Rosa JL, Dobrocsi G (2012). "Inferring the semantic properties of sentences by mining syntactic parse trees". Data & Knowledge Engineering. 81-82: 21–45. doi:10.1016/j.datak.2012.07.003.
  18. Sauper, Christina & Barzilay, Regina (2009). "Automatically Generating Wikipedia Articles: A Structure-Aware Approach". Proceedings of ACL.
  28. Generation Challenges 2009

Further reading
  • ACL Special Interest Group on Generation (SIGGEN)
  • SIGGEN part of ACL Anthology (contains NLG research papers)
  • ACL NLG Portal (contains list of NLG resources)
  • Bateman and Zock's 'almost complete' list of NLG systems now maintained as a Wiki with a variety of visualisations and overview tables available on demand
  • Ehud Reiter's blog on Natural Language Generation
  • RosaeNLG RosaeNLG is an open-source (MIT) NLG library written in JavaScript, based on the Pug template engine (supports any language and is currently shipped with resources for English, French, German and Italian)
  • KPML — general-purpose natural language generation system with graphical user interface for development and grammars of varying sizes for Chinese, Czech, Dutch, English, French, German, Greek, Japanese, Portuguese, Russian and Spanish
  • SimpleNLG — Open source Java library to assist in NLG (English only)
  • SimpleNLG-EnFr — Open source Java library adaption of SimpleNLG which adds French support.
  • Cerebellum — Character-based Language Model generator for English, Georgian, Hungarian, Japanese, Russian, Turkish, Slovak, Czech and Latin
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.