Applied artificial intelligence technology for using natural language processing and concept expression templates to train a natural language generation system

An apparatus for natural language processing, the apparatus comprising:

• a processor configured to (1) perform natural language processing (NLP) on training data to detect a plurality of linguistic features in the training data, wherein the training data comprises a plurality of words arranged as a sentence in a natural language, and wherein the detected linguistics feature include a concept expression template that models how the training data describes a defined concept, and (2) generate a specification data structure based on the detected linguistic features, the specification data structure arranged for training a natural language generation (NLG) system to produce natural language output that stylistically resembles the training data;
• wherein the processor is further configured to, as part of the NLP to detect the linguistic features, (1) compare the sentence with a plurality of anchor words to determine whether any of the anchor words are present in the sentence, each anchor word having an association with a concept, and (2) in response to a determination that an anchor word is present in the sentence, analyze the sentence to extract an expression template for the concept associated with the present anchor word;
• and
• wherein the processor is further configured to, as part of the sentence analysis to extract the concept expression template, (1) parse the sentence to generate a parse tree structure for the sentence, (2) identify entities in the parse tree structure, (3) prune the parse tree structure by removing clauses and phrases from the parse tree structure that do not contain identified entities, (4) collapse branches of the pruned parse tree structure based on which of the branches contain identified entities, and (5) parameterize the collapsed and pruned parse tree structure to create the concept expression template, wherein the concept expression template comprises enumerated variables in place of identified entities.

A computer program product for natural language processing, the computer program product comprising:

• a plurality of processor-executable instructions that are resident on a non-transitory computer readable storage medium, wherein the instructions are configured, upon execution by a processor, to cause the processor to (1) perform natural language processing (NLP) on training data to detect a plurality of linguistic features in the training data, wherein the training data comprises a plurality of words arranged as a sentence in a natural language, and wherein the detected linguistic features include a concept expression template that models how the training data describes a defined concept, and (2) generate a specification data structure based on the detected linguistic features, the specification data structure arranged for training a natural language generation (NLG) system to produce natural language output that stylistically resembles the training data;
• wherein the instructions for performing the NLP to detect the linguistic features are configured, upon execution by the processor, to cause the processor to (1) compare the sentence with a plurality of anchor words to determine whether any of the anchor words are present in the sentence, each anchor word having an association with a concept, and (2) in response to a determination that an anchor word is present in the sentence, analyze the sentence to extract an expression template for the concept associated with the present anchor word;
• and
• wherein the instructions for analyzing the sentence to extract the concept expression template are configured, upon execution by the processor, to cause the processor to (1) parse the sentence to generate a parse tree structure for the sentence, (2) identify entities in the parse tree structure, (3) prune the parse tree structure by removing clauses and phrases from the parse tree structure that do not contain identified entities, (4) collapse branches of the pruned parse tree structure based on which of the branches contain identified entities, and (5) parameterize the collapsed and pruned parse tree structure to create the concept expression template, wherein the concept expression template comprises enumerated variables in place of identified entities.