# US 2020/0394261 A1 · Natural Language Processing and Candidate Response Identification > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0394261 A1 - Document ID: US20200394261A1 - Status: Published application - Kind: A1 - Application number: 16/439,216 - Filed: 2019-06-12 - Published: 2020-12-17 - Earliest claimed: 2019-06-12 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Stephen A. Boxwell; Keith G. Frost; Kyle M. Brake; Stanley J. Vernier - Web page: https://patentgrep.com/patent/US20200394261A1 ## CPC - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06F 16/90332: {Natural language query formulation or dialogue systems} - G06F 40/205: Parsing - G06F 40/30: Semantic analysis - G06F 40/56: Natural language generation - H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Embodiments relate to an intelligent computer platform to identify and evaluate candidate passages response data in natural language form. Natural language processing is applied to analyze a passage against one or more input tokens to identify matching content. A structure representing the analyzed passage is populated with of matching input and passage tokens. A first count of matching token entries and a second count of evaluated token entries are determined and qualified by closeness criteria. An alignment of the passage to a candidate question is calculated, including assessing a ratio of the first and second counts as a confidence value. Matching passage data is return from the passage with the confidence value. ## Claims ### Claim 1 A computer implemented method comprising: applying natural language processing to analyze a passage in natural language form, including subjecting the passage to analysis against one or more input tokens, and identifying content within the passage matching the one or more input tokens; generating a structure representing a mapping of matching content between the analyzed passage and a focus of the one or more input tokens, an entry in the structure including the focus and a candidate answer from the passage; identifying alignment of the one or more input tokens and matching one or more passage tokens within the generated structure; leveraging the generated structure and the identified alignment and iteratively associating a mapping of one or more input tokens in the passage as an index representation, including determining a first count of matching token entries qualified by a closeness criteria and a second count of evaluated token entries qualified by the closeness criteria; calculating an alignment of the passage to a candidate question represented by the one or more input tokens, including assessing a ratio as a confidence value; and data from the passage qualified by the one or more input tokens is returned with the confidence value. ### Claim 2 (depends on 1) The method of claim 1, wherein the ratio reflects an alignment of the one or more input tokens and the passage. ### Claim 3 (depends on 1) The method of claim 1, wherein applying NLP includes identifying the closeness criteria, and applying the closeness criteria to the passage for identifying matching content. ### Claim 4 (depends on 3) The method of claim 3, wherein the closeness criteria is an n-gram, and the confidence value is a ratio of a quantity of n-grams found in the passage to a quantity of evaluated n-grams. ### Claim 5 (depends on 1) The method of claim 1, wherein matching token entries is based on a match criteria selected from the group consisting of: exact, synonym, lemma, and any method of drawing equivalence between two natural language tokens. ### Claim 6 (depends on 3) The method of claim 3, further comprising representing the identifying matching input and passage tokens in compact form, and performing one or more compact operations on the compact form, including performing a first compact operation to expand one or more compact entities set in the compact form and a second compact operation to discount matching when only a single token is found in a multiple token search. ### Claim 7 A computer program product for identifying and evaluating candidate passages responses, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by a processor to: apply natural language processing (NLP) to analyze a passage in natural language form, including subject the passage to analysis against one or more input tokens, and identify content within the passage matching the one or more input tokens; generate a structure representing a mapping of matching content between the analyzed passage and a focus of the one or more input tokens, an entry in the structure including the focus and a candidate answer from the passage; identify alignment of the one or more input tokens and matching one or more passage tokens within the generated structure; leverage the generated structure and the identified alignment and iteratively associate a mapping of one or more input tokens in the passage as an index representation, including determine a first count of matching token entries qualified by a closeness criteria and a second count of evaluated token entries qualified by the closeness criteria; calculate an alignment of the passage to a candidate question represented by the one or more input tokens, including the director to assess a ratio as a confidence value; and data from the passage qualified by the one or more input tokens is returned with the confidence value. ### Claim 8 (depends on 7) The computer program product of claim 7, wherein the ratio reflects an alignment of the one or more input tokens and the passage. ### Claim 9 (depends on 7) The computer program product of claim 7, further comprising program code to identify the closeness criteria, and apply the identified closeness criteria to the passage to identify matching content. ### Claim 10 (depends on 9) The computer program product of claim 9, wherein the closeness criteria is an n-gram, and the confidence value is a ratio of a quantity of n-grams found in the passage to a quantity of evaluated n-grams. ### Claim 11 (depends on 7) The computer program product of claim 7, wherein the program code to match token entries is based on a match criteria selected from the group consisting of: exact, synonym, lemma, and any method of drawing equivalence between two natural language tokens. ### Claim 12 (depends on 9) The computer program product of claim 9, further comprising program code to represent the identifying matching input and passage tokens in compact form, and perform one or more compact operations on the compact form, including perform a first compact operation to expand one or more compact entities set in the compact form and a second compact operation to discount matching when only a single token is found in a multiple token search. ### Claim 13 A computer system comprising: a processing unit operatively coupled to memory; an artificial intelligence (AI) platform in communication with the processing unit, the AI platform to identify and evaluate candidate passage responses, including a natural language (NL) manager to apply natural language processing (NLP) to analyze a passage in natural language form, including the NL manager to subject the passage to analysis against one or more input tokens, and identify content within the passage matching the one or more input tokens; a structure manager operatively coupled to the NL manager, the structure manager to generate a structure representing a mapping of matching content between the analyzed passage and a focus of the one or more input tokens, an entry in the structure including the focus and a candidate answer from the passage; a token manager operatively coupled to the structure manager, the token manager to identify alignment of the one or more input tokens and matching one or more passage tokens within the generated structure; the token manager to leverage the generated structure and the identified alignment and iteratively associate a mapping of one or more input tokens in the passage as an index representation, including determine a first count of matching token entries qualified by a closeness criteria and a second count of evaluated token entries qualified by the closeness criteria; a director to calculate an alignment of the passage to a candidate question represented by the one or more input tokens, including the director to assess a ratio as a confidence value; and data from the passage qualified by the one or more input tokens is returned with the confidence value. ### Claim 14 (depends on 13) The system of claim 13, wherein the ratio reflects an alignment of the one or more input tokens and the passage. ### Claim 15 (depends on 13) The system of claim 13, further comprising the NL manager to identify the closeness criteria, and apply the identified closeness criteria to the passage to identify matching content. ### Claim 16 (depends on 15) The system of claim 15, wherein the closeness criteria is an n-gram, and the confidence value is a ratio of a quantity of n-grams found in the passage to a quantity of evaluated n-grams. ### Claim 17 (depends on 13) The system of claim 13, wherein matching token entries is based on a match criteria selected from the group consisting of: exact, synonym, lemma, and any method of drawing equivalence between two natural language tokens. ### Claim 18 (depends on 15) The system of claim 15, further comprising the token manager to represent the identifying matching input and passage tokens in compact form, and the format manager to perform one or more compact operations on the compact form, including perform a first compact operation to expand one or more compact entities set in the compact form and a second compact operation to discount matching when only a single token is found in a multiple token search. ### Claim 19 A computer implemented method comprising: analyzing a passage in natural language form, including identifying one or more passage tokens matching one or more input tokens; representing a mapping of matching content between the analyzed passage and a focus of the one or more input tokens, an entry in the mapping including the focus and a candidate from the passage; identifying alignment of the one or more input tokens and matching the one or more passage tokens within a generated structure; iteratively associating a mapping of the one or more input tokens in the passage as an index representation, including determining a first count of matching token entries qualified by a closeness criteria and a second count of evaluated token entries qualified by the closeness criteria; calculating a ratio of matching token entries qualified by the closeness criteria and the evaluated token entries qualified by the closeness criteria; and data from the passage qualified by the one or more input tokens is returned with the calculated ratio. ### Claim 20 (depends on 19) The method of claim 19, further comprising calculating two or more ratios, associating each calculated ratio with a corresponding answer, and returning the answer with a highest calculated ratio. ### Claim 21 (depends on 19) The method of claim 19, further comprising representing the identifying matching input and passage tokens in compact form, and performing one or more compact operations on the compact form, including performing a first compact operation to expand one or more compact entities set in the compact form and a second compact operation to discount matching when only a single token is found in a multiple token search. ### Claim 22 (depends on 19) The method of claim 19, further comprising identifying the closeness criteria, and applying the closeness criteria to the passage for identifying matching content. ### Claim 23 (depends on 22) The method of claim 22, wherein the closeness criteria is an n-gram, and a confidence value is a ratio of a quantity of n-grams found in the passage to a quantity of evaluated n-grams. ### Claim 24 A computer program product for identifying and evaluating candidate passages responses, the computer program product comprising a computer readable storage medium having program code embodied therewith, the program code executable by a processor to: analyze a passage in natural language form, including identifying one or more passage tokens matching one or more input tokens; represent a mapping of matching content between the analyzed passage and a focus of the one or more input tokens, an entry in the mapping including the focus and a candidate from the passage; identify alignment of the one or more input tokens and matching the one or more passage tokens within a generated structure; iteratively associate a mapping of one or more input tokens in the passage as an index representation, including determining a first count of matching token entries qualified by a closeness criteria and a second count of evaluated token entries qualified by the closeness criteria; calculate a ratio of matching token entries qualified by the closeness criteria and the evaluated token entries qualified by the closeness criteria; and data from the passage qualified by the one or more input tokens is returned with the calculated ratio.