# US 11,443,117 B2 · Contextual question answering using human chat logs > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,443,117 B2 - Document ID: US11443117B2 - Status: Granted - Kind: B2 - Application number: 16/688,843 - Filed: 2019-11-19 - Published: 2021-05-20 - Granted: 2022-09-13 - Earliest claimed: 2019-11-19 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Yang Yu; Ming Tan; Shasha Lin; Saloni Potdar - Web page: https://patentgrep.com/patent/US11443117B2 ## CPC - G06F 40/30: Semantic analysis - G06F 40/109: Font handling; Temporal or kinetic typography - G06F 40/284: Lexical analysis, e.g. tokenisation or collocates - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system includes a memory having instructions therein and at least one processor configured to execute the instructions to: receive a natural language question; determine, from a chat log comprising a plurality of chat session logs, a set of chat session logs most relevant to the natural language question; determine a respective plurality of non-overlapping text spans most relevant to the natural language question within each of a respective plurality of conceptual pseudo-documents; determine a conceptual pseudo-document most relevant to the natural language question; extract a question-answer pair most relevant to the natural language question from the most relevant pseudo-document; and convey the most relevant question-answer pair to a user. Each one of the conceptual pseudo-documents corresponds to a respective one of the most relevant chat session logs. ## Claims ### Claim 1 A method for answering a natural language question, the method comprising: receiving the natural language question at a processing device; determining, using a support vector machine, a set of most relevant chat session logs to the natural language question from a chat log comprising a plurality of chat session logs; concatenating question and answer pairs in each of the set of most relevant chat session logs into a respective plurality of conceptual pseudo-documents; determining, using one or more machine reading comprehension (MRC) models, a probability for a start word and an end word for each of a plurality of non-overlapping text spans comprising one or more of the concatenated question and answer pairs for each of the respective plurality of conceptual pseudo-documents; determining, using the one or more MRC models, a most relevant non-overlapping text span to the natural language question from the plurality of non-overlapping text spans of the respective plurality of conceptual pseudo-documents; extracting a most relevant question-answer pair to the natural language question from the most relevant non-overlapping text span based upon the determined probability; and conveying the most relevant question-answer pair to a user device, wherein each one of the conceptual pseudo-documents corresponds to a respective one of the most relevant chat session logs in the set of most relevant chat session logs. ### Claim 2 (depends on 1) The method of claim 1, wherein determining the most relevant non-overlapping text span comprises generating respective fixed-length vector representations of the respective conceptual pseudo-documents. ### Claim 3 (depends on 2) The method of claim 2, wherein generating the respective fixed-length vector representations of the respective conceptual pseudo-documents comprises pooling learned representations of tokens in respective text spans in each of the respective conceptual pseudo-documents. ### Claim 4 (depends on 1) The method of claim 1, wherein extracting the most relevant question-answer pair from the most relevant non-overlapping text span comprises determining that the most relevant question-answer pair includes at least some of the most relevant non-overlapping text span. ### Claim 5 (depends on 4) The method of claim 4, wherein conveying the most relevant question-answer pair to the user comprises emphasizing, in an answer portion of the most relevant question-answer pair, at least a portion of the most relevant non-overlapping text span. ### Claim 6 (depends on 5) The method of claim 5, wherein emphasizing, in an answer portion of the most relevant question-answer pair, at least a portion of the most relevant non-overlapping text span comprises emphasizing at least a portion of the most relevant non-overlapping text span in at least one manner selected from the group consisting of highlighting at least a portion of the most relevant non-overlapping text span, emboldening at least a portion of the most relevant non-overlapping text span, underlining at least a portion of the most relevant non-overlapping text span, and italicizing at least a portion of the most relevant non-overlapping text span. ### Claim 7 (depends on 3) The method of claim 3, wherein extracting the most relevant question-answer pair from the most relevant non-overlapping text span comprises determining that the most relevant question-answer pair includes at least some of the most relevant non-overlapping text span, and wherein conveying the most relevant question-answer pair to the user comprises emphasizing, in an answer portion of the most relevant question-answer pair, at least a portion of the most relevant non-overlapping text span in at least one manner selected from the group consisting of highlighting at least a portion of the most relevant non-overlapping text span, emboldening at least a portion of the most relevant non-overlapping text span, underlining at least a portion of the most relevant non-overlapping text span, and italicizing at least a portion of the most relevant non-overlapping text span. ### Claim 8 A system for answering a natural language question, the system comprising: a memory having instructions therein; and at least one processor in communication with the memory, wherein the at least one processor is configured to execute the instructions to: receive the natural language question; determine, using a support vector machine, a set of most relevant chat session logs to the natural language question from a chat log comprising a plurality of chat session logs; concatenate question and answer pairs in each of the set of most relevant chat session logs into a respective plurality of conceptual pseudo-documents; determine, using the one or more machine reading comprehension (MRC) models, a probability for a start word and an end word for each of a plurality of non-overlapping text spans comprising one or more of the concatenated question and answer pairs for each of the respective plurality of conceptual pseudo-documents; determine a respective plurality of non-overlapping text spans most relevant to the natural language question within each of a respective plurality of conceptual pseudo-documents; determine, using the one or more MRC models, a most relevant non-overlapping text span to the natural language question from the plurality of non-overlapping text spans of the respective plurality of conceptual pseudo-documents; extract a most relevant question-answer pair to the natural language question from the most relevant pseudo-document based upon the determined probability; and convey the most relevant question-answer pair to a user device, wherein each one of the conceptual pseudo-documents corresponds to a respective one of the most relevant chat session logs in the set of most relevant chat session logs. ### Claim 9 (depends on 8) The system of claim 8, wherein the at least one processor is further configured to execute the instructions to: generate respective fixed-length vector representations of the respective conceptual pseudo-documents; and determine the most relevant pseudo-document based on the respective fixed-length vector representations. ### Claim 10 (depends on 9) The system of claim 9, wherein the at least one processor is further configured to execute the instructions to: pool learned representations of tokens in respective text spans in each of the respective conceptual pseudo-documents; and generate the respective fixed-length vector representations based on the learned representations of tokens. ### Claim 11 (depends on 8) The system of claim 8, wherein the at least one processor is further configured to execute the instructions to: determine a most relevant text span within the most relevant pseudo-document; and extract the most relevant question-answer pair from the most relevant pseudo-document based on the most relevant text span. ### Claim 12 (depends on 11) The system of claim 11, wherein the at least one processor is further configured to execute the instructions to emphasize, in an answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span. ### Claim 13 (depends on 12) The system of claim 12, wherein the at least one processor is further configured to execute the instructions to at least one of: highlight, in the answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span; embolden, in the answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span; underline, in the answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span; and italicize, in the answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span. ### Claim 14 (depends on 10) The system of claim 10, wherein the at least one processor is further configured to execute the instructions to: determine a most relevant text span within the most relevant pseudo-document; extract the most relevant question-answer pair from the most relevant pseudo-document based on the most relevant text span; and wherein the at least one processor is further configured to execute the instructions to at least one of: highlight, in an answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span; embolden, in an answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span; underline, in an answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span; and italicize, in an answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span. ### Claim 15 A computer program product for answering a natural language question, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by at least one processor of a system to cause the system to: receive the natural language question; determine, using a support vector machine, a set of most relevant chat session logs to the natural language question from a chat log comprising a plurality of chat session logs; concatenate question and answer pairs in each of the set of most relevant chat session logs into a respective plurality of conceptual pseudo-documents; determine, using the one or more machine reading comprehension (MRC) models, a probability for a start word and an end word for each of a plurality of non-overlapping text spans comprising one or more of the concatenated question and answer pairs for each of the respective plurality of conceptual pseudo-documents; determine a respective plurality of non-overlapping text spans most relevant to the natural language question within each of a respective plurality of conceptual pseudo-documents; determine, using the one or more MRC models, a most relevant non-overlapping text span to the natural language question from the plurality of non-overlapping text spans of the respective plurality of conceptual pseudo-documents; extract a most relevant question-answer pair to the natural language question from the most relevant pseudo-document based upon the determined probability; and convey the most relevant question-answer pair to a user device, wherein each one of the conceptual pseudo-documents corresponds to a respective one of the most relevant chat session logs in the set of most relevant chat session logs. ### Claim 16 (depends on 15) The computer program product of claim 15, wherein the program instructions are further executable by the at least one processor to cause the at least one processor to: generate respective fixed-length vector representations of the respective conceptual pseudo-documents; and determine the most relevant pseudo-document based on the respective fixed-length vector representations. ### Claim 17 (depends on 16) The computer program product of claim 16, wherein the program instructions are further executable by the at least one processor to cause the at least one processor to: pool learned representations of tokens in respective text spans in each of the respective conceptual pseudo-documents; and generate the respective fixed-length vector representations based on the learned representations of tokens. ### Claim 18 (depends on 15) The computer program product of claim 15, wherein the program instructions are further executable by the at least one processor to cause the at least one processor to: determine a most relevant text span within the most relevant pseudo-document; and extract the most relevant question-answer pair from the most relevant pseudo-document based on the most relevant text span. ### Claim 19 (depends on 18) The computer program product of claim 18, wherein the program instructions are further executable by the at least one processor to cause the at least one processor to: emphasize, in an answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span. ### Claim 20 (depends on 19) The computer program product of claim 19, wherein the program instructions are further executable by the at least one processor to cause the at least one processor to: highlight, in the answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span; embolden, in the answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span; underline, in the answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span; and italicize, in the answer portion of the most relevant question-answer pair, at least a portion of the most relevant text span.