# US 12,014,836 B2 · Stream integrity for artificial intelligence conversations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,014,836 B2 - Document ID: US12014836B2 - Status: Granted - Kind: B2 - Application number: 17/117,461 - Filed: 2020-12-10 - Published: 2022-06-16 - Granted: 2024-06-18 - Earliest claimed: 2020-12-10 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Paul R. Bastide; Matthew E. Broomhall; Robert E. Loredo - Web page: https://patentgrep.com/patent/US12014836B2 ## CPC - G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring - G06F 18/24: Classification techniques - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 5/04: Inference or reasoning models - G16H 10/20: for electronic clinical trials or questionnaires - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS ## Abstract A method, system, and computer program product to prevent misinterpretation of communications via artificial intelligence assistants. The method includes monitoring an artificial intelligence enabled communication. The method also includes analyzing messages, participants, and actions within the communication. The method also includes identifying critical data within the communication. The method also includes determining critical sequence data, where the critical sequence data is a sequence of conversation elements that contains the critical data. The method also includes predicting an uncertainty level for the critical sequence data, where the uncertainty level indicates an amount of uncertainty that the artificial intelligence correctly interprets the critical sequence data. The method also includes determining that the uncertainty level for the critical sequence data is above a threshold uncertainty value, resulting in critical sequence data that is subject to misinterpretation. The method also includes resolving any uncertainties for the critical sequence data. ## Claims ### Claim 1 A computer-implemented method comprising: monitoring an artificial intelligence enabled communication between an artificial intelligence assistant and a user; analyzing messages, participants, and actions within the communication; identifying critical data within the communication, wherein the critical data is data within conversation elements of the communication that is important for performing the actions within the communication; determining critical sequence data, wherein the critical sequence data is a sequence of the conversation elements that is within the communication and that contains the critical data; predicting an uncertainty level for the critical sequence data, wherein the uncertainty level indicates an amount of uncertainty that the artificial intelligence assistant correctly interprets the critical sequence data; determining that the uncertainty level for the critical sequence data is above a threshold uncertainty value, resulting in critical sequence data that is subject to misinterpretation from the artificial intelligence assistant; and in response to determining that the critical sequence data is subject to misinterpretation, resolving any uncertainties for the critical sequence data prior to concluding a communication sequence, within the communication, related to the critical data. ### Claim 2 (depends on 1) The method of claim 1, wherein determining critical sequence data comprises: analyzing linked concepts and unlinked concepts within the communication; determining, based on the analyzing, whether any conversation elements comprise linked concepts, resulting in linked conversation elements; and determining whether the linked conversation elements comprise the critical data. ### Claim 3 (depends on 2) The method of claim 2, wherein determining whether any conversation elements comprise linked concepts comprises: analyzing each conversation element of the communication; determining whether the concepts within each conversation element are associated with other conversation elements within the communication; in response to determining that the concepts within a first conversation element from the conversation elements are associated with an other conversation element within the communication, determining a proximity between the first conversation element and the other conversation element; determining whether the proximity between the first conversation element and the other conversation element is below a threshold proximity; and in response to determining that the proximity between the first conversation element and the other conversation element is below the threshold proximity, determining that the first conversation element and the other conversation element are linked. ### Claim 4 (depends on 3) The method of claim 3, further comprising: in response to determining that the concepts within a second conversation element are not associated with the other conversation elements within the conversation, determining whether concepts within the second conversation element are unresolved and will be resolved in a future conversation element; and in response to determining that the concepts within the second conversation element are unresolved and will be resolved in the future conversation element, determining that the concepts within the conversation element are associated with the future conversation element. ### Claim 5 (depends on 1) The method of claim 1, wherein predicting the uncertainty level for the critical sequence data comprises: analyzing conversation elements surrounding the critical sequence data; and determining whether there are multiple interpretations of the critical sequence data based on the conversation elements surrounding the critical sequence data. ### Claim 6 (depends on 5) The method of claim 5, wherein predicting the uncertainty level for the critical sequence data further comprises determining consequences of the multiple interpretations of the critical sequence data, wherein determining the consequences comprises: predicting how likely the multiple interpretations of the critical sequence data are to change actions in response to the critical sequence data. ### Claim 7 (depends on 5) The method of claim 5, wherein analyzing the conversation elements surrounding the critical sequence data comprises: analyzing the conversation elements in parallel form, in inverse form, and in forward form. ### Claim 8 (depends on 1) The method of claim 1, wherein resolving any uncertainties for the critical sequence data comprises: determining that a conversation element related to the critical sequence data has not been received; and stopping progress in any conversation elements relating to the critical sequence data until the one or more conversation elements are received. ### Claim 9 (depends on 1) The method of claim 1, wherein resolving any uncertainties for the critical sequence data comprises: identifying any conversation elements with the critical sequence data, resulting in critical conversation elements; and labelling the critical conversation elements as unresolved. ### Claim 10 (depends on 1) The method of claim 1, wherein resolving any uncertainties for the critical sequence data comprises: transmitting one or more clarifying inquiries to a user, the one or more clarifying inquiries querying the user to clarify one or more conversation elements relating to the critical sequence data. ### Claim 11 (depends on 1) The method of claim 1, wherein the artificial intelligence enabled communication is a healthcare communication. ### Claim 12 A system having one or more computer processors, the system configured to: monitor an artificial intelligence enabled communication between an artificial intelligence assistant and a user; analyze messages, participants, and actions within the communication; identify critical data within the communication, wherein the critical data is data within conversation elements of the communication that is important for performing the actions within the communication; determine critical sequence data, wherein the critical sequence data is a sequence of the conversation elements that is within the communication and that contains the critical data; predict an uncertainty level for the critical sequence data, wherein the uncertainty level indicates an amount of uncertainty that the artificial intelligence assistant correctly interprets the critical sequence data; determine that the uncertainty level for the critical sequence data is above a threshold uncertainty value, resulting in critical sequence data that is subject to misinterpretation from the artificial intelligence assistant; and in response to determining that the critical sequence data is subject to misinterpretation, resolve any uncertainties for the critical sequence data prior to concluding a communication sequence, within the communication, related to the critical data. ### Claim 13 (depends on 12) The system of claim 12, wherein determining critical sequence data comprises: analyzing linked concepts and unlinked concepts within the communication; determining, based on the analyzing, whether any conversation elements comprise linked concepts, resulting in linked conversation elements; and determining whether the linked conversation elements comprise the critical data. ### Claim 14 (depends on 13) The system of claim 13, wherein determining whether any conversation elements comprise linked concepts comprises: analyzing each conversation element of the communication; determining whether the concepts within each conversation element are associated with other conversation elements within the communication; in response to determining that the concepts within a first conversation element from the conversation elements are associated with an other conversation element within the communication, determining a proximity between the first conversation element and the other conversation element; determining whether the proximity between the first conversation element and the other conversation element is below a threshold proximity; and in response to determining that the proximity between the first conversation element and the other conversation element is below the threshold proximity, determining that the first conversation element and the other conversation element are linked. ### Claim 15 (depends on 12) The system of claim 12, wherein predicting the uncertainty level for the critical sequence data comprises: analyzing conversation elements surrounding the critical sequence data; and determining whether there are multiple interpretations of the critical sequence data based on the conversation elements surrounding the critical sequence data. ### Claim 16 (depends on 12) The system of claim 12, wherein resolving any uncertainties for the critical sequence data comprises: identifying any conversation elements with the critical sequence data, resulting in critical conversation elements; and labelling the critical conversation elements as unresolved. ### Claim 17 A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a server to cause the server to perform a method, the method comprising: monitoring an artificial intelligence enabled communication between an artificial intelligence assistant and a user; analyzing messages, participants, and actions within the communication; identifying critical data within the communication, wherein the critical data is data within conversation elements of the communication that is important for performing the actions within the communication; determining critical sequence data, wherein the critical sequence data is a sequence of the conversation elements that is within the communication and that contains the critical data; predicting an uncertainty level for the critical sequence data, wherein the uncertainty level indicates an amount of uncertainty that the artificial intelligence assistant correctly interprets the critical sequence data; determining that the uncertainty level for the critical sequence data is above a threshold uncertainty value, resulting in critical sequence data that is subject to misinterpretation from the artificial intelligence assistant; and in response to determining that the critical sequence data is subject to misinterpretation, resolving any uncertainties for the critical sequence data prior to concluding a communication sequence, within the communication, related to the critical data. ### Claim 18 (depends on 17) The computer program product of claim 17, wherein determining critical sequence data comprises: analyzing linked concepts and unlinked concepts within the communication; determining, based on the analyzing, whether any conversation elements comprise linked concepts, resulting in linked conversation elements; and determining whether the linked conversation elements comprise the critical data. ### Claim 19 (depends on 18) The computer program product of claim 18, wherein determining whether any conversation elements comprise linked concepts comprises: analyzing each conversation element of the communication; determining whether the concepts within each conversation element are associated with other conversation elements within the communication; in response to determining that the concepts within a first conversation element from the conversation elements are associated with an other conversation element within the communication, determining a proximity between the first conversation element and the other conversation element; determining whether the proximity between the first conversation element and the other conversation element is below a threshold proximity; and in response to determining that the proximity between the first conversation element and the other conversation element is below the threshold proximity, determining that the first conversation element and the other conversation element are linked. ### Claim 20 (depends on 17) The computer program product of claim 17, wherein predicting the uncertainty level for the critical sequence data comprises: analyzing conversation elements surrounding the critical sequence data; and determining whether there are multiple interpretations of the critical sequence data based on the conversation elements surrounding the critical sequence data.