# US 2019/0378024 A1 · Systems to augment conversations with relevant information or automation using proactive bots > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0378024 A1 - Document ID: US20190378024A1 - Status: Published application - Kind: A1 - Application number: 16/469,585 - Filed: 2017-12-15 - Published: 2019-12-12 - Earliest claimed: 2016-12-16 - Assignees: Second Mind Labs, Inc. - Applicants: Second Mind Labs, Inc. - Inventors: Kul Singh; Andras Kornai; Yurii Pohrebniak - Web page: https://patentgrep.com/patent/US20190378024A1 ## CPC - G06F 16/3329: Natural language query formulation - G06N 5/043: Distributed expert systems; Blackboards - G06F 16/90332: {Natural language query formulation or dialogue systems} - G06F 16/9038: Presentation of query results - G06N 20/00: Machine learning - H04L 51/02: using automatic reactions or user delegation, e.g. automatic replies or chatbot-generated messages - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract Aspects of the technology described herein relate generally to a platform that enables the deployment of autonomous bots that identify and deliver relevant content in real-time based on received information. These bots may be designed to proactively provide relevant content without any explicit trigger from a user. For example, a bot may analyze speech and/or text in a primary communication channel (e.g., a telephone, email, webchat, or videophone) and proactively provide content relevant to the speech and/or text in one or more secondary communication channels (e.g., displayed on a computer screen, a mobile device screen, and/or a pair of smart glasses). ## Claims ### Claim 1 A system, comprising: at least one virtualized or hardware processor; at least one non-transitory computer-readable storage medium storing processor-executable instructions organized as a first bot computer program that, when executed by the at least one virtualized or hardware processor, cause the at least one virtualized or hardware processor to perform acts of: receiving information; searching of at least one first data source using a first technique to find information to augment the received information; determining an evaluation of the at least one first data source responsive to the received information; determining a result from the search of the at least one first data source from the at least one first data source responsive to the evaluation of the at least one first data source; and providing at least one first message comprising the determined result to at least one subscriber. ### Claim 2 (depends on 1) The system of claim 1, wherein the received information includes at least one of: a news article, speech by a subject, a discussion between two or more subjects, a video stream, an audio stream, a streaming dataflow, and a scientific study. ### Claim 3 (depends on 1) The system of claim 1 or any other preceding claim, further comprising receiving feedback from the at least one subscriber regarding the at least one first message and updating at least one characteristic of the first technique based on the received feedback from the at least one subscriber regarding the at least one first message. ### Claim 4 (depends on 1) The system of claim 1 or any other preceding claim, wherein the processor-executable instructions are organized as a plurality of bot computer programs, and wherein the plurality of bot computer programs each are adapted to process a same input of received information. ### Claim 5 (depends on 4) The system of claim 4 or any other preceding claim, wherein the same input of received information comprises a received audio stream. ### Claim 6 (depends on 1) The system of claim 1 or any other preceding claim, wherein the at least one first bot computer program includes a plurality of adjustable parameter comprising at least one of a resource parameter that defines a computer resource limit that can be used by the at least one first bot computer program. ### Claim 7 (depends on 1) The system of claim 1 or any other preceding claim, wherein the at least one first bot computer program includes a plurality of adjustable parameter comprising at least one of an error rejection parameter that defines a rate of error rejection used by the at least one first bot computer program. ### Claim 8 (depends on 1) The system of claim 1 or any other preceding claim, wherein the processor-executable instructions organized as the first bot computer program further cause the at least one virtualized or hardware processor to perform: providing the at least one first message to the at least one subscriber responsive to generating the at least one first message within a predetermined period of time from receiving the information. ### Claim 9 (depends on 8) The system of claim 8 or any other preceding claim, wherein the predetermined period of time is no more than 2 seconds. ### Claim 10 (depends on 8) The system of claim 8 or any other preceding claim, wherein the predetermined period of time is no more than 4 seconds. ### Claim 11 (depends on 1) The system of claim 1 or any other preceding claim, wherein the first technique includes at least one artificial intelligence technique. ### Claim 12 (depends on 1) The system of claim 1 or any other preceding claim, wherein the at least one subscriber is non-human. ### Claim 13 (depends on 1) The system of claim 1 or any other preceding claim, wherein the at least one subscriber is a bot computer program. ### Claim 14 (depends on 1) The system of claim 1 or any other preceding claim, wherein the at least one non-transitory computer-readable storage medium further stores processor-executable instructions organized as a second bot computer program that, when executed by the at least one virtualized or hardware processor, cause the at least one virtualized or hardware processor to perform: receiving the information; searching at least one second data source using a second technique to find information to augment the received information; and providing at least one second message to at least one subscriber based on a result from the search of the at least one second data source. ### Claim 15 (depends on 14) The system of claim 14 or any other preceding claim, wherein the processor-executable instructions organized as the second bot computer program further cause the at least one virtualized or hardware processor to perform: receiving feedback from the at least one subscriber regarding the at least one second message; and updating at least one characteristic of the second technique based on the received feedback from the at least one subscriber regarding the at least one second message. ### Claim 16 A method, comprising: receiving information; searching of at least one first data source using a first technique to find information to augment the received information; determining an evaluation of the at least one first data source responsive to the received information; determining a result from the search of the at least one first data source from the at least one first data source responsive to the evaluation of the at least one first data source; and providing at least one first message comprising the determined result to at least one subscriber. ### Claim 17 At least one computer-readable storage medium storing computer-executable instructions that, when executed, perform a method comprising: receiving information; searching of at least one first data source using a first technique to find information to augment the received information; determining an evaluation of the at least one first data source responsive to the received information; determining a result from the search of the at least one first data source from the at least one first data source responsive to the evaluation of the at least one first data source; and providing at least one first message comprising the determined result to at least one subscriber. ### Claim 18 A system, comprising: at least one virtualized or hardware processor; at least one non-transitory computer-readable storage medium storing processor-executable instructions organized as a first bot computer program that, when executed by the at least one virtualized or hardware processor, cause the at least one virtualized or hardware processor to perform acts of: receiving a voice signal indicative of speech; identifying at least one first data source from a plurality of data sources; searching the identified at least one first data source using a first technique to find information to augment at least a portion of the speech; determining a result from the search of the at least one first data source; and providing at least one first message comprising the determined result to at least one subscriber. ### Claim 19 (depends on 18) The system of claim 18, wherein the processor-executable instructions organized as the first bot computer program further cause the at least one virtualized or hardware processor to perform acts of: receiving feedback from the at least one subscriber regarding the at least one first message; and updating at least one characteristic of the first technique based on the received feedback from the at least one subscriber regarding the at least one first message. ### Claim 20 (depends on 18) The system of claim 18 or any other preceding claim, wherein the at least one non-transitory computer-readable storage medium further stores processor-executable instructions organized as a second bot computer program that, when executed by the at least one virtualized or hardware processor, cause the at least one virtualized or hardware processor to perform acts of: receiving the voice signal indicative of speech; searching at least one second data source from the plurality of data sources using a second technique to find information to augment at least a portion of the speech; and providing at least one second message to at least one subscriber based on a result from the search of the at least one second data source. ### Claim 21 (depends on 20) The system of claim 20 or any other preceding claim, wherein the processor-executable instructions organized as the second bot computer program further cause the at least one virtualized or hardware processor to perform acts of: receiving feedback from the at least one subscriber regarding the at least one second message; and updating at least one characteristic of the second technique based on the received feedback from the at least one subscriber regarding the at least one second message. ### Claim 22 (depends on 18) The system of claim 18 or any other preceding claim, wherein the first technique includes at least one artificial intelligence technique. ### Claim 23 (depends on 18) The system of claim 18 or any other preceding claim, wherein the at least one subscriber is non-human. ### Claim 24 (depends on 23) The system of claim 23 or any other preceding claim, wherein the at least one subscriber is a bot computer program. ### Claim 25 (depends on 18) The system of claim 18 or any other preceding claim, further comprising a display and wherein providing the at least one message to the at least one subscriber includes displaying the at least one message on the display. ### Claim 26 (depends on 18) The system of claim 18 or any other preceding claim, wherein the processor-executable instructions organized as the first bot computer program further cause the at least one virtualized or hardware processor to perform an act of: providing the at least one first message to the at least one subscriber responsive to generating the at least one first message within a predetermined period of time from receiving the voice signal. ### Claim 27 (depends on 26) The system of claim 26 or any other preceding claim, wherein the predetermined period of time is no more than 2 seconds. ### Claim 28 (depends on 26) The system of claim 26 or any other preceding claim, wherein the predetermined period of time is no more than 4 seconds. ### Claim 29 (depends on 18) The system of claim 18 or any other preceding claim, wherein at least one device that receives a voice signal indicative of speech includes at least one of a group of devices comprising augmented reality glasses, mixed reality glasses, virtual reality glasses, or a smartphone paired to any of the group of devices. ### Claim 30 (depends on 29) The system of claim 29 or any other preceding claim, wherein the at least one of a group of devices or the smartphone paired to any of the group of devices is adapted to project or output information to a screen. ### Claim 31 (depends on 30) The system of claim 30 or any other preceding claim, further comprising a specialized version of augmented or mixed reality glasses not having computer vision capabilities. ### Claim 32 A method, comprising: receiving a voice signal indicative of speech; identifying at least one first data source from a plurality of data sources; searching the identified at least one first data source using a first technique to find information to augment at least a portion of the speech; determining a result from the search of the at least one first data source; and providing at least one first message comprising the determined result to at least one subscriber. ### Claim 33 At least one computer-readable storage medium storing computer-executable instructions that, when executed, perform a method comprising: receiving a voice signal indicative of speech; identifying at least one first data source from a plurality of data sources; searching the identified at least one first data source using a first technique to find information to augment at least a portion of the speech; determining a result from the search of the at least one first data source; and providing at least one first message comprising the determined result to at least one subscriber.