# US 2020/0401370 A1 · Artificial intelligence based response to a user based on engagement level > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2020/0401370 A1 - Document ID: US20200401370A1 - Status: Published application - Kind: A1 - Application number: 16/445,375 - Filed: 2019-06-19 - Published: 2020-12-24 - Earliest claimed: 2019-06-19 - Assignees: International Business Machines Corporation - Applicants: International Business Machines Corporation - Inventors: Shilpa Shetty; Mithun Das; Amitabha Chanda; Sarbajit K. Rakshit - Web page: https://patentgrep.com/patent/US20200401370A1 ## CPC - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06N 20/00: Machine learning - G10L 13/00: Speech synthesis; Text to speech systems - G10L 15/18: using natural language modelling - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 17/22: Interactive procedures; Man-machine interfaces - G10L 2015/227: {of the speaker; Human-factor methodology} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING ## Abstract Aspects of the present invention disclose a method for delivering an artificial intelligence-based response to a voice command to a user. The method includes one or more processors identifying an audio command received by a computing device. The method further includes determining a first engagement level of a user, wherein an engagement level corresponds to an attentiveness level of the user in relation to the computing device based at least in part on indications of activities of the user. The method further includes identifying a first set of conditions within an immediate operating environment of the computing device, wherein the first set of conditions indicate whether to deliver a voice response to the identified audio command. The method further includes determining whether to deliver the voice response to the identified audio command to the user based at least in part on the first engagement level and first set of conditions. ## Claims ### Claim 1 A method comprising: identifying, by one or more processors, an audio command received by a computing device; determining, by one or more processors, a first engagement level of a user, wherein an engagement level corresponds to an attentiveness level of the user in relation to the computing device based at least in part on indications of one or more activities of the user; identifying, by one or more processors, a first set of conditions within an immediate operating environment of the computing device, wherein the first set of conditions indicate whether to deliver a voice response to the identified audio command; and determining, by one or more processors, whether to deliver the voice response to the identified audio command to the user based at least in part on the first engagement level and the first set of conditions. ### Claim 2 (depends on 1) The method of claim 1, further comprising: determining, by one or more processors, a time-frame to deliver the voice response to the identified audio command; and responsive to detecting the time-frame to deliver the voice response to the identified audio command and determining that the first engagement level of the user exceeds a defined threshold score, providing, by one or more processors, the voice response to the identified audio command to the user, wherein providing the response to the audio command includes performing a defined action. ### Claim 3 (depends on 1) The method of claim 1, further comprising: determining, by one or more processors, a second engagement level of the user; determining, by one or more processors, whether the second engagement level of the user exceeds a threshold score; and in response to determining whether the second engagement level of the user exceeds the threshold score, determining, by one or more processors, whether to perform a defined action wherein the defined action includes instructions on whether to play, pause, and delegate the voice response. ### Claim 4 (depends on 1) The method of claim 1, wherein determining whether to deliver the voice response to the identified audio command to the user based at least in part on the first engagement level and the first set of conditions, further comprises: identifying, by one or more processors, a first set of preferential parameters of data of the voice response; identifying, by one or more processors, a second set of preferential parameters of the first engagement level of the user; and determining, by one or more processors, a comparative priority between the first set of preferential parameters and the second set of preferential parameters. ### Claim 5 (depends on 4) The method of claim 4, further comprising: responsive to determining the first set of preferential parameters having the comparative priority, determining, by one or more processors, a defined action to increase the first engagement level, wherein the defined action is selected from the group consisting of: repeating the voice response, flashing lights, audible notifications, and increasing volume of the voice response; and performing, by one or more processors, one or more of the defined actions to increase the first engagement level, wherein the comparative priority is determined at least in part on a second set of conditions that correspond to a historical interaction between the user and the computing device. ### Claim 6 (depends on 4) The method of claim 4, wherein determining the comparative priority between the first set of preferential parameters and the second set of preferential parameters, further comprises: identifying, by one or more processors, a rank of the first set of preferential parameters and the second set of preferential parameters, wherein preferential parameters are selected from a group consisting of: activity type, time constraints, status of an identified person, emergency situation, and conversation topic; comparing, by one or more processors, the rank of the first set of preferential parameters and the second set of preferential parameters; and determining, by one or more processors, an order of priority of the compared ranks based at least in part on a hierarchy of preferential parameters, wherein the hierarchy of preferential parameters is based at least in part on historical interactions of the user with the computing device. ### Claim 7 (depends on 1) The method of claim 1, wherein determining the first engagement level of the user, further comprises: retrieving, by one or more processors, data of the computing device and one or more Internet of Things (IoT) devices within the immediate operating environment of the computing device; identifying, by one or more processors, factors that comprise the first engagement level of the user, wherein the identified factors are selected from the group consisting of: user activities, user eye gaze direction, a person, user conversations, and user vital signs; and determining, by one or more processors, a score corresponding to the first engagement level based at least in part on the identified factors. ### Claim 8 A computer program product comprising: one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising: program instructions to identify an audio command received by a computing device; program instructions determine a first engagement level of a user, wherein an engagement level corresponds to an attentiveness level of the user in relation to the computing device based at least in part on indications of one or more activities of the user; program instructions identify a first set of conditions within an immediate operating environment of the computing device, wherein the first set of conditions indicate whether to deliver a voice response to the identified audio command; and program instructions determine whether to deliver the voice response to the identified audio command to the user based at least in part on the first engagement level and the first set of conditions. ### Claim 9 (depends on 8) The computer program product of claim 8, further comprising program instructions, stored on the one or more computer readable storage media, to: determine a time-frame to deliver the voice response to the identified audio command; and responsive to detecting the time-frame to deliver the voice response to the identified audio command and determining that the first engagement level of the user exceeds a defined threshold score, provide the voice response to the identified audio command to the user, wherein providing the response to the audio command includes performing a defined action. ### Claim 10 (depends on 8) The computer program product of claim 8, further comprising program instructions, stored on the one or more computer readable storage media, to: determine a second engagement level of the user; determine whether the second engagement level of the user exceeds a threshold score; and in response to determining whether the second engagement level of the user exceeds the threshold score, determine whether to perform a defined action wherein the defined action includes instructions on whether to play, pause, and delegate the voice response. ### Claim 11 (depends on 8) The computer program product of claim 8, wherein program instructions to determine whether to deliver the voice response to the identified audio command to the user based at least in part on the first engagement level and the first set of conditions, further comprise program instructions to: identify a first set of preferential parameters of data of the voice response; identify a second set of preferential parameters of the first engagement level of the user; and determine a comparative priority between the first set of preferential parameters and the second set of preferential parameters. ### Claim 12 (depends on 11) The computer program product of claim 11, further comprising program instructions, stored on the one or more computer readable storage media, to: responsive to determining the first set of preferential parameters having the comparative priority, determine a defined action to increase the first engagement level, wherein the defined action is selected from the group consisting of: repeating the voice response, flashing lights, audible notifications, and increasing volume of the voice response; and perform one or more of the defined actions to increase the first engagement level, wherein the comparative priority is determined at least in part on a second set of conditions that correspond to a historical interaction between the user and the computing device. ### Claim 13 (depends on 11) The computer program product of claim 11, wherein program instructions to determine a comparative priority between the first set of preferential parameters and the second set of preferential parameters, further comprise program instructions to: identify a rank of the first set of preferential parameters and the second set of preferential parameters, wherein preferential parameters are selected from a group consisting of: activity type, time constraints, status of an identified person, emergency situation, and conversation topic; compare the rank of the first set of preferential parameters and the second set of preferential parameters; and determine an order of priority of the compared ranks based at least in part on a hierarchy of preferential parameters, wherein the hierarchy of preferential parameters is based at least in part on historical interactions of the user with the computing device. ### Claim 14 (depends on 8) The computer program product of claim 8, wherein program instructions to determine the first engagement level of the user, further comprise program instructions to: retrieve data of the computing device and one or more Internet of Things (IoT) devices within the immediate operating environment of the computing device; identify factors that comprise the first engagement level of the user, wherein the identified factors are selected from the group consisting of: user activities, user eye gaze direction, a person, user conversations, and user vital signs; and determine a score corresponding to the first engagement level based at least in part on the identified factors. ### Claim 15 A computer system comprising: one or more computer processors; one or more computer readable storage media; and program instructions stored on the computer readable storage media for execution by at least one of the one or more processors, the program instructions comprising: program instructions to identify an audio command received by a computing device; program instructions determine a first engagement level of a user, wherein an engagement level corresponds to an attentiveness level of the user in relation to the computing device based at least in part on indications of one or more activities of the user; program instructions identify a first set of conditions within an immediate operating environment of the computing device, wherein the first set of conditions indicate whether to deliver a voice response to the identified audio command; and program instructions determine whether to deliver the voice response to the identified audio command to the user based at least in part on the first engagement level and the first set of conditions. ### Claim 16 (depends on 15) The computer system of claim 15, further comprising program instructions, stored on the one or more computer readable storage media, to: determine a time-frame to deliver the voice response to the identified audio command; and responsive to detecting the time-frame to deliver the voice response to the identified audio command and determining that the first engagement level of the user exceeds a defined threshold score, provide the voice response to the identified audio command to the user, wherein providing the response to the audio command includes performing a defined action. ### Claim 17 (depends on 15) The computer system of claim 15, further comprising program instructions, stored on the one or more computer readable storage media, to: determine a second engagement level of the user; determine whether the second engagement level of the user exceeds a threshold score; and in response to determining whether the second engagement level of the user exceeds the threshold score, determine whether to perform a defined action wherein the defined action includes instructions on whether to play, pause, and delegate the voice response. ### Claim 18 (depends on 15) The computer system of claim 15, wherein program instructions to determine whether to deliver the voice response to the identified audio command to the user based at least in part on the first engagement level and the first set of conditions, further comprise program instructions to: identify a first set of preferential parameters of data of the voice response; identify a second set of preferential parameters of the first engagement level of the user; and determine a comparative priority between the first set of preferential parameters and the second set of preferential parameters. ### Claim 19 (depends on 18) The computer system of claim 18, further comprising program instructions, stored on the one or more computer readable storage media, to: responsive to determining the first set of preferential parameters having the comparative priority, determine a defined action to increase the first engagement level, wherein the defined action is selected from the group consisting of: repeating the voice response, flashing lights, audible notifications, and increasing volume of the voice response; and perform one or more of the defined actions to increase the first engagement level, wherein the comparative priority is determined at least in part on a second set of conditions that correspond to a historical interaction between the user and the computing device. ### Claim 20 (depends on 18) The computer system of claim 18, wherein program instructions to determine a comparative priority between the first set of preferential parameters and the second set of preferential parameters, further comprise program instructions to: identify a rank of the first set of preferential parameters and the second set of preferential parameters, wherein preferential parameters are selected from a group consisting of: activity type, time constraints, status of an identified person, emergency situation, and conversation topic; compare the rank of the first set of preferential parameters and the second set of preferential parameters; and determine an order of priority of the compared ranks based at least in part on a hierarchy of preferential parameters, wherein the hierarchy of preferential parameters is based at least in part on historical interactions of the user with the computing device.