Artificial intelligence voice response system for speech impaired users

A computer system for voice responses, comprising:

• one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage medium, and program instructions stored on at least one of the one or more tangible storage medium for execution by at least one of the one or more processors via at least one of the one or more memories, wherein the computer system is capable of performing a method comprising:
• gathering user data from at least one connected device, wherein the connected device comprises an artificial intelligence (AI) system that can observe human conversations and detect behavioral signals or biometric patterns of a user within a range of sensors used to detect the user's behavioral signals or biometric patterns;
• predicting when a user desires to submit a voice command and whether the user has the ability to submit the voice command, wherein heuristics and health conditions of the user are used to predict a topic of a voice command and voice request and to provide a spoken menu from which the user may choose at least one voice command;
• analyzing user speech stream data via natural language processing (NLP) algorithms to dynamically determine a satisfaction or frustration level of a user;
• passing the gathered user data to a random forest algorithm to perform a binary classification;
• training a voice response system based on the gathered user data, wherein the trained voice response system includes a plurality of voice menus, one of the plurality of voice menus being dynamically modified to include a set of hierarchical questions relating to an observed interaction or set of interactions with a user, until a customized menu is determined;
• using a deep reinforcement learning model, such as a long short term memory recurrent neural network model (LSTM-RNN), to improve a knowledge corpus by correlating gathered behavioral input, body language, and biometric signals with an intended topic or a hierarchical voice menu related to the intended topic;
• identifying a wakeup signal for the connected device based on observed behavioral signals or biometric patterns of the user within the range of the sensors used to detect the user's behavioral signals or biometric patterns, wherein the identified wakeup signal comprises a change in a behavior or a biometric parameter of the user;
• determining that user engagement is intended based on identifying the wakeup signal;
• handling a non-routine event of the user by initiating a phone call to a live person who may assist in understanding the non-routine event;
• determining the customized menu based on the intended user engagement;
• and
• engaging with the user through the at least one connected device.

A computer program product for voice responses, comprising:

• one or more non-transitory computer-readable storage media and program instructions stored on at least one of the one or more tangible storage media, the program instructions executable by a processor to cause the processor to perform a method comprising:
• gathering user data from at least one connected device, wherein the connected device comprises an artificial intelligence (AI) system that can observe human conversations and detect behavioral signals or biometric patterns of a user within a range of sensors used to detect the user's behavioral signals or biometric patterns;
• predicting when a user desires to submit a voice command and whether the user has the ability to submit the voice command, wherein heuristics and health conditions of the user are used to predict a topic of a voice command and voice request and to provide a spoken menu from which the user may choose at least one voice command;
• analyzing user speech stream data via natural language processing (NLP) algorithms to dynamically determine a satisfaction or frustration level of a user;
• passing the gathered user data to a random forest algorithm to perform a binary classification;
• training a voice response system based on the gathered user data, wherein the trained voice response system includes a plurality of voice menus, one of the plurality of voice menus being dynamically modified to include a set of hierarchical questions relating to an observed interaction or set of interactions with a user, until a customized menu is determined;
• using a deep reinforcement learning model, such as a long short term memory recurrent neural network model (LSTM-RNN), to improve a knowledge corpus by correlating gathered behavioral input, body language, and biometric signals with an intended topic or a hierarchical voice menu related to the intended topic;
• identifying a wakeup signal for the connected device based on observed behavioral signals or biometric patterns of the user within the range of the sensors used to detect the user's behavioral signals or biometric patterns, wherein the identified wakeup signal comprises a change in a behavior or a biometric parameter of the user;
• determining that user engagement is intended based on identifying the wakeup signal;
• handling a non-routine event of the user by initiating a phone call to a live person who may assist in understanding the non-routine event;
• determining the customized menu based on the intended user engagement;
• and
• engaging with the user through the at least one connected device.