System and method for generating user specific interactive voice responses based on user speech and voice characteristics

A method, comprising:

• receiving a call from a first user of a plurality of users, wherein the call comprises a potential request to initiate an execution of one or more interactions with a first user profile of a plurality of user profiles associated with a plurality of users, wherein the first user profile is associated with a first user, and wherein the call is received by an interactive voice response (IVR) system configured to service calls with respect to the plurality of user profiles, and, in response:
• generating, based at least in part on the call from the first user, a first voice interaction configured to prompt the first user to perform an utterance of a second voice interaction;
• detecting, based at least in part on the first voice interaction, the utterance of the second voice interaction performed by the first user;
• in response to detecting the utterance of the second voice interaction, executing a first machine-learning model trained to identify speech characteristics and voice characteristics of the first user and to generate, based at least in part on the identified speech characteristics and the identified voice characteristics, a third voice interaction reflective of the identified speech characteristics and the identified voice characteristics, wherein the third voice interaction comprises a personalized generated voice interaction including a speech and a voice personalized to the first user;
• providing the third voice interaction to the first user;
• executing a second machine-learning model trained to identify an intent and one or more named entities of a request of the first user based at least in part on the second voice interaction and the identified speech characteristics and the identified voice characteristics;
• and
• in response to identifying the intent and the one or more named entities of the request of the first user, initiating the execution of the one or more interactions with the first user profile in accordance with the identified intent and the one or more named entities of the request.

A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to:

• receive a call from a first user of a plurality of users, wherein the call comprises a potential request to initiate an execution of one or more interactions with a first user profile of a plurality of user profiles associated with a plurality of users, wherein the first user profile is associated with a first user, and wherein the call is received by an interactive voice response (IVR) system configured to service calls with respect to the plurality of user profiles, and, in response:
• generate, based at least in part on the call from the first user, a first voice interaction configured to prompt the first user to perform an utterance of a second voice interaction;
• detect, based at least in part on the first voice interaction, the utterance of the second voice interaction performed by the first user;
• in response to detecting the utterance of the second voice interaction, execute a first machine-learning model trained to identify speech characteristics and voice characteristics of the first user and to generate, based at least in part on the identified speech characteristics and the identified voice characteristics, a third voice interaction reflective of the identified speech characteristics and the identified voice characteristics, wherein the third voice interaction comprises a personalized generated voice interaction including a speech and a voice personalized to the first user;
• provide the third voice interaction to the first user;
• execute a second machine-learning model trained to identify an intent and one or more named entities of a request of the first user based at least in part on the second voice interaction and the identified speech characteristics and the identified voice characteristics;
• and
• in response to identifying the intent and the one or more named entities of the request of the first user, initiate the execution of the one or more interactions with the first user profile in accordance with the identified intent and the one or more named entities of the request.