Voice vector framework for authenticating user interactions

A system, comprising:

• a non-transitory memory;
• and
• one or more hardware processors coupled to the non-transitory memory and configured to execute instructions from the non-transitory memory to cause the system to perform operations comprising:
• receiving user interaction data associated with an interaction between a user device and a service provider server associated with a service, the user interaction data comprising audio data associated with the interaction;
• extracting, using a feature extraction engine, a plurality of user attributes from the audio data;
• determining an intent of the interaction from the plurality of user attributes;
• selecting, from a plurality of machine learning-trained classifiers, one or more machine learning-trained classifiers that correspond to the determined intent;
• applying each of the one or more machine learning-trained classifiers to the plurality of user attributes;
• generating a plurality of voice vectors, wherein each voice vector of the plurality of voice vectors is associated with a corresponding one of the one or more machine learning-trained classifiers;
• selecting one of the one or more machine learning-trained classifiers that corresponds to one of the plurality of voice vectors having a highest aggregate score;
• determining a voice signature associated with the selected one of the one or more machine learning-trained classifiers;
• determining whether the user device interacted with the service provider server in a prior interaction based on the voice signature;
• and
• sending a notification to a communication device associated with the service provider server, the notification comprising an indication of whether the user device interacted with the service provider server in the prior interaction.

A non-transitory machine-readable medium having instructions stored thereon, the instructions executable to cause a machine to perform operations comprising:

• receiving user interaction data associated with an interaction between a user device and a service provider server associated with a service, the user interaction data comprising audio data associated with the interaction;
• extracting, using a feature extraction engine, a plurality of user attributes from the audio data;
• obtaining user account information from a user account associated with the user device;
• selecting a machine learning-trained classifier that corresponds to a combination of features based on the user account information;
• applying the machine learning-trained classifier to the plurality of user attributes;
• generating a voice vector comprising a plurality of scores based on the applying of machine learning-trained classifier, wherein each of the plurality of scores in the voice vector indicates a likelihood that a first respective attribute of the plurality of user attributes corresponds to a second respective attribute of the combination of features;
• comparing the voice vector to a baseline vector that corresponds to the combination of features;
• and
• sending a notification to a communication device associated with the service provider server, the notification comprising an indication of whether the user device is verified based on the comparing.