Confirming alignment between contemporaneous visual and audio feedback of the agent during customer calls

A method for destressing an agent in a contact center of an enterprise, the method comprising:

• running, using a processor, an application to identify a state of the agent, said state comprising a state of stress;
• wherein:
• the application is a monitoring and triggering application (“MTA”);
• the MTA runs an artificial intelligence machine learning (“MTA AI/ML”) algorithm to identify the state of stress, said MTA AI/ML algorithm comprising:
• multidimensional transformers that extract numerical features:
• from a video clip to obtain a first data stream feature (“first feature”) comprising a facial expression feature, a gesture recognition feature, or a combination thereof, and
• from a speech sample to obtain a second data stream feature (“second feature”) comprising an audio sentiment score feature, a transcript tone score feature, a voice tone score feature, or a combination thereof;
• transformer units trained separately for the agent;
• weighted decision functions that give differential weight to the first feature and the second feature;
• and
• a contemporaneous output function that validates alignment of the first feature and the second feature using timestamp tracking;
• wherein:
• the first feature has a first emotion of the agent that registers above a first feature threshold, said first feature threshold corresponding to the first feature;
• the second feature that is contemporaneous with the first feature has a second emotion of the agent that is aligned with the first emotion of the agent;
• the second feature is defined as being contemporaneous with the first feature when the first feature and the second feature have timestamps that are separated by sixty seconds or less from each other;
• the MTA subscribes to a statistics server for agent performance data;
• upon identifying the state of stress, the MTA contacts an agent scheduling workforce management (WFM) tool to coordinate provision of a destress break to avoid efficiency metric penalties for the agent;
• and
• the MTA communicates with a telephony interaction server to route incoming calls away from the agent and to provide the destress break on a desktop computer of the agent.

A non-transitory computer-readable medium storing instructions readable by a processor for destressing an agent in a contract center of an enterprise, the instructions causing the processor to perform processes comprising:

• running, using the processor, an application to identify a state of the agent, said state comprising a state of stress;
• wherein:
• the application is a monitoring and triggering application (“MTA”);
• the MTA runs an artificial intelligence machine learning (“MTA AI/ML”) algorithm to identify the state of stress, said MTA AI/ML algorithm comprising:
• multidimensional transformers that extract numerical features:
• from a video clip to obtain a first data stream feature (“first feature”) comprising a facial expression feature, a gesture recognition feature, or a combination thereof, and;
• from a speech sample to obtain a second data stream feature (“second feature”) comprising an audio sentiment score feature, a transcript tone score feature, a voice tone score feature, or a combination thereof;
• transformer units trained separately for the agent;
• weighted decision functions that give differential weight to the first feature and the second feature;
• and a contemporaneous output function that validates alignment of the first feature and the second feature using timestamp tracking;
• wherein:
• the first feature has a first emotion of the agent that registers above a first feature threshold, said first feature threshold corresponding to the first feature;
• the second feature that is contemporaneous with the first feature has a second emotion of the agent that is aligned with the first emotion of the agent:
• the second emotion is aligned with the first emotion when the first emotion registers above the first feature threshold and the second emotion, contemporaneous with the first emotion, registers above a second feature threshold;
• the first feature threshold is determined before a call;
• the second feature threshold is determined before the call;
• the second feature is defined as being contemporaneous with the first feature when the first feature and the second feature have timestamps that are separated by sixty seconds or less from each other;
• the MTA AI/ML algorithm measures the first feature and the second feature during the call;
• the MTA AI/ML algorithm identifies the state of stress of the agent during the call;
• the MTA subscribes to a statistics server for agent performance data;
• when identifying that the agent is in the state of stress:
• providing, using the processor, notification to a supervisor of the agent that the agent is eligible for a destress break;
• receiving, using the processor, an approval from the supervisor for the agent to receive the destress break;
• contacting, using the processor, an agent scheduling workforce management (WFM) tool to coordinate provision of the destress break to avoid efficiency metric penalties for the agent;
• after the agent completes the call, communicating, using the processor, with a telephony interaction server to route incoming calls away from the agent;
• providing, using the processor, on a desktop computer of the agent the destress break;
• and
• upon completion of the destress break, communicating, using the processor, with the telephony interaction server to route incoming calls to the agent.