Conversational speech agent

A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to:

• operate an audio interface to receive a caller audio signal during a call session;
• generate an audio transcript comprising a sentiment score from the caller audio signal through operation of a sentiment analysis engine configured by a sentiment model;
• communicate the audio transcript to a user interface switch configured to receive inputs from a user model;
• communicate a response control from the user interface switch to a speech synthesizer engine trained with historical conversation data from the user model;
• operate the speech synthesizer engine to:
• generate a response signal for the caller audio signal and the audio transcript through operation of a response logic engine configured by the historical conversation data;
• and
• generate a synthesized audio response comprising an ambient signal and a synthesized user model response from the response signal through operation of a speech synthesis model configured by the historical conversation data;
• communicate the synthesized audio response, responsive to the caller audio signal, through the audio interface during the call session;
• receive a user model input through the user interface switch from the user model, in response to the receiving the caller audio signal through the audio interface;
• communicate a user model audio response, responsive to the caller audio signal, to the audio interface, the user model audio response comprising response audio and background noise;
• store the audio transcript, the caller audio signal, and the user model audio response as the historical conversation data in a controlled memory data structure;
• and
• operate the speech synthesis model to generate the ambient signal from the background noise of user model responses in the historical conversation data.

A computing apparatus, the computing apparatus comprising:

• a processor;
• and
• a memory storing instructions that, when executed by the processor, configure the apparatus to:
• operate an audio interface to receive a caller audio signal during a call session;
• generate an audio transcript comprising a sentiment score from the caller audio signal through operation of a sentiment analysis engine configured by a sentiment model;
• communicate the audio transcript to a user interface switch configured to receive inputs from a user model;
• communicate a response control from the user interface switch to a speech synthesizer engine trained with historical conversation data from the user model;
• operate the speech synthesizer engine to:
• generate a response signal for the caller audio signal and the audio transcript through operation of a response logic engine configured by the historical conversation data;
• and
• generate a synthesized audio response comprising an ambient signal and a synthesized user model response from the response signal through operation of a speech synthesis model configured by the historical conversation data;
• communicate the synthesized audio response responsive to the caller audio signal through the audio interface during the call session;
• receive a user model input through the user interface switch from the user model, in response to the receiving the caller audio signal through the audio interface;
• communicate a user model audio response, responsive to the caller audio signal, to the audio interface, the user model audio response comprising response audio and background noise;
• store the audio transcript, the caller audio signal, and the user model audio response as the historical conversation data in a controlled memory data structure;
• and
• operate the speech synthesis model to generate the ambient signal from the background noise of user model responses in the historical conversation data.