Audio turn understanding system
AssigneeCharacter Technologies Inc.
InventorsPrajit Ramachandran, Yi Cui
Life of this patent
Estimating the term…
Abstract
A real-time audio stream associated with a user is segmented into one or more chunks of audio. The one or more segmented chunks of audio are provided to an audio understanding model. It is determined that the user is finished with their turn in a conversation. In response to determining that the user has finished with their turn in the conversation, a response is provided based on the real-time audio stream.
Claims30
A method, comprising:
- analyzing a real-time audio stream associated with a user;
- determining whether a silent portion associated with the real-time audio stream is greater than or equal to a silent threshold;
- in response to determining that the silent portion associated with the real-time audio stream is greater than or equal to the silent threshold, providing the real-time audio stream to an audio understanding model, wherein the audio understanding model is trained using diarized data;
- determining, by the audio understanding model, that the user is finished with their turn in a conversation;
- in response to determining that the user is finished with their turn in the conversation, waiting a buffer period;
- and
- in response to the buffer period lapsing, providing written text associated with the real-time audio stream to a large language model.
The method of claim 16, further comprising changing to a listening mode and receiving a second real-time audio stream associated with the user.
The method of 18, wherein the provided response is the pre-generated response generated by the large language model.
A system, comprising:
- a processor configured to:
- analyze a real-time audio stream associated with a user;
- determine whether a silent portion associated with the real-time audio stream is greater than or equal to a silent threshold;
- in response to determining that the silent portion associated with the real-time audio stream is greater than or equal to the silent threshold, provide the real-time audio stream to an audio understanding model, wherein the audio understanding model is trained using diarized data;
- determine, by the audio understanding model, that the user is finished with their turn in a conversation;
- in response to determining that the user is finished with their turn in the conversation, wait a buffer period;
- and
- in response to the buffer period lapsing, provide written text associated with the real-time audio stream to a large language model.
A computer program product embodied in a non-transitory computer readable medium and comprising computer instructions for:
- analyzing a real-time audio stream associated with a user;
- determining whether a silent portion associated with the real-time audio stream is greater than or equal to a silent threshold;
- in response to determining that the silent portion associated with the real-time audio stream is greater than or equal to the silent threshold, providing the real-time audio stream to an audio understanding model, wherein the audio understanding model is trained using diarized data;
- determining, by the audio understanding model, that the user is finished with their turn in a conversation;
- in response to determining that the user is finished with their turn in the conversation, waiting a buffer period;
- and
- in response to the buffer period lapsing, providing written text associated with the real-time audio stream to a large language model.
Dependencies are inferred from the claim text.