Back to results
US 11,227,626 B1Granted
Audio response messages
AssigneeSnap Inc.
InventorsGurunandan Krishnan Gorumkonda, Shree K. Nayar
Life of this patent
Estimating the term…
Abstract
An audio response system can generate multimodal messages that can be dynamically updated on viewer's client device based on a type of audio response detected. The audio responses can include keywords or continuum-based signal (e.g., levels of wind noise). A machine learning scheme can be trained to output classification data from the audio response data for content selection and dynamic display updates.
Claims18
A method comprising:
- storing, on a user device, an electronic message comprising an image and content data associated with an audio interaction for the electronic message;
- causing, on a display device of the user device, a presentation displaying the electronic message and an indication to perform the audio interaction with the user device;
- storing, on the user device, visual sound data of sound generated while the electronic message is displayed on the user device, the visual sound data recording an environment of the user device;
- generating a sound classification by applying a convolutional neural network to the visual sound data;
- determining that the visual sound data comprises the audio interaction of the electronic message;
- and
- displaying the content data on the display device in response to determining that the visual sound data comprises the audio interaction,
- wherein the indication prompts a user of the user device to perform a non-verbal audio interaction, and wherein the convolutional neural network is trained to detect non-verbal sounds.
5 more elements
A system comprising:
- one or more processors of a machine;
- and
- a memory storing instructions that, when executed by the one or more processors, cause the machine to perform operations comprising:
- storing an electronic message comprising an image and content data associated with an audio interaction for the electronic message;
- causing a presentation displaying the electronic message and an indication to perform the audio interaction;
- storing visual sound data of sound generated while the electronic message is displayed, the visual sound data recording an environment;
- generating a sound classification by applying a convolutional neural network to the visual sound data;
- determining that the visual sound data comprises the audio interaction captured while the electronic message is displayed;
- and
- displaying the content data on the display device in response to determining that the visual sound data comprises the audio interaction,
- wherein the indication prompts a user of the user device to perform a non-verbal audio interaction, and wherein the convolutional neural network is trained to detect non-verbal sounds.
8 more elements
A non-transitory machine-readable storage medium embodying instructions that, when executed by a machine, cause the machine to perform operations comprising:
- storing an electronic message comprising an image and content data associated with an audio interaction for the electronic message;
- causing a presentation displaying the electronic message and an indication to perform the audio interaction;
- storing visual sound data of sound generated while the electronic message is displayed, the visual sound data recording an environment;
- generating a sound classification by applying a convolutional neural network to the visual sound data;
- determining that the visual sound data comprises the audio interaction captured while the electronic message is displayed;
- and
- displaying the content data on the display device in response to determining that the visual sound data comprises the audio interaction,
- wherein the indication prompts a user of the user device to perform a non-verbal audio interaction, and wherein the convolutional neural network is trained to detect non-verbal sounds.
5 more elements
Dependencies are inferred from the claim text.