Virtual audio augmentation using computer vision
AssigneeNVIDIA Corporation
InventorsNitin Mahesh Gode, Ashish Anand, Ambrish Dantrey, Murali Krishna Kamisetty
Life of this patent
Estimating the term…
Abstract
Disclosed are apparatuses, systems, and techniques that provide virtual immersion sound experience and spatialization effects with an audio device supporting a low number of sound channels, according to at least one embodiment. The techniques include but are not limited to associating input audio channels of an audio stream with virtual speakers, identifying, using an optical sensor, positioning of a user's head relative to the virtual speakers, determining simulated sound intensities at one or more reference locations associated with the user's head, and generating, based on the simulated sound intensities, output audio signals configured for physical speakers.
Claims19
A method comprising:
- associating audio data of an audio stream transmitted over a plurality of input audio channels with a plurality of virtual speakers;
- and
- for each of a plurality of times:
- identifying, using an optical sensor, a position of a user's head relative to the plurality of virtual speakers at a respective time of the plurality of times;
- estimating, for the respective time and based at least on the identified position of the user's head, distances from each of the plurality of virtual speakers to one or more reference locations associated with the user's head;
- determining, for the respective time and based at least on the estimated distances, a plurality of simulated sound intensities at the one or more reference locations associated with the position of the user's head, wherein the simulated sound intensities are associated with the plurality of virtual speakers;
- and
- generating, for the respective time and based on the plurality of simulated sound intensities, a plurality of output audio signals configured for a plurality of physical speakers.
A system comprising:
- one or more processing devices to:
- associate audio data of an audio stream transmitted using a plurality of input audio channels with a plurality of virtual speakers;
- and
- for each of a plurality of times:
- identify, using an optical sensor, a position of a user's head relative to the plurality of virtual speakers at a respective time of the plurality of times;
- estimate, for the respective time and based at least on the identified position of the user's head, distances from each of the plurality of virtual speakers to one or more reference locations associated with the user's head;
- determine, for the respective time and based at least on the estimated distances, a plurality of simulated sound intensities at the one or more reference locations associated with the user's head, wherein the simulated sound intensities are associated with the plurality of virtual speakers;
- and
- generate, for the respective time and based on the plurality of simulated sound intensities, a plurality of output audio signals configured for a plurality of physical speakers.
A processor comprising one or more processing units to:
- associate audio data of an audio stream transmitted using a plurality of input audio channels with a plurality of virtual speakers;
- and
- for each of a plurality of times:
- identify, using an optical sensor, a position of a user's head relative to the plurality of virtual speakers at a respective time of the plurality of times;
- estimate, for the respective time and based at least on the identified position of the user's head, distances from each of the plurality of virtual speakers to one or more reference locations associated with the user's head;
- determine, for the respective time and based at least on the estimated distances, a plurality of simulated sound intensities at the one or more reference locations associated with the user's head, wherein the simulated sound intensities are associated with the plurality of virtual speakers;
- and
- generate, for the respective time and based on the plurality of simulated sound intensities, a plurality of output audio signals configured for a plurality of physical speakers.
Dependencies are inferred from the claim text.