Real-time sound field synthesis by modifying produced audio streams
AssigneeLittle Dog Live, LLC
InventorJody Elff
Life of this patent
Estimating the term…
Abstract
A client device is disclosed that receives, from a server, a live video stream and a production quality live ambisonic audio stream generated during performance of a live event at a venue. The live ambisonic audio stream is generated from audio channels captured by audio capture devices disposed at the venue. The audio channels captured at the event, and modified by a producer, can be compared to audio captured by an ambisonic microphone positioned within the event space to determine the phase and relative amplitude of those channels as received by a particular ambisonic microphone channel. In this manner, raw and/or produced audio channels captured at the event can be shifted and mixed together to generate a production quality ambisonic stream.
Claims19
A client device comprising:
- at least one processor;
- at least one memory storing instructions, which when executed by the at least one processor, cause the at least one processor to:
- receive, from a server, a live video stream and a live ambisonic audio stream captured during performance of a live event at a venue, the live ambisonic audio stream including multiple audio channels generated from multiple audio source inputs captured by respective audio capture devices disposed at the venue;
- in accordance with a cranial structure characteristic of the user of the client device, modify the respective gain or the respective phase of the one or more audio channels of the live ambisonic audio stream determine a current head orientation of a user of the client device;
- in accordance with the determined head orientation of the user of the client device:
- modify a respective phase of one or more audio channels of the live ambisonic audio stream;
- cause display of at least a portion of the live video stream on a display of the client device while playing the ambisonic audio stream having the respective phase modified for the one or more audio channels.
A computing device, comprising:
- at least one processor;
- and
- at least one memory storing instructions, which when executed by the at least one processor, cause the at least one processor to:
- receive captured live ambisonic audio stream from multiple audio capture devices disposed at a venue during performance of a live event;
- in accordance with a respective location of each audio capture device of the multiple audio capture devices and for a particular location at the venue, determine a respective value of phase delay for one or more audio channels of the captured live ambisonic audio stream;
- generate a live ambisonic audio stream by applying the determined respective value of phase delay for the one or more audio channels of the captured live ambisonic audio stream;
- and
- transmit, to a client device of multiple client devices, a live video stream and the generated live ambisonic audio stream to cause display of at least a portion of the live video stream on a display of the client device while playing the live ambisonic audio stream, the client device modifying a respective gain or the respective phase of the one or more audio channels of the live ambisonic audio stream in accordance with a cranial structure characteristic of a user of the client device.
A method comprising:
- receiving, at a client device from a server, a live video stream and a live ambisonic audio stream captured during performance of a live event at a venue, the live ambisonic audio stream including multiple audio channels generated from multiple audio source inputs captured by respective audio capture devices disposed at the venue;
- modify, by the client device, at least one of the one or more audio channels based at least in part on a head-related transfer function specific to a user of the client device;
- determining, by the client device, movement details of the user of the client device;
- in accordance with the determined movement details of the user of the client device:
- modifying a respective phase of one or more audio channels in the live ambisonic audio stream;
- and
- causing display of at least a portion of the live video stream on a display of the client device while playing the ambisonic audio stream having the respective phase modified for the one or more audio channels.
Dependencies are inferred from the claim text.