# US 12,003,946 B2 · Adaptable spatial audio playback > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,003,946 B2 - Document ID: US12003946B2 - Status: Granted - Kind: B2 - Application number: 17/630,098 - Filed: 2020-07-16 - Published: 2022-10-20 - Granted: 2024-06-04 - Earliest claimed: 2019-07-30 - Assignees: Dolby Laboratories Licensing Corporation; Dolby International AB - Applicants: Dolby Laboratories Licensing Corporation; Dolby International AB - Inventors: Alan J. Seefeldt; Joshua B. Lando; Daniel Arteaga; Glenn N. Dickins; Mark Richard Paul Thomas - Web page: https://patentgrep.com/patent/US12003946B2 ## CPC - H04S 7/302: {Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence)} - H04R 5/02: Spatial or constructional arrangements of loudspeakers - H04R 27/00: Public address systems (circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; {monitoring or testing arrangements for public address systems H04R29/007}; amplifiers H03F) - H04R 29/005: {Microphone arrays} - H04R 2205/024: Positioning of loudspeaker enclosures for spatial sound reproduction - H04R 2227/005: Audio distribution systems for home, i.e. multi-room use - H04S: STEREOPHONIC SYSTEMS - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract A rendering mode may be determined for received audio data, including audio signals and associated spatial data. The audio data may be rendered for reproduction via a set of loudspeakers of an environment according to the rendering mode, to produce rendered audio signals. Rendering the audio data may involve determining relative activation of a set of loudspeakers in an environment. The rendering mode may be variable between a reference spatial mode and one or more distributed spatial modes. The reference spatial mode may have an assumed listening position and orientation. In the distributed spatial mode(s), one or more elements of the audio data may each be rendered in a more spatially distributed manner than in the reference spatial mode and spatial locations of remaining elements of the audio data may be warped such that they span a rendering space of the environment more completely than in the reference spatial mode. ## Claims ### Claim 1 An audio processing system, comprising: an interface system; and a control system configured for: receiving audio data via the interface system, the audio data including one or more audio signals and associated spatial data, the spatial data indicating an intended perceived spatial position corresponding to an audio signal, the spatial data including at least one of channel data or spatial metadata; receiving, via the interface system, a rendering mode indication, wherein receiving the rendering mode indication involves receiving an indication of a number of people in a listening area; determining a rendering mode based, at least in part, on the number of people in the listening area; rendering the audio data for reproduction via a set of loudspeakers of an environment according to the rendering mode, to produce rendered audio signals, wherein: rendering the audio data comprises determining relative activation of the set of loudspeakers in an environment; the rendering mode is variable between a reference spatial mode and one or more distributed spatial modes; the reference spatial mode has an assumed listening position and orientation; and in the one or more distributed spatial modes, one or more elements of the audio data is or are each rendered in a more spatially distributed manner than in the reference spatial mode and spatial locations of remaining elements of the audio data are warped such that they span a rendering space of the environment more completely than in the reference spatial mode; and providing, via the interface system, the rendered audio signals to at least some loudspeakers of the set of loudspeakers of the environment. ### Claim 2 (depends on 1) The audio processing system of claim 1, wherein receiving the rendering mode indication involves receiving microphone signals corresponding to a voice command. ### Claim 3 (depends on 1) The audio processing system of claim 1, further comprising a display device and a sensor system proximate the display device, wherein: the control system is further configured for controlling the display device to present a graphical user interface; and receiving the rendering mode indication involves receiving sensor signals corresponding to user input via the graphical user interface. ### Claim 4 (depends on 3) The audio processing system of claim 3, wherein the sensor signals are touch sensor signals or gesture sensor signals. ### Claim 5 (depends on 1) The audio processing system of claim 1, wherein the indication of the number of people in the listening area is based on at least one of microphone data from a microphone system or image data from a camera system. ### Claim 6 (depends on 1) The audio processing system of claim 1, wherein the rendering mode is selectable from a continuum of rendering modes ranging from the reference spatial mode to a most distributed spatial mode. ### Claim 7 (depends on 1) The audio processing system of claim 1, wherein the control system is further configured to determine the assumed listening position of the reference spatial mode, the assumed listening orientation of the reference spatial mode, or both, according to reference spatial mode data received via the interface system. ### Claim 8 (depends on 7) The audio processing system of claim 7, wherein the reference spatial mode data comprises at least one of microphone data from a microphone system or image data from a camera system. ### Claim 9 (depends on 7) The audio processing system of claim 7, further comprising a display device and a sensor system proximate the display device, wherein: the control system is further configured for controlling the display device to present a graphical user interface; and receiving reference spatial mode data involves receiving sensor signals corresponding to user input via the graphical user interface. ### Claim 10 (depends on 1) The audio processing system of claim 1, wherein the one or more elements of the audio data each rendered in a more spatially distributed manner correspond to one or more of front sound stage data, music vocals, dialogue, bass, percussion, or other solo or lead instruments. ### Claim 11 (depends on 10) The audio processing system of claim 10, wherein the front sound stage data comprises one or more of the left, right or center signals of audio data received in, or upmixed to, a Dolby 5.1, Dolby 7.1 or Dolby 9.1 format. ### Claim 12 (depends on 10) The audio processing system of claim 10, wherein the front sound stage data comprises audio data received in Dolby Atmos format and having spatial metadata indicating an (x,y) spatial position wherein y<0.5. ### Claim 13 (depends on 1) The audio processing system of claim 1, wherein the audio data includes spatial distribution metadata indicating which elements of the audio data are to be rendered in a more spatially distributed manner and wherein the control system is configured for identifying the one or more elements of the audio data to be rendered in a more spatially distributed manner according to the spatial distribution metadata. ### Claim 14 (depends on 1) The audio processing system of claim 1, wherein the control system is configured for implementing a content type classifier to identify the one or more elements of the audio data to be rendered in a more spatially distributed manner. ### Claim 15 (depends on 1) The audio processing system of claim 1, wherein at least one of the one or more distributed spatial modes involves applying a time-varying modification to the spatial location of the at least one element. ### Claim 16 (depends on 15) The audio processing system of claim 15, wherein the time-varying modification is a periodic modification. ### Claim 17 (depends on 16) The audio processing system of claim 16, wherein the periodic modification corresponds with at least one of user input, a tempo of music being reproduced in the environment, a beat of music being reproduced in the environment, or one or more other features of audio data being reproduced in the environment. ### Claim 18 (depends on 1) The audio processing system of claim 1, wherein rendering the one or more elements of the audio data in a more spatially distributed manner than in the reference spatial mode involves creating copies of the one or more elements and rendering all of the copies simultaneously at a distributed set of positions across the environment. ### Claim 19 (depends on 1) The audio processing system of claim 1, wherein the rendering is based on Center of Mass Amplitude Panning, Flexible Virtualization or a combination thereof, and wherein rendering the one or more elements of the audio data in a more spatially distributed manner than in the reference spatial mode involves warping a rendering position of each of the one or more elements towards a zero radius. ### Claim 20 An audio processing method, comprising: receiving audio data by a control system and via an interface system, the audio data including one or more audio signals and associated spatial data, the spatial data indicating an intended perceived spatial position corresponding to an audio signal, the spatial data including at least one of channel data or spatial metadata; determining, by the control system, a rendering mode by implementing a content type classifier to identify one or more elements of the audio data to be rendered in a more spatially distributed manner; rendering, by the control system, the audio data for reproduction via a set of loudspeakers of an environment according to the rendering mode, to produce rendered audio signals, wherein: rendering the audio data comprises determining relative activation of a set of loudspeakers in an environment; the rendering mode is variable between a reference spatial mode and one or more distributed spatial modes; the reference spatial mode has an assumed listening position and orientation; and in the one or more distributed spatial modes, the one or more elements of the audio data is or are each rendered in a more spatially distributed manner than in the reference spatial mode and spatial locations of remaining elements of the audio data are warped such that they span a rendering space of the environment more completely than in the reference spatial mode; and providing, via the interface system, the rendered audio signals to at least some loudspeakers of the set of loudspeakers of the environment. ### Claim 21 (depends on 20) The audio processing method of claim 20, wherein determining the rendering mode involves receiving, via the interface system, a rendering mode indication. ### Claim 22 (depends on 21) The audio processing method of claim 21, wherein receiving the rendering mode indication involves receiving microphone signals corresponding to a voice command. ### Claim 23 One or more non-transitory media having software encoded thereon, the software including instructions for controlling one or more devices to perform an audio processing method comprising: receiving audio data by a control system and via an interface system, the audio data including one or more audio signals and associated spatial data, the spatial data indicating an intended perceived spatial position corresponding to an audio signal, the spatial data including at least one of channel data or spatial metadata; determining, by the control system, a rendering mode; rendering, by the control system, the audio data for reproduction via a set of loudspeakers of an environment according to the rendering mode, to produce rendered audio signals, wherein: rendering the audio data comprises determining relative activation of a set of loudspeakers in an environment; the rendering mode is variable between a reference spatial mode and one or more distributed spatial modes; the reference spatial mode has an assumed listening position and orientation; and in the one or more distributed spatial modes, one or more elements of the audio data is or are each rendered in a more spatially distributed manner than in the reference spatial mode and spatial locations of remaining elements of the audio data are warped such that they span a rendering space of the environment more completely than in the reference spatial mode, wherein rendering the one or more elements of the audio data in a more spatially distributed manner than in the reference spatial mode involves creating copies of the one or more elements and rendering all of the copies simultaneously at a distributed set of positions across the environment; and providing, via the interface system, the rendered audio signals to at least some loudspeakers of the set of loudspeakers of the environment. ### Claim 24 (depends on 23) The one or more non-transitory media of claim 23, wherein determining the rendering mode involves receiving, via the interface system, a rendering mode indication. ### Claim 25 (depends on 24) The one or more non-transitory media of claim 24, wherein receiving the rendering mode indication involves receiving microphone signals corresponding to a voice command. ### Claim 26 An audio processing system, comprising: an interface system; and a control system configured for: receiving audio data via the interface system, the audio data including one or more audio signals and associated spatial data, the spatial data indicating an intended perceived spatial position corresponding to an audio signal, the spatial data including at least one of channel data or spatial metadata; determining a rendering mode; rendering the audio data for reproduction via a set of loudspeakers of an environment according to the rendering mode, to produce rendered audio signals, wherein: rendering the audio data comprises determining relative activation of a set of loudspeakers in an environment; the rendering mode is variable between a reference spatial mode and one or more distributed spatial modes, wherein at least one of the one or more distributed spatial modes involves applying a periodic modification to the spatial location of the at least one element; the reference spatial mode has an assumed listening position and orientation; and in the one or more distributed spatial modes, one or more elements of the audio data is or are each rendered in a more spatially distributed manner than in the reference spatial mode and spatial locations of remaining elements of the audio data are warped such that they span a rendering space of the environment more completely than in the reference spatial mode; providing, via the interface system, the rendered audio signals to at least some loudspeakers of the set of loudspeakers of the environment. ### Claim 27 (depends on 26) The audio processing system of claim 26, wherein the periodic modification corresponds with at least one of user input, a tempo of music being reproduced in the environment, a beat of music being reproduced in the environment, or one or more other features of audio data being reproduced in the environment. ### Claim 28 An audio processing system, comprising: an interface system; and a control system configured for: receiving audio data via the interface system, the audio data including one or more audio signals and associated spatial data, the spatial data indicating an intended perceived spatial position corresponding to an audio signal, the spatial data including at least one of channel data or spatial metadata; determining a rendering mode; rendering the audio data for reproduction via a set of loudspeakers of an environment according to the rendering mode, to produce rendered audio signals, wherein: the rendering is based on Center of Mass Amplitude Panning, Flexible Virtualization or a combination thereof; rendering the audio data comprises determining relative activation of a set of loudspeakers in an environment; the rendering mode is variable between a reference spatial mode and one or more distributed spatial modes; the reference spatial mode has an assumed listening position and orientation; and in the one or more distributed spatial modes, one or more elements of the audio data is or are each rendered in a more spatially distributed manner than in the reference spatial mode and spatial locations of remaining elements of the audio data are warped such that they span a rendering space of the environment more completely than in the reference spatial mode; rendering the one or more elements of the audio data in a more spatially distributed manner than in the reference spatial mode involves warping a rendering position of each of the one or more elements towards a zero radius; and providing, via the interface system, the rendered audio signals to at least some loudspeakers of the set of loudspeakers of the environment. ### Claim 29 An audio processing system, comprising: an interface system; and a control system configured for: receiving audio data via the interface system, the audio data including one or more audio signals and associated spatial data, the spatial data indicating an intended perceived spatial position corresponding to an audio signal, the spatial data including at least one of channel data or spatial metadata; determining a rendering mode; rendering the audio data for reproduction via a set of loudspeakers of an environment according to the rendering mode, to produce rendered audio signals, wherein: rendering the audio data comprises determining relative activation of a set of loudspeakers in an environment; the rendering mode is variable between a reference spatial mode and one or more distributed spatial modes; the reference spatial mode has an assumed listening position and orientation; and in the one or more distributed spatial modes, one or more elements of the audio data is or are each rendered in a more spatially distributed manner than in the reference spatial mode and spatial locations of remaining elements of the audio data are warped such that they span a rendering space of the environment more completely than in the reference spatial mode, the one or more elements of the audio data each rendered in a more spatially distributed manner corresponding to one or more of front sound stage data, music vocals, dialogue, bass, percussion, or other solo or lead instruments, the front sound stage data comprising audio data received in Dolby Atmos format and having spatial metadata indicating an (x,y) spatial position wherein y<0.5; and providing, via the interface system, the rendered audio signals to at least some loudspeakers of the set of loudspeakers of the environment.