# US 10,121,485 B2 · Spatial audio resource management and mixing for applications > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,121,485 B2 - Document ID: US10121485B2 - Status: Granted - Kind: B2 - Application number: 15/460,140 - Filed: 2017-03-15 - Published: 2017-10-05 - Granted: 2018-11-06 - Earliest claimed: 2016-03-30 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Robert Norman Heitkamp; Philip Andrew Edry; Paul J. Radek - Web page: https://patentgrep.com/patent/US10121485B2 ## CPC - G10L 19/20: using sound class specific coding, hybrid encoders or object based coding - G06F 3/16: Sound input; Sound output (speech processing G10L) - G06F 3/162: {Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs} - G06F 3/165: {Management of the audio stream, e.g. setting of volume, audio stream path} - G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing - H04L 65/1104: Session initiation protocol [SIP] - H04S 3/002: {Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution (control circuits for electronic adaptation of the sound field H04S7/30)} - H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)} - H04S 7/308: {Electronic adaptation dependent on speaker or headphone connection} - H04S 7/303: {Tracking of listener position or orientation} - H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04S: STEREOPHONIC SYSTEMS ## Abstract The techniques disclosed herein enable a system to coordinate audio objects that are generated by multiple applications. A system can receive contextual data from several applications and dynamically determine an allocation of a number of audio objects for each application based on the contextual data. The allocation can be based on a status of one or more applications, user interactions with one or more applications, and other factors. Policy data can also cause the system to allocate a number of audio objects to one or more applications based on an application type and other factors. For instance, a policy may cause a system to allocate more audio objects to a game application vs. a communications application. As a user interacts with an application, e.g., moves or resizes a user interface, closes an application, increases or decreases a level of interaction, the system can reallocate audio objects to individual applications. ## Claims ### Claim 1 A computing device, comprising: a processor; a computer-readable storage medium in communication with the processor, the computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the processor to: receive capabilities data indicating a threshold number of audio objects that can be processed by a speaker configuration of an endpoint device in communication with the computing device; receive policy data defining priorities for individual applications of a plurality of applications; generate allocation data indicating a number of allocated audio objects for at least one application of the plurality of applications based, at least in part, on the policy data defining a priority for the at least one application, wherein the number of allocated audio objects for the at least one application does not exceed the threshold number of audio objects that can be processed by the speaker configuration of the endpoint device; and transmit the allocation data to the at least one application causing the at least one application to control a number of generated audio objects, wherein the number of generated audio objects does not exceed the number of allocated audio objects, wherein the at least one application causes an encoder to render the generated audio objects at the endpoint device. ### Claim 2 (depends on 1) The computing device of claim 1, wherein the computer-executable instructions further cause the processor to: receive updated contextual data indicating a change associated with the individual applications; generate updated allocation data indicating an updated number of allocated audio objects for the at least one application based on the updated contextual data; and transmit updated allocation data to the at least one application for modifying the number of audio objects generated by the at least one application. ### Claim 3 (depends on 1) The computing device of claim 1, wherein the computer-executable instructions further cause the processor to: receive updated capabilities data indicating a change to the threshold number of audio objects that can be processed by a speaker configuration of an endpoint device in communication with the computing device; generate updated allocation data indicating an updated number of allocated audio objects for the at least one application based on the threshold number of audio objects; and transmit updated allocation data to the at least one application for modifying the number of audio objects generated by the at least one application. ### Claim 4 (depends on 1) The computing device of claim 1, wherein the at least one application comprises a first application and a second application, wherein the allocation data indicates a first number of allocated audio objects for the first application and a second number of allocated objects for the second application, wherein the first number of allocated audio objects is greater than the second number of allocated objects when the policy data indicates that a priority value associated with the first application is higher than a priority value associated with the second application. ### Claim 5 (depends on 1) The computing device of claim 1, wherein the computer-executable instructions further cause the processor to receive contextual data associated with individual applications of a plurality of applications, wherein the contextual data indicates a level of user interaction with the individual applications, wherein generating the allocation data indicating the allocation of audio objects for the at least one application is further based on the level of user interaction with the individual applications. ### Claim 6 (depends on 1) The computing device of claim 1, wherein the computer-executable instructions further cause the processor to receive contextual data associated with individual applications of a plurality of applications, wherein the contextual data indicates a priority value that is based on a user interface arrangement, wherein generating the allocation data indicating the allocation of audio objects for the at least one application is further based on the priority value. ### Claim 7 (depends on 1) The computing device of claim 1, wherein the computer-executable instructions further cause the processor to: cause the encoder to generate a rendered output signal based on the generated audio objects; and cause a communication of the rendered output signal from the encoder to the endpoint device for producing a spatialized audio output based on the endpoint device, wherein the spatialized audio output is an audible output. ### Claim 8 (depends on 1) The computing device of claim 1, wherein the at least one application comprises a first application and a second application, wherein the allocation data indicates a first number of allocated audio objects for the first application and a second number of allocated audio objects for the second application, wherein the first number of allocated audio objects is greater than the second number of allocated audio objects when a first graphical user interface associated with the first application has a larger display area than a second graphical user interface associated with the second application. ### Claim 9 (depends on 1) The computing device of claim 1, wherein the at least one application comprises a first application and a second application, wherein the allocation data indicates a first number of allocated audio objects for the first application and a second number of allocated objects for the second application, wherein the first number of allocated audio objects is greater than the second number of allocated objects when a first graphical user interface associated with the first application is positioned within a predetermined area and a second graphical user interface associated with the second application is positioned outside of the predetermined area. ### Claim 10 (depends on 1) The computing device of claim 1, wherein the at least one application comprises a first application and a second application, wherein the allocation data indicates a first number of allocated audio objects for the first application and a second number of allocated objects for the second application, wherein the first number of allocated audio objects is greater than the second number of allocated objects when input data from an input device indicates that a user is viewing a first graphical user interface associated with the first application instead of a second graphical user interface associated with the second application. ### Claim 11 A computing device, comprising: a processor; a computer-readable storage medium in communication with the processor, the computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the processor, cause the processor to: receive contextual data indicating a status with individual applications of a plurality of applications; receive policy data defining priorities for individual applications of a plurality of applications; generate allocation data indicating a number of allocated audio objects for at least one application of the plurality of applications based, at least in part, on the policy data defining a priority and a status for the at least one application, wherein the number of allocated audio objects for the at least one application does not exceed a threshold number of audio objects that can be processed by a speaker configuration of an endpoint device; and transmit the allocation data to the at least one application causing the at least one application to control a number of generated audio objects, wherein the number of generated audio objects does not exceed the number of allocated audio objects, wherein the at least one application causes an encoder to render the generated audio objects at the endpoint device. ### Claim 12 (depends on 11) The computing device of claim 11, wherein the computer-executable instructions further cause the processor to: receive updated contextual data indicating a change associated with the individual applications; generate updated allocation data indicating an updated number of allocated audio objects for the at least one application based on the updated contextual data; and transmit updated allocation data to the at least one application for modifying the number of audio objects generated by the at least one application. ### Claim 13 (depends on 11) The computing device of claim 11, wherein the computer-executable instructions further cause the processor to: receive updated capabilities data indicating a change to the threshold number of audio objects that can be processed by a speaker configuration of an endpoint device in communication with the computing device; generate updated allocation data indicating an updated number of allocated audio objects for the at least one application based on the updated contextual data; and transmit updated allocation data to the at least one application for modifying the number of audio objects generated by the at least one application. ### Claim 14 (depends on 11) The computing device of claim 11, wherein the at least one application comprises a first application and a second application, wherein the allocation data indicates a first number of allocated audio objects for the first application and a second number of allocated audio objects for the second application, wherein the first number of allocated audio objects is greater than the second number of allocated audio objects when the policy data indicates that a priority value associated with the first application is higher than a priority value associated with the second application. ### Claim 15 (depends on 11) The computing device of claim 11, wherein the contextual data indicates a level of user interaction with the individual applications, wherein generating the allocation data indicating the allocation of audio objects for the at least one application is further based on the level of user interaction with the individual applications. ### Claim 16 (depends on 11) The computing device of claim 11, wherein the contextual data indicates a priority value that is based on a user interface arrangement, wherein generating the allocation data indicating the allocation of audio objects for the at least one application is further based on the priority value. ### Claim 17 (depends on 11) The computing device of claim 11, wherein the computer-executable instructions further cause the processor to: cause the encoder to generate a rendered output signal based on the generated audio objects; and cause a communication of the rendered output signal from the encoder to the endpoint device for producing a spatialized audio output based on the endpoint device, wherein the spatialized audio output is an audible output. ### Claim 18 (depends on 11) The computing device of claim 11, wherein the at least one application comprises a first application and a second application, wherein the allocation data indicates a first number of allocated audio objects for the first application and a second number of allocated audio objects for the second application, wherein the first number of allocated audio objects is greater than the second number of allocated audio objects when a first graphical user interface associated with the first application has a larger display area then a second graphical user interface associated with the second application. ### Claim 19 (depends on 11) The computing device of claim 11, wherein the at least one application comprises a first application and a second application, wherein the allocation data indicates a first number of allocated audio objects for the first application and a second number of allocated audio objects for the second application, wherein the first number of allocated audio objects is greater than the second number of allocated audio objects when a first graphical user interface associated with the first application is positioned within a predetermined area and a second graphical user interface associated with the second application is positioned outside of the predetermined area. ### Claim 20 (depends on 11) The computing device of claim 11, wherein the at least one application comprises a first application and a second application, wherein the allocation data indicates a first number of allocated audio objects for the first application and a second number of allocated audio objects for the second application, wherein the first number of allocated audio objects is greater than the second number of allocated audio objects when input data from an input device indicates that a user is viewing a first graphical user interface associated with the first application instead of a second graphical user interface associated with the second application. ### Claim 21 A method, comprising: receiving contextual data indicating a status with individual applications of a plurality of applications; receiving policy data defining priorities for individual applications of a plurality of applications; generating allocation data indicating a number of allocated audio objects for at least one application of the plurality of applications based, at least in part, on the policy data defining a priority for the at least one application, wherein the number of allocated audio objects for the at least one application does not exceed a threshold number of audio objects that can be processed by the speaker configuration of an endpoint device; and transmitting the allocation data to the at least one application causing the at least one application to control a number of generated audio objects, wherein the number of generated audio objects does not exceed the number of allocated audio objects, wherein the at least one application causes an encoder to render the generated audio objects at the endpoint device. ### Claim 22 (depends on 21) The method of claim 21, further comprising: receiving updated contextual data indicating a change associated with the individual applications; generating updated allocation data indicating an updated number of allocated audio objects for the at least one application based on the updated contextual data; and transmitting updated allocation data to the at least one application for modifying the number of audio objects generated by the at least one application. ### Claim 23 (depends on 21) The method of claim 21, further comprising: receiving updated capabilities data indicating a change to the a threshold number of audio objects that can be processed by a speaker configuration of an endpoint device in communication with the computing device; generating updated allocation data indicating an updated number of allocated audio objects for the at least one application based on the updated contextual data; and transmit updated allocation data to the at least one application for modifying the number of audio objects generated by the at least one application. ### Claim 24 (depends on 21) The method of claim 21, wherein the at least one application comprises a first application and a second application, wherein the allocation data indicates a first number of allocated audio objects for the first application and a second number of allocated objects for the second application, wherein the first number of allocated audio objects is greater than the second number of allocated objects when the policy data indicates that a priority value associated with the first application is higher than a priority value associated with the second application. ### Claim 25 (depends on 21) The method of claim 21, wherein the contextual data indicates a level of user interaction with the individual applications, wherein generating the allocation data indicating the allocation of audio objects for the at least one application is further based on the level of user interaction with the individual applications.