# US 10,504,529 B2 · Binaural audio encoding/decoding and rendering for a headset > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,504,529 B2 - Document ID: US10504529B2 - Status: Granted - Kind: B2 - Application number: 15/807,806 - Filed: 2017-11-09 - Published: 2019-05-09 - Granted: 2019-12-10 - Earliest claimed: 2017-11-09 - Assignees: Cisco Technology, Inc. - Applicants: Cisco Technology, Inc. - Inventors: Haohai Sun - Web page: https://patentgrep.com/patent/US10504529B2 ## CPC - G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing - H04S 1/005: {For headphones} - H04S 7/304: {For headphones} - G10L 19/167: {Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes} - G10L 2021/02166: {Microphone arrays; Beamforming} - H04R 2201/401: 2D or 3D arrays of transducers - H04R 2201/403: Linear arrays of transducers - H04R 2201/405: Non-uniform arrays of transducers or a plurality of uniform arrays with different transducer spacing - H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04S: STEREOPHONIC SYSTEMS - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) ## Abstract A method and apparatus for providing binaural audio for a headset is provided. In one embodiment, a method includes encoding audio signals to provide binaural audio to a headset. The method includes receiving audio signals from a microphone array comprising a first plurality of elements and applying far-field array processing to the audio signals to generate a first plurality of channels. The channels can be beam channels and each channel is associated with a particular beam angle. The method further includes selecting a second plurality of channels from the first plurality of channels that is a subset of the first plurality of channels. The method includes encoding the audio signals from the selected second plurality of channels with information associated with the particular beam angle for each of the selected second plurality of channels. The encoded audio signals are configured to provide binaural audio to a headset. ## Claims ### Claim 1 A method of encoding audio signals to provide binaural audio to a headset, the method comprising: receiving audio signals from a microphone array comprising a first plurality of elements; generating a first plurality of channels based on the audio signals received from the first plurality of elements of the microphone array, wherein the first plurality of channels are active beam channels and each active beam channel of the first plurality of channels is associated with a particular beam angle for that active beam channel; selecting a second plurality of channels from the first plurality of channels that satisfy a defined activity criteria among the active beam channels of the first plurality of channels, wherein the second plurality of channels is a subset of the first plurality of channels and includes a smaller number of channels than the first plurality of channels; including a beam identifier, instead of a corresponding particular beam angle, with each of the selected second plurality of channels, wherein the beam identifier is a number that represents an association of a channel of the selected second plurality of channels with the corresponding particular beam angle; estimating, using a speaker tracking function, a distance from a sound source for each of the selected second plurality of channels; and encoding the audio signals from the selected second plurality of channels with information associated with the particular beam angle and the distance for each of the selected second plurality of channels, wherein the encoded audio signals are configured to provide the binaural audio to the headset. ### Claim 2 (depends on 1) The method of claim 1, wherein the second plurality of channels are the most active channels of the first plurality of channels. ### Claim 3 (depends on 1) The method of claim 1, wherein the defined activity criteria is based on at least one of a sound pressure level or a sound pressure ratio. ### Claim 4 (depends on 1) The method of claim 1, wherein the defined activity criteria is based on at least one of a signal-to-noise ratio or a signal-to-reverberation ratio. ### Claim 5 (depends on 1) The method of claim 1, wherein the second plurality of channels are configured to cover at least between 180 to 360 degrees of a three-dimensional space of environment 180 continuous degrees. ### Claim 6 (depends on 1) The method of claim 1, wherein the particular beam angle of each beam channel is a fixed angle. ### Claim 7 (depends on 1) The method of claim 1, wherein the microphone array is one of a linear array, a planar array, a circular array, or a spherical array. ### Claim 8 (depends on 1) The method of claim 1, further comprising: directly transmitting the encoded audio signals configured to provide the binaural audio to the headset, wherein the first plurality of channels are virtual microphone channels. ### Claim 9 A method of rendering binaural audio for a headset, the method comprising: receiving audio signals comprising a plurality of channels, wherein each channel is encoded with information associated with a particular beam angle for that channel, wherein the information includes a beam identifier which is a number that represents an association of a beam channel with a corresponding particular beam angle for the beam channel; receiving a signal associated with a head rotation angle from a head tracking sensor of the headset; determining a rotated beam angle for each of the particular beam angles associated with each channel of the plurality of channels by subtracting the head rotation angle received from the head tracking sensor from the particular beam angle for the beam channel determined based on the beam identifier; after determining the rotated beam angle for each of the particular beam angles associated with each channel of the plurality of channels, generating a plurality of binaural audio signals by applying a head related transfer function to each channel of the plurality of channels, wherein the head related transfer function of each channel of the plurality of channels is based on a plurality of sound sources being located at certain distances estimated by a speaker tracking function integrated with the headset; combining the plurality of binaural audio signals into a single binaural audio channel; and providing the single binaural audio channel to the headset. ### Claim 10 (depends on 9) The method of claim 9, further comprising extracting the plurality of channels and associated particular beam angle for each channel, wherein beam identifiers in the information are retrieved from a table. ### Claim 11 (depends on 9) The method of claim 9, wherein the association between the beam identifier and the corresponding particular beam angle for the beam channel is stored in a table. ### Claim 12 (depends on 9) The method of claim 9, wherein applying the head related transfer function to each channel is based on a sound source at a fixed distance. ### Claim 13 (depends on 9) The method of claim 9, wherein applying the head related transfer function to each channel is based on a sound source having an estimated distance. ### Claim 14 (depends on 9) The method of claim 9, wherein the particular beam angle of each beam channel is a fixed angle. ### Claim 15 An apparatus for encoding audio signals to provide binaural audio to a headset comprising: a microphone array comprising a first plurality of elements; at least one processor in communication with the microphone array and configured to: receive the audio signals from the first plurality of elements; generate a first plurality of channels based on the audio signals, wherein the first plurality of channels are active beam channels and each active beam channel of the first plurality of channels is associated with a particular beam angle for that active beam channel; select a second plurality of channels from the first plurality of channels that satisfy a defined activity criteria among the active beam channels of the first plurality of channels, wherein the second plurality of channels is a subset of the first plurality of channels and includes a smaller number of channels than the first plurality of channels; include a beam identifier, instead of a corresponding particular beam angle, with each of the selected second plurality of channels, wherein the beam identifier is a number that represents an association of a channel of the second plurality of channels with the corresponding particular beam angle; estimate, using a speaker tracking function, a distance from a sound source for each of the selected second plurality of channels; and encode the audio signals from the selected second plurality of channels with information associated with the particular beam angle and the distance for each of the selected second plurality of channels, wherein the encoded audio signals are configured to provide the binaural audio to the headset. ### Claim 16 (depends on 15) The apparatus of claim 15, wherein the defined activity criteria is based on at least one of a sound pressure level, a sound pressure ratio, a signal-to-noise ratio, or a signal-to-reverberation ratio. ### Claim 17 (depends on 15) The apparatus of claim 15, wherein the second plurality of channels are configured to cover at least between 180 to 360 degrees of a three-dimensional space of environment. ### Claim 18 (depends on 15) The apparatus of claim 15, wherein the microphone array is one of a linear array, a planar array, a circular array, or a spherical array. ### Claim 19 (depends on 15) The apparatus of claim 15, wherein the defined activity criteria is based on at least one of a sound pressure level or a sound pressure ratio. ### Claim 20 (depends on 15) The apparatus of claim 15, wherein the second plurality of channels are the most active channels of the first plurality of channels.