# US 12,114,149 B2 · Virtual sound localization for video teleconferencing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,114,149 B2 - Document ID: US12114149B2 - Status: Granted - Kind: B2 - Application number: 17/861,876 - Filed: 2022-07-11 - Published: 2023-02-09 - Granted: 2024-10-08 - Earliest claimed: 2021-08-04 - Assignees: DSP Group Ltd. - Applicants: DSP Group Ltd. - Inventors: Avraham Keren - Web page: https://patentgrep.com/patent/US12114149B2 ## CPC - H04N 7/147: {Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals (selecting H04Q)} - H04S 7/304: {For headphones} - H04L 12/1822: {Conducting the conference, e.g. admission, detection, selection or grouping of participants, correlating users to one or more conference sessions, prioritising transmission} - H04L 12/1827: {Network arrangements for conference optimisation or adaptation} - H04N 7/15: Conference systems - H04S 7/30: {Control circuits for electronic adaptation of the sound field} - H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence) - H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD] - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - H04S: STEREOPHONIC SYSTEMS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract This disclosure provides methods, devices, and systems for videoconferencing. The present implementations more specifically relate to audio signal processing techniques that can be used to identify speakers in a videoconference. In some aspects, an audio signal processor may map each speaker in a videoconference to a respective spatial direction and transform the audio signals received from each speaker using one or more transfer functions associated with the spatial direction to which the speaker is mapped. The audio signal processor may further transmit the transformed audio signals to an audio output device that emits sounds waves having a directionality associated with the transformation. For example, the audio signal processor may apply one or more head-related transfer functions to the audio signals received from a particular speaker so that the sound waves emitted by the audio output device are perceived as originating from the spatial direction to which the speaker is mapped. ## Claims ### Claim 1 A method of processing audio signals, comprising: receiving a first audio signal associated with a videoconference that includes a plurality of participants; detecting a first speaker associated with the first audio signal, the first speaker being one of the plurality of participants in the videoconference; receiving position information indicating a relative position of each of the plurality of participants on a display; determining a position of the first speaker on the display based on the received position information; mapping the first speaker to a first spatial direction associated with the position of the first speaker on the display; transforming the first audio signal based on one or more transfer functions associated with the first spatial direction; and transmitting the transformed first audio signal to an audio output device. ### Claim 2 (depends on 1) The method of claim 1, wherein the determining of the position of the first speaker comprises: receiving video frames associated with the videoconference, the position of the first speaker being determined based at least in part on the received video frames. ### Claim 3 (depends on 1) The method of claim 1, wherein the detecting of the first speaker associated with the first audio signal comprises: receiving video frames associated with the videoconference; and performing an image processing operation on the received video frames that identifies the first speaker associated with the first audio signal. ### Claim 4 (depends on 1) The method of claim 1, wherein the one or more transfer functions include a head-related transfer function (HRTF) that causes sound waves emitted by the audio output device to be perceived as originating from the first spatial direction. ### Claim 5 (depends on 1) The method of claim 1, wherein the one or more transfer functions include a first transfer function associated with a left audio channel of the audio output device and a second transfer function associated with a right audio channel of the audio output device, the first transfer function being different than the second transfer function. ### Claim 6 (depends on 1) The method of claim 1, further comprising: receiving feedback associated with the first audio signal; and adjusting the one or more transfer functions based on the received feedback. ### Claim 7 (depends on 1) The method of claim 1, further comprising: receiving a second audio signal associated with the videoconference; detecting a second speaker associated with the second audio signal, the second speaker being one of the plurality of participants different than the first speaker; mapping the second speaker to a second spatial direction different than the first spatial direction; transforming the second audio signal based on one or more transfer functions associated with the second spatial direction; and transmitting the transformed second audio signal to the audio output device. ### Claim 8 A videoconferencing system, comprising: a processing system; and a memory storing instructions that, when executed by the processing system, causes the videoconferencing system to: receive a first audio signal associated with a videoconference that includes a plurality of participants; detect a first speaker associated with the first audio signal, the first speaker being one of the plurality of participants in the videoconference; receive position information indicating a relative position of each of the plurality of participants on a display; determine a position of the first speaker on the display based on the received position information; map the first speaker to a first spatial direction associated with the position of the first speaker on the display; transform the first audio signal based on one or more transfer functions associated with the first spatial direction; and transmit the transformed first audio signal to an audio output device. ### Claim 9 (depends on 8) The videoconferencing system of claim 8, wherein the determining of the position of the first speaker comprises: receiving video frames associated with the videoconference, the position of the first speaker being determined based at least in part on the received video frames. ### Claim 10 (depends on 8) The videoconferencing system of claim 8, wherein the detecting of the first speaker associated with the first audio signal comprises: receiving video frames associated with the videoconference; and performing an image processing operation on the received video frames that identifies the speaker associated with the received audio signal. ### Claim 11 (depends on 8) The videoconferencing system of claim 8, wherein the one or more transfer functions include a head-related transfer function (HRTF) that causes sound waves emitted by the audio output device to be perceived as originating from the first spatial direction. ### Claim 12 (depends on 8) The videoconferencing system of claim 8, wherein the one or more transfer functions include a first transfer function associated with a left audio channel of the audio output device and a second transfer function associated with a right audio channel of the audio output device, the first transfer function being different than the second transfer function. ### Claim 13 (depends on 12) The videoconferencing system of claim 12, wherein the audio output device is a stereo headset. ### Claim 14 (depends on 8) The videoconferencing system of claim 8, wherein execution of the instructions further causes the videoconferencing system to: receive a second audio signal associated with the videoconference; detect a second speaker associated with the second audio signal, the second speaker being one of the plurality of participants different than the first speaker; map the second speaker to a second spatial direction different than the first spatial direction; transform the second audio signal based on one or more transfer functions associated with the second spatial direction; and transmit the transformed second audio signal to the audio output device. ### Claim 15 A videoconferencing system comprising: a display device configured to display a videoconference that includes a plurality of participants; an audio output device configured to emit sound waves based on audio signals associated with the videoconference; and an audio signal processor configured to: receive a first audio signal associated with the videoconference; detect a first speaker associated with the first audio signal, the first speaker being one of the plurality of participants in the videoconference; receive position information indicating a relative position of each of the plurality of participants on the display device; determine a position of the first speaker on the display based on the received position information; map the first speaker to a first spatial direction associated with the position of the first speaker on the display device; transform the first audio signal based on one or more transfer functions associated with the first spatial direction; and provide the transformed first audio signal to the audio output device. ### Claim 16 (depends on 15) The video conferencing system of claim 15, wherein the audio signal processor is further configured to: receive a second audio signal associated with the videoconference; detect a second speaker associated with the second audio signal, the second speaker being one of the plurality of participants different than the first speaker; map the second speaker to a second spatial direction different than the first spatial direction; transform the second audio signal based on one or more transfer functions associated with the second spatial direction; and provide the transformed second audio signal to the audio output device.