# US 2026/0075381 A1 · Method and apparatus for managing audio in a multi-speaker environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0075381 A1
- Document ID: US20260075381A1
- Status: Published application
- Kind: A1
- Application number: 19/368,699
- Filed: 2025-10-24
- Published: 2026-03-12
- Earliest claimed: 2024-09-09
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Ranjan Kumar SAMAL; Rishabh GUPTA; Somesh NANDA
- Web page: https://patentgrep.com/patent/US20260075381A1

## CPC

- H04S 7/304: {For headphones}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- H04R 1/10: Earpieces; Attachments therefor {; Earphones; Monophonic headphones (H04R1/28 takes precedence; stereophonic headphones H04R5/033)}
- H04R 1/1041: {Mechanical or electronic switches, or control elements (switches in general H01H)}
- H04R 3/00: Circuits for transducers (arrangements for producing a reverberation or echo sound G10K15/08; amplifiers H03F)
- H04R 5/033: Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)}
- H04S 7/303: {Tracking of listener position or orientation}
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2400/15: Aspects of sound capture and related signal processing for recording or reproduction
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- H04S: STEREOPHONIC SYSTEMS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

A method of managing audio in a multi-speaker environment, performed by a wearable audio device may be provided. The method may include generating, based on a binaural audio signal captured by the wearable audio device, a virtual sound source map indicating a localized position of one or more sources of sound. The method may include estimating, based on the virtual sound source map, one or more target sources indicating sources of sound of interest to a user of the wearable audio device. The method may include transmitting metadata associated with the wearable audio device, to an electronic device coupled to the wearable audio device, to cause the electronic device to refine the one or more target sources based on the metadata. The method may include receiving, from the electronic device, a processed audio signal associated with at least one refined target source.

## Claims

### Claim 1

A method of managing audio in a multi-speaker environment, performed by a wearable audio device, the method comprising:
generating, based on a binaural audio signal captured by the wearable audio device, a virtual sound source map indicating a localized position of one or more sources of sound;
estimating, based on the virtual sound source map, one or more target sources indicating sources of sound of interest to a user of the wearable audio device;
transmitting metadata associated with the wearable audio device, to an electronic device coupled to the wearable audio device, to cause the electronic device to refine the one or more target sources based on the metadata; and
receiving, from the electronic device, a processed audio signal associated with at least one refined target source.

### Claim 2 (depends on 1)

The method of claim 1, wherein the one or more sources of sound comprise the user of the wearable audio device and one or more speakers in the multi-speaker environment.

### Claim 3 (depends on 1)

The method of claim 1, wherein the generating the virtual sound source map comprises:
estimating a head movement of the user based on data obtained from one or more head sensors associated with the wearable audio device;
determining relative positions of the one or more sources of sound with respect to the head movement of the user;
computing horizontal offset angles and vertical offset angles for the one or more sources of sound;
generating embedding vectors indicating identification of the one or more sources of sound; and
generating the virtual sound source map based on:
the relative positions of the one or more sources of sound,
the horizontal offset angles and the vertical offset angles, and
the embedding vectors.

### Claim 4 (depends on 1)

The method of claim 1, wherein the metadata comprises the virtual sound source map and the one or more target sources.

### Claim 5 (depends on 1)

The method of claim 1, wherein the estimating the one or more target sources comprises:
estimating, based on the virtual sound source map, directions of conversation of the one or more sources of sound;
estimating a relative movement of a head of the user with respect to an initial head position of the user;
monitoring one or more head gestures of the user and classifying the one or more head gestures, the classification of the one or more head gestures comprising agreement gestures and disagreement gestures;
determining, using a target detection model, one or more sound source pairs between which a live interaction is present based on the directions of conversation;
generating, using the target detection model, an interaction timeline of the user associated with the one or more sources of sound, based on the one or more sound source pairs, the relative movement, and the one or more head gestures,
wherein the interaction timeline comprises a time duration and a type of interaction of the user associated with the one or more sources of sound, and wherein the type of interaction includes: direct, indirect and passive; and
estimating, using the target detection model, the one or more target sources based on the interaction timeline.

### Claim 6 (depends on 1)

The method of claim 1, wherein the receiving the processed audio signal comprises receiving an amplified audio signal associated with the at least one refined target source, and wherein the method further comprises playing back the amplified audio signal to the user from the wearable audio device.

### Claim 7 (depends on 1)

The method of claim 1, wherein the metadata comprises information corresponding to the binaural audio signal, the virtual sound source map, and an interaction timeline of the user.

### Claim 8 (depends on 5)

The method of claim 5, wherein the generating the virtual sound source map comprises rendering and serializing the virtual sound source map in a three-dimensional space, and
wherein the estimating the one or more target sources comprises:
estimating, using one or more head sensors associated with the wearable audio device, one or more head gestures of the user;
generating the interaction timeline based on the one or more head gestures, a head position of the user, and a head direction of the user; and
updating the one or more target sources in response to a change in the head direction of the user.

### Claim 9 (depends on 8)

The method of claim 8,
wherein the estimating the one or more target sources comprises:
assigning, using a trained AI model, priorities to at least one source of sound based on a direction of conversation with respect to the user; and
selecting the one or more target sources based on the priorities and the interaction timeline.

### Claim 10 (depends on 5)

The method of claim 5, wherein the estimating the directions of conversation comprises:
inputting, into a trained AI model, a head direction of the user (102) and directions of the one or more sources of sound from the virtual sound source map (412); and
outputting, from the trained AI model, estimates of the directions of conversation for the one or more sources of sound.

### Claim 11

A wearable audio device for managing audio in a multi-speaker environment, the wearable audio device comprising:
a microphone;
memory storing instructions; and
at least on processor;
wherein the instructions, when executed by the at least one processor, individually or collectively, cause the wearable audio device to:
generate, based on a binaural audio signal captured by the microphone, a virtual sound source map indicating a localized position of one or more sources of sound;
estimate one or more target sources;
transmit metadata associated with the wearable audio device to an electronic device coupled to the wearable audio device to cause the electronic device to refine the one or more target sources based on the metadata; and
receive, from the electronic device, a processed audio signal associated with at least one refined target source.

### Claim 12 (depends on 11)

The wearable audio device as claimed in claim 11, wherein the one or more sources of sound comprise the user of the wearable audio device and one or more speakers in the multi-speaker environment.

### Claim 13 (depends on 11)

The wearable audio device of claim 11, wherein the instructions, when executed by the at least one processor, individually or collectively, cause the wearable audio device to:
estimate a head movement of the user based on data obtained from one or more head sensors associated with the wearable audio device;
determine relative positions of the one or more sources of sound with respect to the head movement of the user;
compute horizontal offset angles and vertical offset angles for the one or more sources of sound;
generate embedding vectors indicating identification of the one or more sources of sound; and
generate the virtual sound source map based on:
the relative positions of the one or more sound sources,
the horizontal offset angles and the vertical offset angles, and
the embedding vectors.

### Claim 14 (depends on 11)

The wearable audio device of claim 11, wherein the metadata comprises the virtual sound source map and the one or more target sources.

### Claim 15 (depends on 11)

The wearable audio device of claim 11, wherein the instructions, when executed by the at least one processor, individually or collectively, cause the wearable audio device to:
estimate, based on the virtual sound source map, directions of conversation of the one or more sources of sound;
estimate a relative movement of a head of the user with respect to an initial head position of the user;
monitor one or more head gestures of the user and classify the one or more head gestures, the classification of the one or more head gestures comprising agreement gestures and disagreement gestures;
determine, using a target detection model, one or more sound source pairs between which a live interaction is present based on the directions of conversation;
generate, using the target detection model, an interaction timeline of the user associated with the one or more sources of sound, based on the one or more sound source pairs, the relative movement, and the one or more head gestures,
wherein the interaction timeline comprises a time duration and a type of interaction of the user associated with the one or more sources of sound, and wherein the type of interaction includes: direct, indirect and passive; and
estimate, using the target detection model, the one or more target sources based on the interaction timeline.

### Claim 16 (depends on 11)

The wearable audio device of claim 11, wherein the instructions, when executed by the at least one processor, individually or collectively, cause the wearable audio device to:
receive, from the electronic device, an amplified audio signal associated with the at least one refined target source; and
play back the amplified audio signal to the user from the wearable audio device.

### Claim 17 (depends on 11)

The wearable audio device of claim 11, wherein the metadata comprises information corresponding to the binaural audio signal, the virtual sound source map, and an interaction timeline of the user.

### Claim 18 (depends on 11)

The wearable audio device of claim 11, wherein the instructions, when executed by the at least one processor, individually or collectively, cause the wearable audio device to:
render and serialize the virtual sound source map in a three-dimensional space;
estimate, using one or more head sensors associated with the wearable audio device, one or more head gestures of the user;
generate the interaction timeline based on the one or more head gestures, a head position of the user, and a head direction of the user; and
update the one or more target sources in response to a change in the head direction of the user.

### Claim 19 (depends on 18)

The wearable audio device of claim 18, wherein the instructions, when executed by the at least one processor, individually or collectively, cause the wearable audio device to:
assign, using a trained AI model, priorities to at least one source of sound based on a direction of conversation with respect to the user; and
select the one or more target sources based on the priorities and the interaction timeline.

### Claim 20

A non-transitory computer-readable recording medium having at least one instruction recorded thereon, that, when executed by at least one processor, individually or collectively, cause the wearable audio device to:
generate, based on a binaural audio signal captured by the microphone, a virtual sound source map indicating a localized position of one or more sources of sound;
estimate one or more target sources;
transmit metadata associated with the wearable audio device to an electronic device coupled to the wearable audio device to cause the electronic device to refine the one or more target sources based on the metadata; and
receive, from the electronic device, a processed audio signal associated with at least one refined target source.
