US 11,115,625 B1 · Positional audio metadata generation

Abstract

At a video conference endpoint including a camera, a microphone array, and one or more microphone assemblies, the video conference endpoint may divide a video output of the camera into one or more tracking sectors and detect a head position for each participant in the video output. The video conference endpoint may determine within which tracking sector each detected head position is located. The video conference endpoint may determine active sound source positions of the actively speaking participants based on sound being detected or captured by the microphone array and microphone assemblies, and may determine within which tracking sector the active sound source positions are located. For each tracking sector that contains an active sound source position, the video conference endpoint may update the positional audio metadata for that particular tracking sector based on the active sound source positions and the detected head positions located in that tracking sector.

Claims20

Dependent claims

A method comprising:

at a video conference endpoint including a camera and a microphone array:
dividing a video output of the camera into one or more tracking sectors;
detecting a head position for each participant of one or more participants in the video output of the camera;
determining, for each detected head position, whether the detected head position is located within a tracking sector of the one or more tracking sectors of the video output of the camera;
determining one or more active sound source positions of one or more actively speaking participants of the one or more participants based on sound from the one or more actively speaking participants being detected by the microphone array;
determining whether any of the one or more active sound source positions are located in the tracking sector of the one or more tracking sectors;
and
if any of the one or more active sound source positions are located in the tracking sector of the one or more tracking sectors, updating positional audio metadata for the tracking sector based on the one or more active sound source positions located in the tracking sector and the detected head positions located within the tracking sector.

5 more elements

An apparatus comprising:

a camera;
a microphone array;
and
a processor coupled to the microphone array and the camera, and configure to:
divide a video output of the camera into one or more tracking sectors;
detect a head position for each participant of one or more participants in the video output of the camera;
determine, for each detected head position, whether the detected head position is located within a tracking sector of the one or more tracking sectors of the video output of the camera;
determine one or more active sound source positions of one or more actively speaking participants of the one or more participants based on sound from the one or more actively speaking participants being detected by the microphone array;
determine whether any of the one or more active sound source positions are located in the tracking sector of the one or more tracking sectors;
and
if any of the one or more active sound source positions are located in the tracking sector of the one or more tracking sectors, update positional audio metadata for the tracking sector based on the one or more active sound source positions located in the tracking sector and the detected head positions located within the tracking sector.

8 more elements

One or more non-transitory computer readable storage media encoded with instructions that, when executed by a processor, cause the processor to:

divide a video output of a camera of a video conference endpoint into one or more tracking sectors;
detect a head position for each participant of one or more participants in the video output of the camera;
determine, for each detected head position, whether the detected head position is located within a tracking sector of the one or more tracking sectors of the video output of the camera;
determine one or more active sound source positions of one or more actively speaking participants of the one or more participants based on sound from the one or more actively speaking participants being detected by a microphone array of the video conference endpoint;
determine whether any of the one or more active sound source positions are located in the tracking sector of the one or more tracking sectors;
and
if any of the one or more active sound source positions are located in the tracking sector of the one or more tracking sectors, update positional audio metadata for the tracking sector based on the one or more active sound source positions located in the tracking sector and the detected head positions located within the tracking sector.

4 more elements

Dependencies are inferred from the claim text.

Positional audio metadata generation

Life of this patent

Positional audio metadata generation

Life of this patent

Abstract

Claims20