Back to results
US 12,597,251 B2Granted
Video framing based on tracked characteristics of meeting participants
AssigneeHuddly AS
InventorsVegard Hammer, Jon Tore Hafstad
Life of this patent
Estimating the term…
Abstract
Multi- and single-camera video conferencing systems are disclosed including at least one camera, each configured to generate an overview video output stream representative of a meeting environment, and at least one video processing unit. The video processing unit(s) are configured to automatically analyze the overview video output stream(s) to identify one or more meeting participants and assign respective role designations to the meeting participants based on analysis of the overview video output stream(s). The video processing unit(s) are further configured to generate, and output for display, one or more primary video streams including one or more of the meeting participants, based on the assigned roles.
Claims68
A video conferencing system, comprising:
- at least one camera, each configured to generate an overview video output stream representative of a meeting environment;
- and
- at least one video processing unit configured to:
- automatically analyze the at least one overview video output stream to identify at least a first meeting participant and a second meeting participant represented in the overview video output stream;
- assign a first role designation to the first meeting participant and a second role designation to the second meeting participant based on one or more detected characteristics associated with at least one of the first meeting participant or the second meeting participant, wherein the first role designation is different from the second role designation;
- select a framed representation of the first meeting participant or the second meeting participant based on relative priorities associated with the first role designation and the second role designation and based on an expected speaking duration associated with the first meeting participant or the second meeting participant, wherein the expected speaking duration is based on one or more previous speaking durations associated with the first meeting participant or the second meeting participant;
- and
- generate, as an output, a primary video stream including the selected framed representation of the first meeting participant or the second meeting participant.
5 more elements
A video conferencing system, comprising:
- at least one camera, each configured to generate an overview video output stream representative of a meeting environment;
- and
- at least one video processing unit configured to:
- automatically analyze the at least one overview video output stream to identify at least a first meeting participant represented in the at least one overview video output stream;
- assign a first role designation to the first meeting participant based on one or more detected characteristics associated with the first meeting participant;
- determine, based on the first role designation assigned to the first meeting participant, a likelihood that the first meeting participant will begin to speak;
- perform, in response to the likelihood that the first meeting participant will begin to speak meeting a likeliness condition, one or more actions to prepare the video processing unit to output a primary video stream featuring a framed representation of the first meeting participant for display, wherein the framed representation is based on an expected speaking duration associated with the first meeting participant, and wherein the expected speaking duration is based on one or more previous speaking durations associated with the first meeting participant;
- determine, based on analysis of the at least one overview video output stream, whether the first meeting participant begins to speak;
- and
- in response to a determination that the first meeting participant begins to speak, outputting for display the primary video stream featuring the framed representation of the first meeting participant.
7 more elements
A video conferencing system, comprising:
- at least one camera, each configured to generate an overview video output stream representative of a meeting environment;
- and
- at least one video processing unit configured to:
- automatically analyze the at least one overview video output stream to identify a plurality of meeting participants featured in the at least one overview video output stream;
- identify a plurality of non-speaking meeting participants amongst the identified meeting participants;
- assign a respective role designation to each of the identified non-speaking meeting participants based on one or more detected characteristics associated with each of the non-speaking meeting participants;
- generate, as an output for display, a primary video stream featuring one or more of the non-speaking meeting participants, wherein selection of the one or more non-speaking meeting participants for inclusion in the primary video stream is based on the roles assigned to each of the non-speaking meeting participants;
- identify a speaking meeting participant amongst the identified meeting participants;
- generate, as an output for display, a second primary video stream featuring the speaking meeting participant;
- determine an expected speaking duration associated with the speaking meeting participant based on one or more determined previous speaking durations associated with the speaking meeting participant;
- and
- based on the expected speaking duration, determine when to output the second primary video stream for display.
9 more elements
A video conferencing system, comprising:
- at least one camera, each configured to generate an overview video output stream representative of a meeting environment;
- and
- at least one video processing unit configured to:
- automatically analyze the at least one overview video output stream to identify at least a first meeting participant and a second meeting participant represented in the at least one overview video output stream;
- assign a first role designation to the first meeting participant and a second role designation to the second meeting participant based on one or more detected characteristics associated with at least one of the first meeting participant or the second meeting participant, wherein the first role designation and the second role designation are the same;
- assign a first attribute designation to the first meeting participant and a second attribute designation to the second meeting participant based on one or more detected properties or actions associated with at least one of the first meeting participant or the second meeting participant, wherein the first attribute designation is different from the second attribute designation;
- determine a relative framing priority between the first meeting participant and the second meeting participant based on a first combination, including the first role designation and the first attribute designation, and based on a second combination, including the second role designation and the second attribute designation;
- select a framed representation of the first meeting participant or the second meeting participant based on the determined relative framing priority and based on an expected speaking duration associated with the first meeting participant or the second meeting participant, wherein the expected speaking duration is based on one or more previous speaking durations associated with the first meeting participant or the second meeting participant;
- and
- generate, as an output, a primary video stream including the selected framed representation of the first meeting participant or the second meeting participant.
7 more elements
A video conferencing system, comprising:
- at least one camera configured to generate video output representative of a meeting environment;
- and
- at least one video processing unit configured to:
- automatically analyze the video output to identify at least a first meeting participant and a second meeting participant represented in the video output;
- assign a first role designation to the first meeting participant and a second role designation to the second meeting participant based on one or more detected characteristics associated with at least one of the first meeting participant or the second meeting participant, wherein the first role designation is different from the second role designation;
- and
- generate, as an output, a first primary video stream including a framed representation of the first meeting participant during a first period of time and a second primary video stream including a framed representation of the second meeting participant during a second period of time, wherein the framed representation is based on an expected speaking duration associated with the first meeting participant or the second meeting participant, and wherein the expected speaking duration is based on one or more previous speaking durations associated with the first meeting participant or the second meeting participant;
- wherein relative durations of the first and second periods of time are based on relative priorities associated with the first role designation and the second role designation.
5 more elements
A video conferencing system, comprising:
- at least one camera, each configured to generate an overview video output stream representative of a meeting environment;
- and
- at least one video processing unit configured to:
- automatically analyze the at least one overview video output stream to identify at least a first meeting participant and a second meeting participant represented in the at least one overview video output stream;
- assign a first role designation to the first meeting participant and a second role designation to the second meeting participant based on one or more detected characteristics associated with at least one of the first meeting participant or the second meeting participant;
- determine, based on priorities associated with the first and second role designations relative framing priorities associated with the first meeting participant and the second meeting participant;
- and
- generate, based on the relative framing priorities, one or more primary video streams featuring the first meeting participant and/or the second meeting participant, the one or more primary video streams including at least one framed representation of the first meeting participant or the second meeting participant, wherein the at least one framed representation is based on at least one expected speaking duration associated with the first meeting participant or the second meeting participant, and wherein the at least one expected speaking duration is based on one or more previous speaking durations associated with the first meeting participant or the second meeting participant.
5 more elements
A video processing method performed by at least one video processing unit, the method comprising:
- automatically analyzing at least one overview video output stream representative of a meeting environment generated by at least one camera to identify at least a first meeting participant and a second meeting participant represented in the at least one overview video output stream;
- assigning a first role designation to the first meeting participant and a second role designation to the second meeting participant based on one or more detected characteristics associated with at least one of the first meeting participant or the second meeting participant;
- determining, based on priorities associated with the first and second role designations, relative framing priorities associated with the first meeting participant and the second meeting participant;
- and
- generating as an output, based on the relative framing priorities, one or more primary video streams featuring the first meeting participant and/or the second meeting participant, the one or more primary video streams including at least one framed representation of the first meeting participant or the second meeting participant, wherein the at least one framed representation is based on at least one expected speaking duration associated with the first meeting participant or the second meeting participant, and wherein the at least one expected speaking duration is based on one or more previous speaking durations associated with the first meeting participant or the second meeting participant.
2 more elements
Dependencies are inferred from the claim text.