Back to results
US 2019/0158733 A1Published application
Optimal view selection method in a video conference
AssigneePolycom, Inc.
InventorsJinwei Feng, Peter Chu
Life of this patent
Estimating the term…
Abstract
A system for ensuring that the best available view of a person's face is included in a video stream when the person's face is being captured by multiple cameras at multiple angles at a first endpoint. The system uses one or more microphone arrays to capture direct-reverberant ratio information corresponding to the views, and determines which view most closely matches a view of the person looking directly at the camera, thereby improving the experience for viewers at a second endpoint.
Claims20
A method for determining an optimal camera view from amongst a plurality of views, the method comprising:
- ascertaining an active talker using a first sound location module, capturing a first view of the active talker using a first camera unit, and deriving a first set of second-order information corresponding to the first view;
- ascertaining the active talker using a second sound location module, capturing a second view of the active talker using a second camera, and deriving a second set of second-order information corresponding to the second view;
- detecting low-frequency energy (LFE) corresponding to the first and second views;
- determining that the LFE corresponding to the first view differs from the LFE corresponding to the second view in excess of a predetermined value;
- including, based on the determination, the LFE corresponding to the first view within the first set of second-order information, and the LFE corresponding to the second view within the second set of second-order information;
- determining an optimal view from amongst the first and second views based, at least in part, on the first set of second-order information and the second set of second-order information;
- and
- including the optimal view in a video stream.
5 more elements
A non-transitory computer readable storage medium storing instructions executable by one or more processors, the instructions comprising instructions to:
- ascertain an active talker using a first sound location module, capture a first view of the active talker using a first camera unit, and derive a first set of second-order information corresponding to the first view;
- ascertain the active talker using a second sound location module, capturing a second view of the active talker using a second camera, and derive a second set of second-order information corresponding to the second view;
- detect low-frequency energy (LFE) corresponding to the first and second views;
- determine that the LFE corresponding to the first view differs from the LFE corresponding to the second view in excess of a predetermined value;
- include, based on the determination, the LFE corresponding to the first view within the first set of second-order information, and the LFE corresponding to the second view within the second set of second-order information;
- determine an optimal view from amongst the first and second views based, at least in part, on the first set of second-order information and the second set of second-order information;
- and
- include the optimal view in a video stream.
5 more elements
A teleconferencing endpoint comprising:
- one or more processors;
- a first camera unit and a first microphone array, the first camera unit and the first microphone array coupled to the one or more processors;
- a second camera unit and second microphone array, the second camera unit and the second microphone array coupled to the one or more processors;
- and
- a non-transitory computer readable storage medium storing instructions executable by the one or more processors, the instructions comprising instructions to:
- capture first audio using the first microphone array, ascertain an active talker using a first sound location module based on the first audio, capture a first view of the active talker using the first camera unit, and derive a first set of second-order information corresponding to the first view;
- capture second audio using the second microphone array, ascertain the active talker using a second sound location module based on second audio, capture a second view of the active talker using the second camera, and derive a second set of second-order information corresponding to the second view;
- detect low-frequency energy (LFE) corresponding to the first audio and the second audio;
- determine that the LFE corresponding to the first audio differs from the LFE corresponding to the second audio in excess of a predetermined value;
- include, based on the determination, the LFE corresponding to the first audio within the first set of second-order information, and the LFE corresponding to the second audio within the second set of second-order information;
- determine an optimal view from amongst the first and second views based, at least in part, on the first set of second-order information and the second set of second-order information;
- and
- include the optimal view in a video stream.
10 more elements
Dependencies are inferred from the claim text.