Automatic videoconference framing using multiple cameras

A non-transitory computer-readable storage medium having stored thereon instructions that when executed by a processor cause the processor to:

• receive a first video stream of a scene from a first camera;
• receive a second video stream of the scene from a second camera;
• identify a participant of interest in the scene based on the first video stream;
• determine from the first video stream, a first face orientation of the participant of interest relative to the first camera;
• determine from the second video stream, a second face orientation of the participant of interest relative to the second camera;
• rank the first face orientation and the second face orientation based on a percentage of the face of the participant of interest depicted in each of the first video stream and the second video stream;
• and
• send one of the first video stream or the second video stream to a display system based on the rank of the first face orientation and the second face orientation.

A system, comprising:

• a first camera;
• a second camera system comprising at least one camera;
• an electronic processor to:
• receive a first video stream from the first camera;
• receive at least one second video stream, each second video stream being received from one of the at least one cameras in the second camera system;
• receive calibration data relating a coordinate system of the first camera to a coordinate system of each camera in the second camera system;
• identify pixels in the first video stream corresponding to a participant;
• map the pixels in the first video stream to pixels in each second video stream using the calibration data;
• determine a first head pose of the participant in the first video stream;
• determine a second head pose of the participant in each second video stream;
• generate ranking data by ranking the first head pose based on an orientation relative to the first camera and each second head pose based on an orientation relative to each respective camera in the second camera system;
• select as a selected video stream one of the first video stream or one of the at least one second video streams having a higher ranking score in the ranking data;
• and
• send the selected video stream to a remote endpoint.