Systems and methods for multi-user video communication with engagement detection and adjustable fidelity

An electronic device comprising:

• one or more displays;
• one or more non-transitory computer-readable storage media including instructions;
• and
• one or more processors coupled to the storage media, the one or more processors configured to execute the instructions to:
• maintain a video communication between two or more client devices, wherein each client device is associated with a respective video stream in the video communication, and wherein each video stream is associated with a respective match scale measured based on a height of frames of the video stream and a depth of subjects within the frames;
• determine, for each of the video streams, a respective scaling factor and a respective cropping, wherein the respective scaling factor is calculated based on the match scale associated with the corresponding video stream and a target match scale determined from the match scales associated with the video streams associated with the two or more client devices, and wherein the respective cropping is determined based on a uniformity of positions of the subjects within the frames across the video streams associated with the two or more client devices;
• generate, based on the respective scaling factor and cropping of each video stream, a merged video stream from each of the video streams for the video communication;
• and
• send, to one or more of the client devices, instructions for presenting the merged video stream.

A computer-readable non-transitory storage media comprising instructions executable by a processor to:

• maintain a video communication between two or more client devices, wherein each client device is associated with a respective video stream in the video communication, and wherein each video stream is associated with a respective match scale measured based on a height of frames of the video stream and a depth of subjects within the frames;
• determine, for each of the video streams, a respective scaling factor and a respective cropping, wherein the respective scaling factor is calculated based on the match scale associated with the corresponding video stream and a target match scale determined from the match scales associated with the video streams associated with the two or more client devices, and wherein the respective cropping is determined based on a uniformity of positions of the subjects within the frames across the video streams associated with the two or more client devices;
• generate, based on the respective scaling factor and cropping of each video stream, a merged video stream from each of the video streams for the video communication;
• and
• send, to one or more of the client devices, instructions for presenting the merged video stream.