Multi-party optimization for audiovisual enhancement

A computing system configured for selectively applying audiovisual enhancement functions to an audiovisual communications stream, the computing system comprising:

• a sender computing system comprising one or more processors and one or more memory devices storing computer readable instructions that, when implemented, cause the one or more processors to perform operations, the operations comprising:
• transmitting an audiovisual communication stream to a receiver computing system;
• receiving, from the receiver computing system, one or more receiver perception feedback signals;
• applying one or more audiovisual enhancement functions to the audiovisual communication stream based at least in part on the one or more receiver perception feedback signals, wherein the one or more receiver perception feedback signals are generated by a gaze detection model and the one or more audiovisual enhancement functions comprise a video fidelity adjustment and/or an audio fidelity adjustment;
• and
• responsive to the one or more receiver perception feedback signals indicating that the gaze of a user operating the receiver computing system is focused outside a display region on which a visual component of the audiovisual communication stream is presented, applying, by the sender computer system, the one or more audiovisual enhancement functions to decrease a fidelity of the visual component of the audiovisual communication stream and/or to increase a fidelity of an audio component of the audiovisual communication stream.

A computing system configured for selectively applying audiovisual enhancement functions to an audiovisual communications stream, the computing system comprising:

• a receiver computing system comprising one or more processors and one or more memory devices storing computer readable instructions that, when implemented, cause the one or more processors to perform operations, the operations comprising
• receiving, from a sender computing system, an audiovisual communication stream;
• obtaining one or more receiver perception feedback signals associated with the audiovisual communication stream, the one or more receiver perception feedback signals obtained as output from one or more receiver perception feedback models comprising a gaze detection model;
• and
• transmitting the one or more receiver perception feedback signals to the sender computing system, the one or more receiver perception feedback signals indicating that a gaze of a user is on a particular spatial region of a visual component of the audiovisual communication stream;
• and
• receiving, from the sender computing system, a modified audiovisual communication stream, the modified audiovisual communication stream including an audio fidelity adjustment to a part of an audio component of the audiovisual communication stream corresponding to the particular spatial region.

A computer-implemented method for selectively applying audiovisual enhancement functions in a multiparty communication, the method comprising:

• receiving, by a receiver computing system, at least one audiovisual communication stream comprising a plurality of audiovisual communication channels from a plurality of sender computing systems;
• obtaining, by the receiver computing system, a plurality of receiver perception feedback signals respectively associated with each of the plurality of audiovisual communication channels, the one or more receiver perception feedback signals obtained as output from one or more receiver perception feedback models comprising a gaze detection model at the receiver computing system and descriptive of perception of a respective audiovisual communication channel by a user operating the receiver computing system;
• and
• transmitting, by the receiver computing system, the plurality of receiver perception feedback signals to a respective sender computing system of the plurality of sender computing systems, the one or more receiver perception feedback signals indicating that the gaze of the user operating the receiver computing system is focused outside a display region on which a visual component of the audiovisual communication stream is presented;
• and
• receiving, from the sender computing system, at least one modified audiovisual communication stream, the modified audiovisual communication stream including a decreased fidelity of the visual component of the audiovisual communication stream and/or an increased fidelity of an audio component of the audiovisual communication stream.