Method for enabling synthetic autopilot video functions and for publishing a synthetic video feed as a virtual camera during a video call

method of claim 1:

• further comprising, at the first device, interpreting a first emotion of the first user during the first time period based on facial features of the first user detected in the first sequence of frames;
• and
• wherein retrieving the prerecorded autopilot sequence of facial landmark containers from the memory comprises:
• accessing a set of prerecorded autopilot sequence of facial landmark containers representing the first user exhibiting a set of discrete emotions;
• and
• retrieving the prerecorded autopilot sequence of facial landmark containers, from the set of prerecorded autopilot sequence of facial landmark containers, associated with the first emotion.

The method of claim 12:

• wherein generating a first sequence of facial landmark containers comprises:
• initializing a first facial landmark container, the first sequence of facial landmark containers, for a target frame in the first sequence of frames;
• and
• for each action unit, in a predefined set of action units representing action of human facial muscles:
• detecting a facial region of the first user, depicted in the target frame, containing a muscle associated with the action unit;
• interpreting an intensity of action of the muscle based on a set of features extracted from the facial region depicted in the first sequence of frames;
• and
• representing the intensity of action of the muscle in the first facial landmark container;
• and
• wherein retrieving the prerecorded autopilot sequence of facial landmark containers from the memory comprises retrieving a first prerecorded autopilot facial landmark container representing intensities of actions of muscles, associated with the predefined set of action units, corresponding to the first emotion.

A method comprising:

• during a first time period:
• receiving a first sequence of frames captured by an optical sensor in a first device;
• detecting a face, of the first user, in the first sequence of frames;
• generating a first sequence of facial landmark containers representing facial actions of the face of the first user detected in the first sequence of frames;
• and
• transmitting the first sequence of facial landmark containers to a second device for combination with a first look model, associated with the first user, to generate a first synthetic image feed depicting facial actions of the first user during the first time period, represented in the first sequence of facial landmark containers, according to the first look model;
• detecting a trigger event;
• and
• during a second time period, in response to detecting a first trigger event:
• entering an autopilot mode;
• constructing an autopilot sequence of facial landmark containers based on content excluded from frames captured by the optical sensor during the second time period;
• and
• transmitting the prerecorded autopilot sequence of facial landmarks to a second device for combination with the first look model to generate a second synthetic image feed depicting predefined facial actions, represented in the prerecorded autopilot sequence of facial landmark containers, according to the first look model.

A method comprising:

• during a first time period:
• receiving a first sequence of frames captured by an optical sensor in a first device;
• detecting a face, of a first user, in the first sequence of frames;
• generating a first sequence of facial landmark containers representing facial actions of the face of the first user detected in the first sequence of frames;
• and
• transmitting the first sequence of facial landmark containers to a set of devices, comprising a second device, for combination with local copies of the first look model to generate synthetic image feeds depicting facial actions of the first user during the first time period;
• detecting a first trigger event;
• and
• during a second time period, in response to detecting the first trigger event:
• entering an autopilot mode;
• retrieving a prerecorded autopilot sequence of facial landmark containers from a memory;
• and
• transmitting the prerecorded autopilot sequence of facial landmark containers to the set of devices, comprising the second device, for combination with local copies of the first look model to generate synthetic image feeds depicting facial actions of the first user during the second time period.