Method, system, and medium for enhancing a 3d image during electronic communication

A system comprising:

• a network interface;
• a processor communicatively coupled to the network interface;
• and
• a non-transitory computer readable medium communicatively coupled to the processor and having stored thereon computer program code that is executable by the processor and that, when executed by the processor, causes the processor to perform a method comprising:
• obtaining visual modeling data and verbal communication data of a conference participant in a current time frame as a current sample;
• encoding the current sample into a first vector comprising a plurality of states for the current sample as an encoded result, wherein the plurality of states comprises one or more of texture, shape, facial expression, head movement, landmark position, speech, or script, and wherein each of the plurality of states is characterized by a plurality of elements;
• updating at least one of the plurality of states by inputting the encoded result to a generative artificial intelligence model as a prompt;
• obtaining the updated plurality of states as a predicted result;
• and
• updating the visual modeling data based on the predicted result for rendering a three-dimensional (3D) representation of the conference participant in the current time frame or in one or more future time frames.

A non-transitory computer readable medium having encoded thereon computer program code that is executable by a processor and that, when executed by the processor, causes the processor to perform a method comprising:

• obtaining visual modeling data and verbal communication data of a conference participant in a current time frame as a current sample;
• encoding the current sample into a first vector comprising a plurality of states for the current sample as an encoded result, wherein the plurality of states comprises one or more of texture, shape, facial expression, head movement, landmark position, speech, or script, and wherein each of the plurality of states is characterized by a plurality of elements;
• updating at least one of the plurality of states by inputting the encoded result to a generative artificial intelligence model as a prompt;
• obtaining the updated plurality of states as a predicted result;
• and
• updating the visual modeling data based on the predicted result for rendering a three-dimensional (3D) representation of the conference participant in the current time frame or in one or more future time frames.