Generating a realistic animated avatar of a user in real-time during a teleconference

A system for generating an animated avatar of a user during a conferencing session, comprising:

• at least one memory;
• and
• at least one hardware processor coupled with the at least one memory and configured, individually or in combination, to:
• collect sensor data from one or more sensors associated with a first user during a conferencing session, wherein the sensor data comprises at least audio data of a voice of the first user;
• generate, using at least one trained AI avatar generation model, an animated avatar of the first user based at least on the collected sensor data, wherein the animated avatar simulates at least one of a physical likeness, one or more facial expressions, one or more speech mannerisms, one or more co-speech gestures, or a voice of the first user;
• and
• display, on a communication device of a second user, the animated avatar of the first user synchronized with the audio data of the voice of the first user during the conferencing session.

A method for training artificial intelligence (AI) avatar generation models for generating an animated avatar of a user, the method comprising:

• storing, in a training database, a plurality of training datasets, wherein each training dataset corresponds to a different visual or audio characteristic of the user, the respective training dataset comprises a corresponding label identifying the visual or audio characteristic, and the respective training dataset further comprises a sequence of images, audio files, or videos;
• filtering, using a filter module, the plurality of training datasets to remove one or more data points, resulting in a plurality of filtered training datasets;
• preprocessing the plurality of filtered training datasets to prepare input data for a plurality of model trainers;
• training, using the plurality of model trainers, a corresponding plurality of AI recognition models, wherein each AI recognition model is trained to predict a corresponding visual or audio characteristic of the user based on a corresponding type of sensor data;
• and
• storing the trained plurality of AI recognition models in an AI avatar generation model database for use in generating an animated avatar of the user during a conferencing session.