Spatial audio and avatar control at headset using audio signals

A computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process comprising:

• receiving audio data captured, by a first computing system, from a human sound source;
• predicting a facial expression;
• causing a second computing system, remote from the first computing system, to play audio, based on the audio data, in relation to a virtual representation of the human sound source;
• and
• causing the second computing system to provide, in a virtual environment and in synchronization with the audio that is played, an additional virtual representation of the predicted expression in relation to the virtual representation of the human sound source, the additional virtual representation being provided responsive to an occluded portion of a face of the human sound source being within a threshold angle of a field of view of the second computing system.

A computing system comprising:

• one or more processors;
• and
• one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising:
• receiving audio data captured, by a first computing system;
• predicting a facial expression;
• causing a second computing system, remote from the first computing system, to play audio, based on the audio data, in relation to a virtual representation of a human sound source;
• and
• causing the second computing system to provide, in a virtual environment and in synchronization with the audio that is played, an additional virtual representation of the predicted expression in relation to the virtual representation of the human sound source, the additional virtual representation being provided responsive to an occluded portion of a face of the human sound source being within a threshold angle of a field of view of the second computing system.