Spatial audio and avatar control at headset using audio signals

A method comprising:

• capturing, using an acoustic sensor of an array of acoustic sensors, audio data from a human sound source, wherein at least a portion of a face of the human sound source is not within a field of view of an imaging device collocated with the human sound source;
• generating audio information for presentation to a remote user, wherein the audio information is presented to the remote user based at least in part on a location of the remote user, within a virtual representation of a local area, relative to a virtual representation of the human sound source;
• providing the audio information to a headset of the remote user, wherein the headset presents the audio information as part of the virtual representation of the local area to the remote user;
• predicting a facial expression for the portion of the face based at least in part on the captured audio data from the human sound source;
• and
• providing, to the headset of the remote user, visual information associated with a virtual representation of the human sound source in the virtual representation of the local area, wherein the visual information is presented, to the headset of the remote user, such that the facial expression of the human sound source is in sync with the presented audio information.

A computer program product comprising a non-transitory computer-readable storage medium having instructions encoded thereon that, when executed by one or more processors, cause the one or more processors to:

• instruct an acoustic sensor of an array of acoustic sensors to capture audio data from a human sound source, wherein at least a portion of a face of the human sound source is not within a field of view of an imaging device collocated with the human sound source;
• generate audio information for presentation to a remote user, wherein the audio information is presented to the remote user based at least in part on a location of the remote user, within a virtual representation of a local area, relative to a virtual representation of the human sound source;
• provide the audio information to a headset of the remote user, wherein the headset presents the audio information as part of the virtual representation of the local area to the remote user;
• predict a facial expression for the portion of the face based at least in part on the captured audio data from the human sound source;
• and
• provide, to the headset of the remote user, visual information associated with a virtual representation of the human sound source in the virtual representation of the local area, wherein the visual information is presented, to the headset of the remote user, such that the facial expression of the human sound source is in sync with the presented audio information.