Audio-visual sound enhancement

A system, comprising:

• a microphone;
• an audio output device;
• an outward sensor device;
• an inward sensor device;
• a memory storing an application;
• and
• a processor that, when executing the application, is configured to:
• acquire, via the outward sensor device, image information associated with an environment;
• acquire, via the inward sensor device, sensor data associated with a gaze of a user;
• determine a source of interest based on the image information and the sensor data;
• separate a set of audio signals associated with the environment based on the image information to isolate an audio signal associated with the source of interest;
• and
• output the isolated audio signal associated with the source of interest to the user.

One or more non-transitory computer-readable storage media including instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of:

• acquiring image information associated with an environment;
• acquiring, from one or more sensors, sensor data associated with a gaze of a user;
• determining a source of interest based on the image information and the sensor data;
• processing a set of audio signals associated with the environment based on the image information to identify an audio signal associated with the source of interest;
• enhancing the audio signal associated with the source of interest relative to other audio signals in the set of audio signals;
• and
• outputting the enhanced audio signal associated with the source of interest to the user.