Hearing protection and situational awareness system

A system for providing hearing protection and situational awareness, the system comprising:

• a wearable device comprising hearing members, wherein the hearing members comprise a left hearing member configured to be disposed on a left ear of a user, and a right hearing member configured to be disposed on a right ear of the user;
• one or more speakers disposed at an inner location of each of the hearing members of the wearable device;
• a sound source localization unit operably coupled to the hearing members of the wearable device, the sound source localization unit comprising:
• one or more beamformers, each configured to generate one or more acoustic beam patterns pointing to one or more sound source directions;
• and
• an array of outer microphones disposed at outer locations of the hearing members of the wearable device, wherein the array of outer microphones is configured to capture sound from at least one of the one or more sound source directions of the one or more acoustic beam patterns, outside of the hearing members, and wherein each of the one or more beamformers is configured to output a sound track, each sound track associated with the captured sound in the at least one of the one or more sound source directions of the one or more acoustic beam patterns;
• one or more inner microphones disposed at an inner location of the each of the hearing members of the wearable device, wherein the one or more inner microphones are configured to facilitate active noise cancelation;
• and
• a computation unit disposed in one or more of the hearing members of the wearable device and operably coupled to the array of outer microphones and the one or more inner microphones of the wearable device, the computation unit comprising:
• at least one processor;
• a memory unit operably and communicatively coupled to the at least one processor, wherein the memory unit is configured to store a database of head-related transfer function filters, and computer program instructions defined by a plurality of signal processing modules and executable by the at least one processor;
• and
• the signal processing modules comprising:
• at least one three-dimensional sound generator configured to:
• for each of the one or more beamformers, retrieve a pair of head-related transfer function filters associated with the one or more sound source directions of the one or more acoustic beam patterns generated by the each of the one or more beamformers, from the database;
• apply the retrieved pair of head-related transfer function filters to the output sound track from the each of the one or more beamformers, to generate two filtered sound tracks for the left ear and the right ear, respectively, wherein each of the two filtered sound tracks represents the sound captured from the at least one of the one or more sound source directions of the one or more acoustic beam patterns and comprises a cue of the at least one of the one or more sound source directions;
• and
• combine the filtered sound tracks generated for the left ear into a left sound channel to drive at least one of the one or more speakers in the left hearing member of the wearable device, and combine the filtered sound tracks generated for the right ear into a right sound channel to drive at least one of the one or more speakers in the right hearing member of the wearable device, thereby generating a three-dimensional binaural sound comprising cues of the sound source directions, wherein the one or more speakers in the each of the hearing members of the wearable device reproduce real sound outside of the wearable device;
• one or more active noise cancelation units configured to cancel noise captured by the one or more inner microphones and to cancel background noise captured by the array of outer microphones, for improving hearing protection;
• and
• an automatic gain control unit configured to control a level of output sound of the one or more speakers by selectively increasing gain when a level of the output sound is substantially low for hearing enhancement and situational awareness, and reducing the gain when the level of the output sound is substantially high for hearing protection.

An artificial intelligence (AI)-based system for providing hearing protection and situational awareness, the AI-based system comprising:

• a wearable device comprising hearing members, wherein the hearing members comprise a left hearing member configured to be disposed on a left ear of a user, and a right hearing member configured to be disposed on a right ear of the user;
• one or more speakers disposed at an inner location of each of the hearing members of the wearable device;
• a sound source localization unit operably coupled to the hearing members of the wearable device, the sound source localization unit comprising:
• one or more beamformers, each configured to generate one or more acoustic beam patterns pointing to one or more sound source directions;
• and
• an array of outer microphones disposed at outer locations of the hearing members of the wearable device, wherein the array of outer microphones is configured to capture sound from at least one of the one or more sound source directions of the one or more acoustic beam patterns, outside of the hearing members, and wherein each of the one or more beamformers is configured to output a sound track, each sound track associated with the captured sound in the at least one of the one or more sound source directions of the one or more acoustic beam patterns;
• one or more inner microphones disposed at an inner location of the each of the hearing members of the wearable device, wherein the one or more inner microphones are configured to facilitate active noise cancelation;
• and
• a computation unit disposed in one or more of the hearing members of the wearable device and operably coupled to the array of outer microphones and the one or more inner microphones of the wearable device, the computation unit comprising:
• at least one processor;
• a memory unit operably and communicatively coupled to the at least one processor, wherein the memory unit is configured to store a database of head-related transfer function filters, and computer program instructions defined by a plurality of signal processing modules and executable by the at least one processor;
• and
• the signal processing modules comprising:
• at least one three-dimensional sound generator configured to:
• for each of the one or more beamformers, retrieve a pair of head-related transfer function filters associated with the one or more sound source directions of the one or more acoustic beam patterns generated by the each of the one or more beamformers, from the database;
• apply the retrieved pair of head-related transfer function filters to the output sound track from the each of the one or more beamformers, to generate two filtered sound tracks for the left ear and the right ear, respectively, wherein each of the two filtered sound tracks represents the sound captured from the at least one of the one or more sound source directions of the one or more acoustic beam patterns and comprises a cue of the at least one of the one or more sound source directions;
• and
• combine the filtered sound tracks generated for the left ear into a left sound channel to drive at least one of the one or more speakers in the left hearing member of the wearable device, and combine the filtered sound tracks generated for the right ear into a right sound channel to drive at least one of the one or more speakers in the right hearing member of the wearable device, thereby generating a three-dimensional binaural sound comprising cues of the sound source directions, wherein the one or more speakers in the each of the hearing members of the wearable device reproduce real sound outside of the wearable device;
• an AI-enabled sound identification module configured to automatically recognize characteristics of the captured sound and identify categories of the captured sound using one or more pretrained AI models;
• and
• a voice assistant operably coupled to the one or more speakers and, in communication with the AI-enabled sound identification module and the sound source localization unit, configured to determine one or more sound source locations from the output sound track of the each of the one or more beamformers and to report the identified categories of the captured sound and a direction of the captured sound with voice assistance via one of the one or more speakers and one or more other interface devices.