Embedded neural codec for low latency speech enhancement and automatic speech recognition in hearables

An apparatus, comprising:

• one or more network interfaces to communicate with a network;
• a processor coupled to the one or more network interfaces and configured to execute one or more processes;
• and
• a memory configured to store a process that is executable by the processor, the process comprising:
• receiving, at a hearable device, an auditory signal;
• encoding, at the hearable device, the auditory signal into a compressed vector representation of the auditory signal;
• decoding, at the hearable device using a speech enhancement decoder, the compressed vector representation of the auditory signal into denoised speech;
• outputting, from the hearable device, the denoised speech;
• and
• transmitting, from the hearable device, an unprocessed vector representation of the auditory signal to a processing device to cause the processing device to decode, using a speech recognition decoder, the compressed vector representation of the auditory signal.

A tangible, non-transitory, computer-readable medium storing program instructions that cause a device to execute a process comprising:

• receiving, at a hearable device, an auditory signal;
• encoding, at the hearable device, the auditory signal into a compressed vector representation of the auditory signal;
• decoding, at the hearable device using a speech enhancement decoder, the compressed vector representation of the auditory signal into denoised speech;
• outputting, from the hearable device, the denoised speech;
• and
• transmitting, from the hearable device, an unprocessed vector representation of the auditory signal to a processing device to cause the processing device to decode, using a speech recognition decoder, the compressed vector representation of the auditory signal.