# US 2023/0276182 A1 · Mobile device that provides sound enhancement for hearing device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0276182 A1
- Document ID: US20230276182A1
- Status: Published application
- Kind: A1
- Application number: 18/024,221
- Filed: 2021-07-15
- Published: 2023-08-31
- Earliest claimed: 2020-09-01
- Assignees: Starkey Laboratories, Inc.
- Applicants: Starkey Laboratories, Inc.
- Inventors: Tao Zhang; Daniel MARQUARDT
- Web page: https://patentgrep.com/patent/US20230276182A1

## CPC

- H04R 25/507: {implemented by neural network or fuzzy logic}
- H04R 25/554: {using a wireless connection, e.g. between microphone and amplifier or using Tcoils}
- H04R 25/04: comprising pocket amplifiers
- H04R 25/558: {Remote control, e.g. of amplification, frequency}
- H04R 2225/41: Detection or adaptation of hearing aid parameters or programs to listening situation, e.g. pub, forest
- H04R 2225/43: Signal processing in hearing aids to enhance the speech intelligibility
- H04R 2225/55: Communication between hearing aids and external devices via a network for data exchange
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

A system includes a mobile device that receives an audio signal from a microphone of the mobile device. The mobile device processes the audio signal via a neural network to obtain a speech-enhanced audio signal. The system includes an ear-wearable device comprising a data interface operable to communicate with the external data interface of the mobile device. The ear-wearable device includes an audio processing path coupled to the data interface and is operable to receive the speech-enhanced audio signal and reproduce the speech-enhanced audio in an ear of a user.

## Claims

### Claim 1

A system, comprising:
a mobile device, comprising:
a microphone;
an external data interface; and
a processor coupled to the microphone and the external data interface, the processor configured with instructions to receive an audio signal from the microphone and process the audio signal via a neural network to obtain a speech-enhanced audio signal; and
an ear-wearable device comprising a data interface operable to communicate with the external data interface of the mobile device, the ear-wearable device comprising an audio processing path coupled to the data interface and operable to receive the speech-enhanced audio signal and reproduce the speech-enhanced audio in an ear of a user.

### Claim 2 (depends on 1)

The system of claim 1, wherein the ear-wearable device comprises a sound processor configured to modify the speech enhanced audio to compensate for hearing loss of the user before reproducing the speech-enhanced audio.

### Claim 3 (depends on 1)

The system of claim 1, wherein the ear-wearable device comprises a sensor configured to detect speech of the user, and wherein the ear-wearable device is operable to send a suppression signal to the mobile device via the data interface in response to detecting the speech, the mobile device modifying the speech-enhanced audio signal to reduce interference of the speech with the speech-enhanced audio signal in response to the suppression signal.

### Claim 4 (depends on 3)

The system of claim 3, wherein modifying the speech enhanced audio signal comprises suppressing the speech-enhanced audio signal.

### Claim 5 (depends on 3)

The system of claim 3, wherein the audio processing path comprises a second neural network that detects the speech of the user.

### Claim 6 (depends on 1)

The system of claim 1, wherein the ear-wearable device comprises a sensor configured to detect an ambient audio signal, and wherein the ear-wearable device is operable to send an ambient descriptor signal that provides at least one of a classification of the ambient audio signal and an estimate of background noise level, the mobile device modifying the speech-enhanced audio signal in response to the ambient descriptor signal.

### Claim 7 (depends on 6)

The system of claim 6, wherein the neural network of the mobile device comprises two or more neural networks, and wherein the processor of the mobile device is further operable to select one of the two or more neural networks to produce the speech enhanced audio signal based on the classification of the ambient descriptor signal received from the ear-wearable device.

### Claim 8 (depends on 1)

The system of claim 1, wherein the neural network comprises any of a feed-forward neural network, a recurrent neural network, and a convolutional neural network.

### Claim 9 (depends on 1)

The system of claim 1, wherein processing the audio signal via the neural network to obtain the speech-enhanced audio signal comprises:
transforming the audio signal from a time domain signal to a frequency domain signal;
mapping features of the frequency domain signal to an input layer of the neural network;
producing a ratio mask from the neural network and apply the ratio mask to the frequency domain signal; and
inverse-transforming the masked frequency domain signal to a time domain to obtain the speech-enhanced signal.

### Claim 10 (depends on 9)

The system of claim 9, wherein processing the audio signal via the neural network to obtain the speech-enhanced audio signal further comprises
performing side-chain processing on the audio signal to determine disturbances to the audio signal; and
using an output of the side-chain processing to perform post processing on the ratio masked frequency domain signal before the inverse-transform.

### Claim 11 (depends on 10)

The system of claim 10, wherein the side-chain processing comprises own-voice detection of speech of the user using the microphone of the mobile device and a second microphone of the ear-wearable device, the own-voice detection based on at least one of phase differences, level differences, and coherence between the microphone and the second microphone.

### Claim 12 (depends on 10)

The system of claim 10, wherein the side-chain processing comprises at least one of environment detection and background noise level estimation.

### Claim 13 (depends on 1)

The system of claim 1, wherein processing the audio signal via the neural network to obtain the speech-enhanced audio signal comprises:
transforming the audio signal from a time domain signal to a latent representation;
mapping features of the latent representation to an input layer of the neural network; and
inverse-transforming an output of the neural network to the speech-enhanced signal.

### Claim 14

A computer-readable medium storing instructions operable by a processor of a mobile device to perform:
coupling the mobile device to an ear-wearable device;
receive an audio signal from a microphone of the mobile device;
processing the audio signal via a neural network to obtain a speech-enhanced audio signal; and
sending the speech-enhanced audio to an ear-wearable device, the ear-wearable device receiving the speech-enhanced audio signal and reproducing the speech-enhanced audio in an ear of a user.

### Claim 15

A method, comprising:
receiving an audio signal from a microphone of a mobile device;
processing the audio signal via a neural network operable on a processor of the mobile device to obtain a speech-enhanced audio signal;
sending the speech-enhanced audio signal to a data interface of an ear-wearable device; and
reproducing the speech-enhanced audio in an ear of a user via an audio processing path of the ear-wearable device.
