# US 2021/0243536 A1 · Hearing device with neural network-based microphone signal processing > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0243536 A1 - Document ID: US20210243536A1 - Status: Published application - Kind: A1 - Application number: 17/302,102 - Filed: 2021-04-23 - Published: 2021-08-05 - Earliest claimed: 2016-04-06 - Assignees: Starkey Laboratories, Inc. - Applicants: Starkey Laboratories, Inc. - Inventors: Buye Xu; Ivo Merks; Frederic Philippe Denis Mustiere - Web page: https://patentgrep.com/patent/US20210243536A1 ## CPC - H04R 25/507: {implemented by neural network or fuzzy logic} - A61N 1/36036: {of the outer, middle or inner ear} - H04R 25/407: {Circuits for combining signals of a plurality of transducers} - H04R 25/552: {Binaural} - H04R 1/1008: {Earpieces of the supra-aural or circum-aural type} - H04R 5/033: Headphones for stereophonic communication {(details thereof, e.g. relating to batteries, cables or control elements H04R1/10)} - H04R 25/554: {using a wireless connection, e.g. between microphone and amplifier or using Tcoils} - H04R 2225/43: Signal processing in hearing aids to enhance the speech intelligibility - H04R 2460/13: Hearing devices using bone conduction transducers - H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K) - A61N: ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY (measurement of bioelectric currents A61B; surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body A61B18/00; anaesthetic apparatus in general A61M; incandescent lamps H01K; infrared radiators for heating H05B) ## Abstract A hearing system performs nonlinear processing of signals received from a plurality of microphones using a neural network to enhance a target signal in a noisy environment. In various embodiments, the neural network can be trained to improve a signal-to-noise ratio without causing substantial distortion of the target signal. An example of the target sound includes speech, and the neural network is used to improve speech intelligibility. ## Claims ### Claim 1 A system for transmitting an output sound to a listener using hearing devices configured to be worn by the listener, the hearing devices configured to produce the output sound using an output signal and including microphones configured to receive input sounds and to produce microphone signals using the input sounds, the system comprising: a control circuit configured to produce the output signal using a processed signal and including a neural network configured to receive the microphone signals and to control directionality of the microphones by processing the microphone signals using a nonlinear signal processing algorithm to produce the processed signal. ### Claim 2 (depends on 1) The system of claim 1, wherein the neural network comprises a nonlinear signal processing path and a linear signal processing path. ### Claim 3 (depends on 2) The system of claim 2, wherein the nonlinear signal processing path comprises a nonlinear hidden layer, and the linear signal processing path is formed by a shortcut connection bypassing the nonlinear hidden layer. ### Claim 4 (depends on 2) The system of claim 2, wherein the linear signal processing path comprises a linear hidden layer configured to receive the microphone signals and to produce a linear output signal, the nonlinear signal processing path comprises a nonlinear hidden layer configured to receive the linear output signal and to produce a nonlinear output signal, and the neural network further comprises an output layer configured to produce the processed signal using the linear output signal and the nonlinear output signal. ### Claim 5 (depends on 1) The system of claim 1, comprising an external device external to the hearing devices and configured to be communicatively coupled to the hearing devices, the external device including the control circuit. ### Claim 6 (depends on 5) The system of claim 5, further comprising one or more remote microphones external to the hearing devices and configured to produce one or more remote microphone signals, and wherein the neural network is configured to receive the microphone signals and the one or more remote microphone signals and to control the directionality of the microphones by processing the microphone signals and the one or more remote microphone signals using the nonlinear signal processing algorithm to produce the processed signal. ### Claim 7 (depends on 1) The system of claim 1, wherein the target sound includes speech, and the neural network is configured to improve speech intelligibility. ### Claim 8 (depends on 7) The system of claim 7, wherein the neural network is configured to be adaptive to a changing environment in which the hearing devices are used. ### Claim 9 A system for transmitting an output sound to a listener, the system comprising: hearing devices configured to be worn by the listener and configured to produce an output sound using an output signal, the hearing devices including microphones configured to receive input sounds and to produce microphone signals using the input sounds; and a control circuit configured to produce the output signal using a processed signal and including a neural network configured to receive the microphone signals and to control directionality of the microphones by processing the microphone signals using a nonlinear signal processing algorithm to produce the processed signal. ### Claim 10 (depends on 9) The system of claim 9, comprising an external device external to the hearing devices and configured to be communicatively coupled to the hearing devices, the external device including the control circuit. ### Claim 11 (depends on 10) The system of claim 10, wherein the neural network comprises: a nonlinear signal processing path including a nonlinear hidden layer; and a linear signal processing path formed by a shortcut connection bypassing the nonlinear hidden layer. ### Claim 12 (depends on 10) The system of claim 10, further comprising one or more remote microphones external to the hearing devices and configured to produce one or more remote microphone signals, and wherein the neural network is configured to receive the microphone signals and the one or more remote microphone signals and to control the directionality of the microphones by processing the microphone signals and the one or more remote microphone signals using the nonlinear signal processing algorithm to produce the processed signal. ### Claim 13 A method for transmitting an output sound to a listener, the method comprising: receiving microphone signals produced by microphones in hearing devices worn by the listener, the microphone signals representing an input sound received by the microphones; processing the microphone signals to control directionality of the microphones using a neural network of a control circuit, the processing including processing the microphone signals using a nonlinear signal processing algorithm to produce a processed signal; producing an output signal based on the processed signal using the control circuit; and producing an output sound based on the output signal using the hearing devices. ### Claim 14 (depends on 13) The method of claim 13, wherein processing the microphone signals comprises processing the microphone signals through a nonlinear signal processing path and a linear signal processing path of the neural network, the nonlinear signal processing path including a nonlinear hidden layer, the linear signal processing path formed by a shortcut connection bypassing the nonlinear hidden layer. ### Claim 15 (depends on 13) The method of claim 13, wherein using the neural network of the control circuit comprises using the network in the control circuit of an external device external to the hearing devices and configured to be communicatively coupled to the hearing devices. ### Claim 16 (depends on 15) The method of claim 15, further comprising: receiving one or more remote microphone signals from one or more remote microphones external to the hearing devices; and processing the microphone signals and the one or more remote microphone signals to control the directionality of the microphones using the neural network of the control circuit. ### Claim 17 (depends on 13) The method of claim 13, wherein processing the microphone signals comprises processing the microphone signals to improve speech intelligibility. ### Claim 18 (depends on 17) The method of claim 17, further comprising training the neural network to process the microphone signals for a desirable balance between a signal-to-noise ratio (SNR) and distortion of a target sound included in the input sound. ### Claim 19 (depends on 18) The method of claim 18, wherein training the neural network comprises training the neural network using a synthesized target sound in a noisy condition and a specified cost function. ### Claim 20 (depends on 19) The method of claim 19, wherein training the neural network comprises training the neural network using a clean speech in a babble noise condition as the synthesized target sound in the noisy condition, and a mean squared error as the cost function.