US 12,640,133 B2Granted

Binaural data sharing in ear-worn devices using neural networks

AssigneeFortell Research Inc.

InventorsIgor Lovchinsky, Nathan Agmon, Philip Meyers, IV, Israel Malkin, Nicholas Morris, Mark Berry

Life of this patent

Estimating the term…

Loading patent…

Abstract

Described herein is binaural data sharing technology for ear-worn devices to improve audio processing performance. Different embodiments may include sharing of various data types, such as processed microphone signals, beamformed signals, neural network products (e.g., masks), and environmental metrics. For beamforming, devices may combine signals from both ears for improved directional selectivity or process separate beamformed signals independently. Devices may be configured to generate identical masks or average mask magnitude portions while preserving device-specific phase components. Neural networks may be trained to handle mixed-latency data, processing current local data with “stale” data from the other device. Environmental metrics like signal-to-noise ratios may be shared for coordinated responses to acoustic conditions. The technology may also apply to integrated devices like eyeglasses.

Claims22

Dependent claims

A system, comprising:

a first ear-worn device comprising:
first neural network circuitry, and
first communication circuitry;
and
a second ear-worn device comprising:
second neural network circuitry, and
second communication circuitry;
wherein:
the first communication circuitry and the second communication circuitry are
configured to communicate over a wireless communication link;
the first neural network circuitry is configured to:
receive one or more first audio signals generated by the first ear-worn device, and
implement one or more first neural network layers, wherein the first neural network circuitry is configured to use the one or more first neural network layers to generate a first neural network product based on the one or more first audio signals;
the second neural network circuitry is configured to:
receive one or more second audio signals generated by the second ear-worn device, and
implement one or more second neural network layers, wherein the second neural network circuitry is configured to use the one or more second neural network layers to generate a second neural network product based on the one or more second audio signals;
the first communication circuitry is configured to:
transmit, to the second communication circuitry over the wireless communication link, first data comprising or originating from the first neural network product, and
receive, from the second communication circuitry over the wireless communication link, second data comprising or originating from the second neural network product;
the first data comprises a first mask and the second data comprises a second mask, or the first data comprises a processed version of the first mask and the second data comprises a processed version of the second mask;
the first ear-worn device is configured to combine the first mask with the second mask, thereby generating a first combined mask;
the first ear-worn device is configured, when combining the first mask with the second mask, to combine a magnitude portion of the first mask with a magnitude portion of the second mask;
and
the first combined mask comprises:
a magnitude portion based on combining the magnitude portion of the first mask with the magnitude portion of the second mask, and
a phase portion based on a phase portion of the first mask.

23 more elements

A system, comprising:

a first ear-worn device comprising:
first neural network circuitry, and
first communication circuitry;
and
a second ear-worn device comprising:
second neural network circuitry, and
second communication circuitry;
wherein:
the first communication circuitry and the second communication circuitry are configured to communicate over a wireless communication link;
the first neural network circuitry is configured to:
receive one or more first audio signals generated by the first ear-worn device, and
implement one or more first neural network layers, wherein the first neural network circuitry is configured to use the one or more first neural network layers to generate a first neural network product based on the one or more first audio signals;
the second neural network circuitry is configured to:
receive one or more second audio signals generated by the second ear-worn device, and
implement one or more second neural network layers, wherein the second neural network circuitry is configured to use the one or more second neural network layers to generate a second neural network product based on the one or more second audio signals;
the first communication circuitry is configured to:
transmit, to the second communication circuitry over the wireless communication link, first data comprising or originating from the first neural network product, and
receive, from the second communication circuitry over the wireless communication link, second data comprising or originating from the second neural network product;
the first data comprises a first mask and the second data comprises a second mask, or the first data comprises a processed version of the first mask and the second data comprises a processed version of the second mask;
the first ear-worn device is configured to compare the first mask with the second mask;
the first ear-worn device further comprises mixing circuitry configured to perform mixing of at least two audio signals, thereby generating an output audio signal;
and
based on the comparison, the mixing circuitry is further configured to modulate weighting of the at least two audio signals in the mixing.

20 more elements

A system, comprising:

a first ear-worn device comprising:
one or more first microphones configured to generate one or more first microphone signals,
first processing circuitry comprising first neural network circuitry, the first processing circuitry configured to process the one or more first microphone signals, thereby generating first data, and
first communication circuitry;
and
a second ear-worn device comprising:
one or more second microphones configured to generate one or more second microphone signals,
second processing circuitry comprising second neural network circuitry, the second processing circuitry configured to process the one or more second microphone signals, thereby generating second data, and
second communication circuitry;
wherein:
the first communication circuitry and the second communication circuitry are configured to communicate over a wireless communication link;
the first communication circuitry is configured to:
transmit the first data to the second communication circuitry over the wireless communication link, and
receive the second data from the second communication circuitry over the wireless communication link;
the second communication circuitry is configured to:
transmit the second data to the first communication circuitry over the wireless communication link, and
receive the first data from the first communication circuitry over the wireless communication link;
the first neural network circuitry is configured to:
implement one or more first neural network layers configured to receive inputs comprising or originating from the first data and the second data;
the second neural network circuitry is configured to:
implement one or more second neural network layers configured to receive the inputs comprising or originating from the first data and the second data;
and
based on the first ear-worn device receiving the second data and the second ear-worn device receiving the first data, the first neural network circuitry and the second neural network circuitry are configured to generate same neural network products having same values.

20 more elements

Dependencies are inferred from the claim text.

Facts

Application: 19/379,332Filed 4 Nov 2025 · us provisional application
Priority: 4 Nov 2024
Published: US 2026/0128028 A17 May 2026
Granted: 26 May 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification9 codes

G10K 11/1752
Using interference effects; Masking sound: masking
H04R 25/507
Using digital signal processing: implemented by neural network or fuzzy logic
H04R 1/1016
Earpieces; Attachments therefor ; Earphones; Monophonic headphones: earpieces of the intra-aural type
H04R 25/552
Using an external connection, either wireless or wired: binaural