# US 2022/0086592 A1 · Spatial audio array processing system and method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0086592 A1
- Document ID: US20220086592A1
- Status: Published application
- Kind: A1
- Application number: 17/539,082
- Filed: 2021-11-30
- Published: 2022-03-17
- Earliest claimed: 2019-09-19
- Assignees: Wave Sciences, LLC
- Applicants: Wave Sciences, LLC
- Inventors: James Keith McElveen; Gregory S. Nordlund, JR.; Leonid Krasny
- Web page: https://patentgrep.com/patent/US20220086592A1

## CPC

- H04S 7/307: {Frequency adjustment, e.g. tone control (H04S7/301 takes precedence)}
- H04R 1/406: {microphones}
- H04R 3/005: {for combining the signals of two or more microphones (specially adapted for hearing aids H04R25/407)}
- H04S 7/303: {Tracking of listener position or orientation}
- H04S 7/305: {Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence)}
- H04S 2400/15: Aspects of sound capture and related signal processing for recording or reproduction
- H04S: STEREOPHONIC SYSTEMS
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

A spatial audio processing system operable to enable audio signals to be spatially extracted from, or transmitted to, discrete locations within an acoustic space. Embodiments of the present disclosure enable an array of transducers being installed in an acoustic space to combine their signals via inverting physical and environmental models that are measured, learned, tracked, calculated, or estimated. The models may be combined with a whitening filter to establish a cooperative or non-cooperative information-bearing channel between the array and one or more discrete, targeted physical locations in the acoustic space by applying the inverted models with whitening filter to the received or transmitted acoustical signals. The spatial audio processing system may utilize a model of the combination of direct and indirect reflections in the acoustic space to receive or transmit acoustic information, regardless of ambient noise levels, reverberation, and positioning of physical interferers.

## Claims

### Claim 1

A method for spatial audio processing comprising:
receiving, with at least one wearable sensor, sensor data corresponding to a direction of a user's head within an acoustic environment;
determining, with at least one processor, at least one source location within the acoustic environment based at least in part on the sensor data;
receiving, with an audio processor, an audio input comprising audio signals captured within the acoustic environment;
converting, with the audio processor, the audio input from a time domain to a frequency domain according to at least one transform function;
determining, with the audio processor, at least one acoustic propagation model for at least one source location;
processing, with the audio processor, the audio input according to the at least one acoustic propagation model to spatially filter at least one target audio signal from one or more non-target audio signals, wherein the at least one target audio signal corresponds to the at least one source location within the acoustic environment; and
applying, with the audio processor, a whitening filter to a spatially filtered target audio signal to derive at least one separated audio output signal.

### Claim 2 (depends on 1)

The method of claim 1 wherein the at least one transform function is selected from the group consisting of Fourier transform, Fast Fourier transform, Short Time Fourier transform and modulated complex lapped transform.

### Claim 3 (depends on 1)

The method of claim 1 wherein the audio input comprises a training audio input.

### Claim 4 (depends on 1)

The method of claim 1 wherein the acoustic environment comprises a waveguide location.

### Claim 5 (depends on 1)

The method of claim 1 further comprising rendering, with the audio processor, an audio file comprising the at least one separated audio output signal.

### Claim 6 (depends on 4)

The method of claim 4 further comprising rendering, with at least one loudspeaker, an audio output comprising the at least one separated audio output signal.

### Claim 7 (depends on 6)

The method of claim 6 wherein the at least one loudspeaker is incorporated within a loudspeaker array.

### Claim 8 (depends on 7)

The method of claim 7 wherein the loudspeaker array corresponds to the waveguide location.

### Claim 9 (depends on 1)

The method of claim 1 wherein the audio input comprises two or more channels of audio input data.

### Claim 10 (depends on 9)

The method of claim 9 wherein each channel in the two or more channels of audio input data corresponds to a transducer located in the acoustic environment.

### Claim 11 (depends on 1)

The method of claim 1 further comprising determining, with the audio processor, the at least one source location according to at least one training audio input.

### Claim 12

A spatial audio processing system, comprising:
at least one wearable sensor configured to receive at least one sensor input corresponding to a movement and direction of a user's head;
a processing device comprising an audio processing module configured to receive an audio input comprising acoustic audio signals captured within an acoustic environment; and
at least one non-transitory computer readable medium communicably engaged with the processing device and having instructions stored thereon that, when executed, cause the processing device to perform one or more audio processing operations, the one or more audio processing operations comprising:
receiving sensor data corresponding to the direction of the user's head within the acoustic environment;
determining at least one source location within the acoustic environment based at least in part on the sensor data;
converting the audio input from a time domain to a frequency domain according to at least one transform function;
determining at least one acoustic propagation model for the at least one source location within the acoustic environment;
processing the audio input according to the at least one acoustic propagation model to spatially filter at least one target audio signal from one or more non-target audio signals, wherein the at least one target audio signal corresponds to the at least one source location; and
applying a whitening filter to a spatially filtered target audio signal to derive at least one separated audio output signal.

### Claim 13 (depends on 12)

The system of claim 12 wherein the at least one transform function is selected from the group consisting of Fourier transform, Fast Fourier transform, Short Time Fourier transform and modulated complex lapped transform.

### Claim 14 (depends on 12)

The system of claim 12 further comprising two or more transducers communicably engaged with the processing device.

### Claim 15 (depends on 14)

The system of claim 14 wherein each transducer in the two or more transducers comprises a separate audio input or output channel.

### Claim 16 (depends on 12)

The system of claim 12 wherein the one or more audio processing operations further comprise rendering an audio file comprising the at least one separated audio output signal.

### Claim 17

A method for spatial audio processing comprising:
receiving, with at least one camera, a live video feed of an acoustic environment;
displaying, on at least one display device, the live video feed of the acoustic environment;
selecting, with at least one input device, an audio source within the live video feed;
determining, with at least one processor, at least one source location within the acoustic environment based at least in part on the selected audio source within the live video feed;
receiving, with an audio processor, an audio input comprising audio signals captured within the acoustic environment;
converting, with the audio processor, the audio input from a time domain to a frequency domain according to at least one transform function;
determining, with the audio processor, at least one acoustic propagation model for the at least one source location;
processing, with the audio processor, the audio input according to the at least one acoustic propagation model to spatially filter at least one target audio signal from one or more non-target audio signals, wherein the at least one target audio signal corresponds to the at least one source location within the acoustic environment; and
applying, with the audio processor, a whitening filter to a spatially filtered target audio signal to derive at least one separated audio output signal.

### Claim 18 (depends on 17)

The method of claim 17 wherein the at least one transform function is selected from the group consisting of Fourier transform, Fast Fourier transform, Short Time Fourier transform and modulated complex lapped transform.

### Claim 19 (depends on 17)

The method of claim 17 further comprising rendering, with the audio processor, an audio file comprising the at least one separated audio output signal.

### Claim 20 (depends on 17)

The method of claim 17 wherein the audio input comprises two or more channels of audio input data.
