# US 2025/0061916 A1 · Neural acoustic modeling for an audio environment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0061916 A1
- Document ID: US20250061916A1
- Status: Published application
- Kind: A1
- Application number: 18/806,864
- Filed: 2024-08-16
- Published: 2025-02-20
- Earliest claimed: 2023-08-16
- Assignees: Shure Acquisition Holdings, Inc.
- Applicants: Shure Acquisition Holdings, Inc.
- Inventors: Paul Becke; Daniel Law; Nathan Seitz
- Web page: https://patentgrep.com/patent/US20250061916A1

## CPC

- G10L 25/30: using neural networks
- G10L 25/48: specially adapted for particular use
- G06N 3/045: Combinations of networks
- G06N 3/08: Learning methods
- G06T 15/005: {General purpose rendering architectures}
- G06T 15/06: Ray-tracing
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G10L 13/00: Speech synthesis; Text to speech systems
- H04S 7/30: {Control circuits for electronic adaptation of the sound field}
- H04S 7/305: {Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence)}
- G06N 3/04: Architecture, e.g. interconnection topology
- G10K 15/08: Arrangements for producing a reverberation or echo sound {(modifying acoustic properties to change reverberation time G10K11/002)}
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2400/15: Aspects of sound capture and related signal processing for recording or reproduction
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04S: STEREOPHONIC SYSTEMS
- G10K: SOUND-PRODUCING DEVICES; METHODS OR DEVICES FOR PROTECTING AGAINST, OR FOR DAMPING, NOISE OR OTHER ACOUSTIC WAVES IN GENERAL; ACOUSTICS NOT OTHERWISE PROVIDED FOR

## Abstract

Techniques are disclosed herein for providing neural acoustic modeling for an audio environment. Examples may include receiving audio data and image data associated with an audio environment, generating an image set comprising a plurality of images each associated with audio samples representing acoustic properties of the audio environment, and generating an audio rendering model for the audio environment based at least in part on the image set and the audio samples. The audio rendering model may include a neural rendering volumetric representation of the audio environment augmented with audio encodings.

## Claims

### Claim 1

An apparatus comprising at least one processor and a memory storing instructions that are operable, when executed by the processor, to cause the apparatus to:
receive audio data and image data associated with an audio environment;
generate, based at least in part on the audio data and the image data, an image set comprising a plurality of images each associated with audio samples representing acoustic properties of the audio environment; and
generate, based at least in part on the image set and the audio samples, an audio rendering model for the audio environment, wherein the audio rendering model comprises a neural rendering volumetric representation of the audio environment augmented with audio encodings.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the audio rendering model comprises an augmented neural radiance field (NeRF) model that is augmented with the audio encodings.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein the audio data and the image data are respectively captured via at least one microphone and at least one camera of a capture device that scans the audio environment.

### Claim 4 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
train weights of the audio rendering model on volumetric probability densities which represent respective locations within the audio environment, wherein the weights are configured based at least in part on latent encoding of physical information and acoustic information for respective locations of the audio environment.

### Claim 5 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
determine, based at least in part on the audio samples, a camera properties set comprising relative audio sample locations and camera orientations associated with the audio samples;
generate the audio rendering model based at least in part on the image set, the audio samples, and the camera properties set.

### Claim 6 (depends on 5)

The apparatus of claim 5, wherein the camera properties set comprise a respective location of the image set with respect to the audio environment.

### Claim 7 (depends on 5)

The apparatus of claim 5, wherein the camera properties set comprise a respective orientation of the image set with respect to the audio environment.

### Claim 8 (depends on 5)

The apparatus of claim 5, wherein the instructions are further operable to cause the apparatus to:
input, to the audio rendering model, a training data vector that comprises impulse responses for the image set augmented with the camera properties set and the audio encodings.

### Claim 9 (depends on 5)

The apparatus of claim 5, wherein the instructions are further operable to cause the apparatus to:
input, to the audio rendering model, a training data vector that comprises material acoustic properties for the image set augmented with the camera properties set and the audio encodings.

### Claim 10 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
determine, based at least in part on the audio rendering model, one or more impulse responses associated with one or more audio sources in the audio environment.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
infer, based at least in part on the audio rendering model, locations of audio sources within the audio environment.

### Claim 12 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
infer, based at least in part on the audio rendering model, acoustic attributes of sound emitted within or from the audio environment.

### Claim 13 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
output one or more candidate audio component locations associated with the audio environment, wherein the one or more candidate audio component locations are generated based at least in part on the audio rendering model.

### Claim 14 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
generate, based at least in part on the audio rendering model, a digital twin of the audio environment.

### Claim 15 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
generate, based at least in part on the audio rendering model, an audio heat map of the audio environment.

### Claim 16 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
generate, based at least in part on the audio rendering model, an audio simulation for the audio environment.

### Claim 17 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
control audio equipment in the audio environment based at least in part on the audio rendering model.

### Claim 18 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
infer, based at least in part on the audio rendering model, material attributes of objects or surfaces within the audio environment.

### Claim 19 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
generate, based at least in part on the audio rendering model, a set of drawings or images of the audio environment along with optimal locations or audio settings of audio equipment within the audio environment.

### Claim 20 (depends on 1)

The apparatus of claim 1, wherein the instructions are further operable to cause the apparatus to:
receive a digital exploration request associated with the audio environment, the digital exploration request comprising an audio environment identifier associated with the audio environment;
identify the audio rendering model based at least in part on the audio environment identifier; and
generate one or more audio inferences based at least in part on the audio rendering model.

### Claim 21

A computer-implemented method comprising:
receiving audio data and image data associated with an audio environment;
generating, based at least in part on the audio data and the image data, an image set comprising a plurality of images each associated with audio samples representing acoustic properties of the audio environment; and
generating, based at least in part on the image set and the audio samples, an audio rendering model for the audio environment, wherein the audio rendering model comprises a neural rendering volumetric representation of the audio environment augmented with audio encodings.

### Claim 22

A computer program product, stored on a computer readable medium, comprising instructions that, when executed by one or more processors of an apparatus, cause the one or more processors to:
receive audio data and image data associated with an audio environment;
generate, based at least in part on the audio data and the image data, an image set comprising a plurality of images each associated with audio samples representing acoustic properties of the audio environment; and
generate, based at least in part on the image set and the audio samples, an audio rendering model for the audio environment, wherein the audio rendering model comprises a neural rendering volumetric representation of the audio environment augmented with audio encodings.
