# US 12,112,521 B2 · Room acoustics simulation using deep learning image analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,521 B2
- Document ID: US12112521B2
- Status: Granted
- Kind: B2
- Application number: 17/354,668
- Filed: 2021-06-22
- Published: 2022-03-31
- Granted: 2024-10-08
- Earliest claimed: 2018-12-24
- Assignees: DTS, Inc.
- Applicants: DTS, Inc.
- Inventors: Martin Walsh; Aoife McDonagh; Michael M. Goodwin; Edward Stein; Peter Corcoran
- Web page: https://patentgrep.com/patent/US12112521B2

## CPC

- G06F 30/27: using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
- G06V 10/764: using classification, e.g. of video objects
- G01H 7/00: Measuring reverberation time {; room acoustic measurements}
- G06F 30/13: Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 3/096: Transfer learning
- G06V 10/82: using neural networks
- G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59)
- G06V 20/20: in augmented reality scenes
- G06V 20/64: Three-dimensional [3D] objects
- H04S 7/305: {Electronic adaptation of stereophonic audio signals to reverberation of the listening space (H04S7/301 takes precedence)}
- G06F 2119/10: Noise analysis or noise optimisation
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- H04S 2400/15: Aspects of sound capture and related signal processing for recording or reproduction
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01H: MEASUREMENT OF MECHANICAL VIBRATIONS OR ULTRASONIC, SONIC OR INFRASONIC WAVES
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04S: STEREOPHONIC SYSTEMS

## Abstract

A method comprises: receiving an image of a real-world environment; using a machine learning classifier, classifying the image to produce classifications associated with acoustic presets for an acoustic environment simulation, the acoustic presets each including acoustic parameters that represent sound reverberation; and selecting an acoustic preset among the acoustic presets based on the classifications.

## Claims

### Claim 1

A method comprising:
receiving an image of a real-world environment;
using a machine learning classifier, trained using training images of real-world environments labeled with one or more room types and training acoustic presets derived from each of the one or more room types, each training acoustic preset having acoustic parameters for sound reverberation including a reverberation decay time, classifying the image to the one or more room types and the acoustic presets associated with the one or more room types, wherein each acoustic preset includes the acoustic parameters for the sound reverberation including the reverberation decay time;
selecting an acoustic preset among the acoustic presets; and
performing an acoustic environment simulation based on the acoustic parameters of the acoustic preset by modeling the sound reverberation for one or more virtual sound objects placed virtually in the real-world environment based on the acoustic parameters of the acoustic preset including the reverberation decay time.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
using the machine learning classifier, classifying the image, or one or more further images, to produce one or more acoustic parameter modifiers; and
modifying the acoustic parameters of the acoustic preset based on the one or more acoustic parameter modifiers, to produce a modified acoustic preset including modified acoustic parameters for the acoustic environment simulation,
wherein performing the acoustic environment simulation includes performing the acoustic environment simulation using the modified acoustic parameters.

### Claim 3 (depends on 1)

The method of claim 1, wherein:
the acoustic parameters further include one or more of a frequency dependent decay time, a room size, a reflection delay, an early onset time, and a diffusion level.

### Claim 4 (depends on 1)

The method of claim 1, wherein classifying includes classifying the image directly to the acoustic presets, without performing an intervening classification or operation.

### Claim 5 (depends on 1)

The method of claim 1, wherein:
classifying includes classifying the image to the acoustic presets such that the acoustic presets have respective confidence levels; and
selecting includes selecting the acoustic preset such that the acoustic preset is associated with a highest one of the respective confidence levels.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
receiving a second image of the real-world environment;
using the machine learning classifier, classifying the second image to produce second acoustic presets that have respective second confidence levels;
determining whether one or more of the respective second confidence levels exceeds a confidence level threshold; and
if one or more of the respective second confidence levels exceeds the confidence level threshold, selecting a second acoustic preset among the acoustic presets, and replacing the acoustic preset with the second acoustic preset for the acoustic environment simulation.

### Claim 7 (depends on 6)

The method of claim 6, further comprising:
if one or more of the respective second confidence levels do not exceed the confidence level threshold, not selecting the second acoustic preset, and not replacing the acoustic preset for the acoustic environment simulation.

### Claim 8 (depends on 1)

The method of claim 1, wherein receiving the image of the real-world environment includes receiving the image as a 360 degree image of the real-world environment.

### Claim 9 (depends on 1)

The method of claim 1, wherein the image represents a composite of different images.

### Claim 10 (depends on 1)

The method of claim 1, wherein the acoustic parameters of the acoustic presets were derived algorithmically from acoustic impulse responses representative of the real-world environments represented in the training images on which the machine learning classifier was trained.

### Claim 11 (depends on 1)

The method of claim 1, wherein the machine learning classifier was trained on the training images of the real-world environments labeled with respective ones of the acoustic presets that most likely match the real-world environments.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
receiving a digital three-dimensional (3D) mesh (3D mesh); and
estimating acoustic properties of physical materials represented m the 3D mesh, to produce early reflection parameters for the acoustic environment simulation.

### Claim 13

An apparatus comprising:
a processor configured to:
receive an image of a real-world environment;
use a machine learning classifier including one or more neural networks, trained on training images of real-world environments labeled with one or more room types and training acoustic presets derived from each of the one or more room types, each training acoustic preset having acoustic parameters for sound reverberation including a reverberation decay time, to classify the image to the one or more room types and the acoustic presets associated with the one or more room types, wherein each acoustic preset includes the acoustic parameters for the sound reverberation including the reverberation decay time;
select an acoustic preset among the acoustic presets; and
perform an acoustic environment simulation based on the acoustic parameters of the acoustic preset including the reverberation decay time.

### Claim 14 (depends on 13)

The apparatus of claim 13, wherein the processor is configured to perform the acoustic environment simulation by modeling the sound reverberation for one or more virtual sound objects placed virtually in the real-world environment based on the acoustic parameters of the acoustic preset.

### Claim 15 (depends on 14)

The apparatus of claim 14, wherein the processor is further configured to:
use the machine learning classifier to further classify the image to produce one or more acoustic parameter modifiers;
modify the acoustic parameters of the acoustic preset based on the one or more acoustic parameter modifiers, to produce a modified acoustic preset including modified acoustic parameters for the acoustic environment simulation; and
further perform the acoustic environment simulation using the modified acoustic parameters.

### Claim 16 (depends on 13)

The apparatus of claim 13, wherein:
the acoustic presets have respective confidence levels; and
the processor is configured to select by selecting the acoustic preset that has a highest one of the respective confidence levels.

### Claim 17 (depends on 13)

The apparatus of claim 13, wherein the machine learning classifier was trained on the training images of the real-world environments labeled with respective ones of the acoustic presets that most likely match the real-world environments.

### Claim 18

A non-transitory computer readable medium encoded with instructions that, when executed by a processor, cause the processor to:
receive an image of a real-world environment;
use a machine learning classifier, previously trained on training images of real-world environments labeled with one or more room types and training acoustic presets derived from each of the one or more room types, each training acoustic presets including acoustic parameters that represent sound reverberation including a reverberation decay time, to classify the image directly to the one or more room types and the acoustic presets associated with the one or more room types, each including the acoustic parameters that represent the sound reverberation including the reverberation decay time;
select an acoustic preset among the acoustic presets; and
perform an acoustic environment simulation based on the acoustic parameters of the acoustic preset.

### Claim 19 (depends on 18)

The non-transitory computer readable medium of claim 18, wherein the instructions to cause the processor to perform the acoustic environment simulation include instructions to cause the processor to model the sound reverberation for one or more virtual sound objects placed virtually in the real-world environment based on the acoustic parameters of the acoustic preset including the reverberation decay time.

### Claim 20 (depends on 19)

The non-transitory computer readable medium of claim 19, further comprising instructions to cause the processor to:
use the machine learning classifier to further classify the image to produce one or more acoustic parameter modifiers;
modify the acoustic parameters of the acoustic preset based on the one or more acoustic parameter modifiers, to produce a modified acoustic preset including modified acoustic parameters for the acoustic environment simulation; and
further perform the acoustic environment simulation using the modified acoustic parameters.