# US 2018/0324540 A1 · Content-Adaptive Surround Sound Virtualization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2018/0324540 A1
- Document ID: US20180324540A1
- Status: Published application
- Kind: A1
- Application number: 15/772,383
- Filed: 2016-11-02
- Published: 2018-11-08
- Earliest claimed: 2015-11-03
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Xin Liu; Lie LU; Alan J. Seefeldt
- Web page: https://patentgrep.com/patent/US20180324540A1

## CPC

- H04S 5/005: {of the pseudo five- or more-channel type, e.g. virtual surround}
- G06F 16/683: using metadata automatically derived from the content
- G06F 17/18: for evaluating statistical data {, e.g. average values, frequency distributions, probability functions, regression analysis (forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04)}
- H04R 1/26: Spatial arrangements of separate transducers responsive to two or more frequency ranges
- H04S 5/00: Pseudo-stereo systems, e.g. in which additional channel signals are derived from monophonic signals by means of phase shifting, time delay or reverberation
- H04S 7/30: {Control circuits for electronic adaptation of the sound field}
- H04S 7/307: {Frequency adjustment, e.g. tone control (H04S7/301 takes precedence)}
- H04S 2400/03: Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 (H04S2400/01 takes precedence)
- H04S 2420/01: Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
- H04S: STEREOPHONIC SYSTEMS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Example embodiments disclosed herein relate to content-adaptive surround sound virtualization. A method of virtualizing surround sound is disclosed. The method includes receiving a set of input audio signals, each of the input audio signals being indicative of sound from one of different sound sources, and determining a probability of the set of input audio signals belonging to a predefined audio content category. The method also includes determining a virtualization amount based on the determined probability, the virtualization amount indicating to which extent the set of input audio signals is virtualized as surround sound. The method further includes performing surround sound virtualization on two or more input audio signals in the set based on the determined virtualization amount and generating output audio signals based on the virtualized input audio signals and other input audio signals in the set. Corresponding system and computer program product for virtualizing surround sound are also disclosed.

## Claims

### Claim 1

A method of virtualizing surround sound, comprising:
receiving a set of input audio signals, each of the input audio signals being indicative of sound from one of different sound sources;
obtaining respective location information of the sound sources of the input audio signals, and respective locations of speakers which are to be driven by output audio signals;
determining a probability of the set of input audio signals belonging to a predefined audio content category;
determining a virtualization amount based on the determined probability, the virtualization amount indicating to which extent the set of input audio signals is virtualized as surround sound;
performing surround sound virtualization on two or more input audio signals in the set based on the determined virtualization amount; and
generating the output audio signals based on the virtualized input audio signals and one or more other input audio signals in the set,
wherein said performing surround sound virtualization comprises adjusting the obtained location information of the sound sources such that, for each of the sound sources of the input audio signals, a shortest distance between a virtual location of the sound source and a virtual line through a predefined listening location and one of the obtained speaker locations corresponds to the virtualization amount, whereby the shortest distance is substantially zero when the virtualization amount is zero.

### Claim 2 (depends on 1)

The method according to claim 1, further comprising:
modifying the virtualization amount in a nonlinear manner, and
wherein adjusting the predetermined location information comprises:
adjusting the predetermined location information based on the modified virtualization amount.

### Claim 3 (depends on 2)

The method according to claim 2, wherein performing the surround sound virtualization on the two or more input audio signals comprises:
obtaining a set of coefficients for head related transfer functions (HRTFs) or binaural room impulse responses (BRIRs) corresponding to the adjusted location information; and
processing the two or more input audio signals based on the obtained set of coefficients.

### Claim 4 (depends on 3)

The method according to claim 3, wherein obtaining the set of coefficients comprises:
in response to finding a predefined set of coefficients for the HRTFs or for the BRIRs, selecting the predefined set of coefficients as the obtained set of coefficients; and
in response to missing the predefined set of coefficients, determining the obtained set of coefficients by interpolating predefined sets of coefficients for further HRTFs, or for further BRIRs, corresponding to further location information.

### Claim 5 (depends on 1)

The method according to claim 1, wherein performing the surround sound virtualization further comprises:
determining an effective frequency range for the two or more input audio signals based on the virtualization amount; and
performing the surround sound virtualization on the two or more input audio signals in the determined effective frequency range.

### Claim 6 (depends on 1)

The method according to claim 1 wherein the predefined audio content category includes a category of music, and
wherein determining the virtualization amount comprises:
determining the virtualization amount as a decreasing function of a probability of the set belonging to the category of music.

### Claim 7 (depends on 1)

The method according to claim 1 wherein the predefined audio content category includes a category of non-music, and
wherein determining the virtualization amount comprises:
determining the virtualization amount as an increasing function of a probability of the set belonging to the category of non-music.

### Claim 8 (depends on 1)

The method according to claim 1 wherein the category of non-music includes at least two of a subcategory of speech, a subcategory of background sound, or a subcategory of noise, and
wherein determining the probability of the set of input audio signals belonging to the predefined audio content category comprises:
determining a probability of the set belonging to each of the at least two subcategories; and
determining the probability of the set belonging to the category of non-music based on the determined probabilities for the at least two subcategories.

### Claim 9 (depends on 1)

A computer program product for virtualizing surround sound, comprising a computer program tangibly embodied on a machine readable medium, the computer program comprising program code for performing the method according to claim 1.

### Claim 10 (depends on 1)

A computer system comprising a central processing unit configured to perform the method according to claim 1.
