# US 2026/0247090 A1 · Method for multichannel audio reconstruction and speaker system using the method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0247090 A1
- Document ID: US20260247090A1
- Status: Published application
- Kind: A1
- Application number: 19/471,010
- Filed: 2023-04-18
- Published: 2026-08-20
- Earliest claimed: 2023-04-18
- Assignees: Harman International Industries, Incorporated
- Applicants: Harman International Industries, Incorporated
- Inventors: Hongyi ZHU; Jianwen ZHENG; Hongfei ZHOU
- Web page: https://patentgrep.com/patent/US20260247090A1

## CPC

- H04S 5/005: {of the pseudo five- or more-channel type, e.g. virtual surround}
- G10L 21/0272: Voice signal separating
- H04R 5/02: Spatial or constructional arrangements of loudspeakers
- H04S: STEREOPHONIC SYSTEMS
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

This disclosure provides a method of multichannel audio reconstruction for a speaker system and the speaker system using the method. The method may comprise: receiving stereo audio; performing vocal extraction on the stereo audio and outputting a stereo vocal signal and a stereo non-vocal signal; performing primary and ambience extraction (PAE) based rendering on the stereo vocal signal and the stereo non-vocal signal, respectively, to obtain a rendered vocal output and a rendered non-vocal output; and mixing the rendered vocal output and the rendered non-vocal output to generate the multichannel audio.

## Claims

### Claim 1

A method of multichannel audio reconstruction for a speaker system comprising:
receiving stereo audio;
performing vocal extraction on the stereo audio and outputting a stereo vocal signal and a stereo non-vocal signal;
performing primary and ambience extraction (PAE) based rendering on the stereo vocal signal and the stereo non-vocal signal, respectively, to obtain a rendered vocal output and a rendered non-vocal output; and
mixing the rendered vocal output and the rendered non-vocal output to generate the multichannel audio.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the performing the PAE based rendering on the stereo vocal signal and the stereo non-vocal signal comprises:
performing Short-Time Fourier Transform (STFT) on the stereo vocal signal and the stereo non-vocal signal to obtain the transformed stereo vocal signal and the transformed stereo non-vocal signal;
performing the PAE on the transformed stereo vocal signal and the transformed stereo non-vocal signal to obtain a primary component and an ambience component of each of the transformed stereo vocal signal and the transformed stereo non-vocal signal;
rendering the primary component and the ambience component of each of the transformed stereo vocal signal and the transformed stereo non-vocal signal, to obtain the rendered primary signals and the rendered ambience signals corresponding to each of the stereo vocal signal and the stereo non-vocal signal; and
performing Inverse Short-Time Fourier Transform (ISTFT) on the rendered primary signals and the rendered ambience signals corresponding to each of the stereo vocal signal and the stereo non-vocal signal to obtain the rendered vocal output and the rendered non-vocal output.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the rendering the primary component comprises:
calculating a direction of a sound source based on spatial orientation information included in the primary component;
determining two adjacent speakers for reproducing the sound source based on the spatial orientation information; and
calculating signals assigned to each speaker of the two adjacent speakers.

### Claim 4 (depends on 3)

The method according to claim 3, wherein the calculating the direction of the sound source comprises performing a gain estimation of the primary component.

### Claim 5 (depends on 3)

The method according to claim 3, wherein the calculating the signals assigned to each speaker of the two adjacent speakers comprises calculating the signals assigned to each speaker based on a reconstruction matrix and a separation matrix.

### Claim 6 (depends on 2)

The method according to claim 2, wherein the rendering the ambience component comprises convolving the ambience component in a left channel and a right channel respectively with different predetermined impulse responses to generate surround signals for surround channels.

### Claim 7 (depends on 1)

The method according to claim 1, further comprising determining whether the speaker system is operating in a party mode or a karaoke mode.

### Claim 8

A speaker system comprising:
a memory configured to store instructions; and
a processor configured to perform the instructions to:
receive stereo audio;
perform vocal extraction on the stereo audio and outputting a stereo vocal signal and a stereo non-vocal signal;
perform primary and ambience extraction (PAE) based rendering on the stereo vocal signal and the stereo non-vocal signal, respectively, to obtain a rendered vocal output and a rendered non-vocal output; and
mix the rendered vocal output and the rendered non-vocal output to generate multichannel audio.

### Claim 9 (depends on 8)

The speaker system according to claim 8, wherein the processor is further configured to:
perform Short-Time Fourier Transform (STFT) on the stereo vocal signal and the stereo non-vocal signal to obtain the transformed stereo vocal signal and the transformed stereo non-vocal signal;
perform the PAE on the transformed stereo vocal signal and the transformed stereo non-vocal signal to obtain a primary component and an ambience component of each of the transformed stereo vocal signal and the transformed stereo non-vocal signal;
render the primary component and the ambience component of each of the transformed stereo vocal signal and the transformed stereo non-vocal signal, to obtain the rendered primary signals and the rendered ambience signals corresponding to each of the stereo vocal signal and the stereo non-vocal signal; and
perform Inverse Short-Time Fourier Transform (ISTFT) on the rendered primary signals and the rendered ambience signals corresponding to each of the stereo vocal signal and the stereo non-vocal signal to obtain the rendered vocal output and the rendered non-vocal output.

### Claim 10 (depends on 9)

The speaker system according to claim 9, wherein the processor is further configured to:
calculate a direction of a sound source based on spatial orientation information included in the primary component;
determine two adjacent speakers for reproducing the sound source based on the spatial orientation information; and
calculate signals assigned to each speaker of the two adjacent speakers.

### Claim 11 (depends on 10)

The speaker system according to claim 10, wherein the processor is further configured to calculate the direction of the sound source by performing a gain estimation of the primary component.

### Claim 12 (depends on 10)

The speaker system according to claim 10, wherein the processor is further configured to calculate the signals assigned to each speaker by calculating the signals assigned to each speaker of the two adjacent speakers based on a reconstruction matrix and a separation matrix.

### Claim 13 (depends on 9)

The speaker system according to claim 9, wherein the processor is further configured to render the ambience component by convolving the ambience component in a left channel and a right channel respectively with different predetermined impulse responses to generate surround signals for surround channels.

### Claim 14 (depends on 8)

The speaker system according to claim 8, wherein the speaker system comprises at least one control element that can be operated to control the speaker system to switch between a party mode and a karaoke mode.

### Claim 15

A non-transitory computer-readable storage medium comprising computer-executable instructions which, when executed by a computer, causes the computer to perform operations including to:
receive stereo audio;
perform vocal extraction on the stereo audio and outputting a stereo vocal signal and a stereo non-vocal signal;
perform primary and ambience extraction (PAE) based rendering on the stereo vocal signal and the stereo non-vocal signal, respectively, to obtain a rendered vocal output and a rendered non-vocal output; and
mix the rendered vocal output and the rendered non-vocal output to generate the multichannel audio.

### Claim 16 (depends on 15)

The non-transitory computer-readable storage medium according to claim 15, wherein to perform the PAE based rendering on the stereo vocal signal and the stereo non-vocal signal comprises to:
perform Short-Time Fourier Transform (STFT) on the stereo vocal signal and the stereo non-vocal signal to obtain the transformed stereo vocal signal and the transformed stereo non-vocal signal;
perform the PAE on the transformed stereo vocal signal and the transformed stereo non-vocal signal to obtain a primary component and an ambience component of each of the transformed stereo vocal signal and the transformed stereo non-vocal signal;
render the primary component and the ambience component of each of the transformed stereo vocal signal and the transformed stereo non-vocal signal, to obtain the rendered primary signals and the rendered ambience signals corresponding to each of the stereo vocal signal and the stereo non-vocal signal; and
perform Inverse Short-Time Fourier Transform (ISTFT) on the rendered primary signals and the rendered ambience signals corresponding to each of the stereo vocal signal and the stereo non-vocal signal to obtain the rendered vocal output and the rendered non-vocal output.

### Claim 17 (depends on 16)

The non-transitory computer-readable storage medium according to claim 16, wherein to render the primary component comprises to:
calculate a direction of a sound source based on spatial orientation information included in the primary component;
determine two adjacent speakers for reproducing the sound source based on the spatial orientation information; and
calculate signals assigned to each speaker of the two adjacent speakers.

### Claim 18 (depends on 17)

The non-transitory computer-readable storage medium according to claim 17, wherein to calculate the direction of the sound source comprises performing a gain estimation of the primary component.

### Claim 19 (depends on 17)

The non-transitory computer-readable storage medium according to claim 17, wherein to calculate the signals assigned to each speaker of the two adjacent speakers comprises to calculate the signals assigned to each speaker based on a reconstruction matrix and a separation matrix.

### Claim 20 (depends on 16)

The non-transitory computer-readable storage medium according to claim 16, wherein to render the ambience component comprises to convolve the ambience component in a left channel and a right channel respectively with different predetermined impulse responses to generate surround signals for surround channels.
