# US 2020/0221242 A1 · Signal processing device and image display apparatus including the same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0221242 A1
- Document ID: US20200221242A1
- Status: Published application
- Kind: A1
- Application number: 16/737,646
- Filed: 2020-01-08
- Published: 2020-07-09
- Earliest claimed: 2019-01-08
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Kichul KIM; Joonil LEE; Jongkun KIM; Sunha JEON; Jongha PARK; Dongyun LEE
- Web page: https://patentgrep.com/patent/US20200221242A1

## CPC

- G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
- H04S 7/302: {Electronic adaptation of stereophonic sound system to listener position or orientation (H04S7/301 takes precedence)}
- G10L 25/30: using neural networks
- G10L 25/51: for comparison or discrimination
- H03G 9/005: {of digital or coded signals}
- H04R 3/04: for correcting frequency response
- H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- H04S 5/005: {of the pseudo five- or more-channel type, e.g. virtual surround}
- G10L 21/0364: for improving intelligibility
- H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
- H04S 2400/11: Positioning of individual sound objects, e.g. moving airplane, within a sound field (H04S2420/13 takes precedence)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04S: STEREOPHONIC SYSTEMS
- H03G: CONTROL OF AMPLIFICATION
- H04R: LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS (producing sounds with frequency not determined by supply frequency G10K)

## Abstract

Disclosed are a signal processing device and an image display apparatus including the same. The signal processing device and an image display apparatus including the same include: a converter configured to convert of frequency of an input stereo audio signal; a primary component analyzer configured to perform primary component analysis based on a signal from the converter; a feature extractor configured to extract a feature of a primary component signal based on a signal from the primary component analyzer; an envelope adjustor configured to perform envelope adjustment based on prediction performed on the basis of a deep neural network model; and an inverse converter configured to inversely convert a signal from the envelope adjustor to output an upmix audio signal of multi-channel. Accordingly, when upmixing the downmix stereo audio signal to a multichannel audio signal, spatial distortion can be improved.

## Claims

### Claim 1

A signal processing device comprising:
a converter configured to convert a frequency of an input stereo audio signal;
a primary component analyzer configured to perform primary component analysis based on the input stereo audio signal from the converter;
a feature extractor configured to extract a feature of a primary component signal based on primary component analysis results from the primary component analyzer, wherein the primary component signal is associated with the input stereo audio signal;
an envelope adjustor configured to perform envelope adjustment by performing a prediction with a deep neural network model based on the extracted feature from the feature extractor; and
an inverse converter configured to inversely convert a second signal output from the envelope adjustor for outputting a multi-channel audio signal.

### Claim 2 (depends on 1)

The signal processing device of claim 1, further comprising a filter bank configured to filter the converted input stereo audio signal from the converter by using a plurality of band pass filters.

### Claim 3 (depends on 1)

The signal processing device of claim 1, wherein the input stereo audio signal is converted by using a Short time Fourier Transform.

### Claim 4 (depends on 1)

The signal processing device of claim 1, further comprising a masking unit configured to perform masking with respect to the prediction result from a model learning predictor.

### Claim 5 (depends on 4)

The signal processing device of claim 4, wherein the masking unit is further configured to perform masking by using time-frequency component to perform channel separation based on the prediction result from the model learning predictor when each channel of the multi-channel audio signal is independent of time and frequency.

### Claim 6 (depends on 1)

The signal processing device of claim 1, wherein the envelope adjustor is further configured to separate a channel by correcting an envelope of a signal in a frequency band of the primary component signal according to a weight function for the frequency band based on a prediction result from a model learning predictor.

### Claim 7 (depends on 1)

The signal processing device of claim 1, wherein the primary component analyzer is further configured to separate the input stereo audio signal into the primary component signal and a sub-component signal.

### Claim 8 (depends on 7)

The signal processing device of claim 7, wherein the envelope adjustor is further configured to adjust a size of a frequency band for both the primary component signal and for the sub-component signal to follow an envelope in a target channel.

### Claim 9 (depends on 1)

The signal processing device of claim 1, wherein the primary component analyzer is further configured to perform at least one of a correlation operation between channels of the primary component signal, a panning gain operation of the primary component signal, or a power operation of the primary component signal.

### Claim 10 (depends on 1)

The signal processing device of claim 1, wherein the feature extractor is further configured to extract at least one of a panning gain of the primary component signal or a power of the primary component signal.

### Claim 11 (depends on 1)

The signal processing device of claim 1, further comprising:
a second converter configured to convert a frequency of a downmix stereo audio signal or a multi-channel audio signal received from a database;
a second primary component analyzer configured to perform primary component analysis based on a third signal from the second converter; and
a second feature extractor configured to extract a second feature of the primary component signal based on the third signal from the second primary component analyzer,
wherein learning is performed based on the deep neural network model based on the second feature extracted by the second feature extractor.

### Claim 12 (depends on 11)

The signal processing device of claim 11, further comprising a model learning predictor configured to perform learning based on the deep neural network model based on the feature extracted by the second feature extractor.

### Claim 13

An image display apparatus comprising:
a display;
an audio output unit; and
a signal processing device configured to be electrically connected to the display and the audio output unit,
wherein the signal processing device comprises:
a converter configured to convert a frequency of an input stereo audio signal;
a primary component analyzer configured to perform primary component analysis based on the input stereo audio signal from the converter;
a feature extractor configured to extract a feature of a primary component signal based on primary component analysis results from the primary component analyzer, wherein the primary component signal is associated with the input stereo audio signal;
an envelope adjustor configured to perform envelope adjustment by performing a prediction with a deep neural network model based on the extracted feature from the feature extractor; and
an inverse converter configured to inversely convert a second signal output from the envelope adjustor for outputting a multi-channel audio signal.

### Claim 14 (depends on 13)

The image display apparatus of claim 13, wherein the signal processing device further comprises a filter bank configured to filter the converted input stereo audio signal from the converter by using a plurality of band pass filters.

### Claim 15 (depends on 13)

The image display apparatus of claim 13, wherein the input stereo audio signal is converted by using a Short time Fourier Transform.

### Claim 16 (depends on 13)

The image display apparatus of claim 13, wherein the signal processing device further comprises a masking unit configured to perform masking with respect to the prediction result from a model learning predictor.

### Claim 17 (depends on 16)

The image display apparatus of claim 16, wherein the masking unit is further configured to perform masking by using time-frequency component to perform channel separation based on the prediction result from the model learning predictor when each channel of the multi-channel audio signal is independent of time and frequency.

### Claim 18 (depends on 13)

The image display apparatus of claim 13, wherein the envelope adjustor is further configured to separates a channel by correcting an envelope of a signal in a frequency band according to a weight function for the frequency band based on a prediction result from a model learning predictor.

### Claim 19 (depends on 13)

The image display apparatus of claim 13, wherein the signal processing device further comprises:
a second converter configured to convert a frequency of a downmix stereo audio signal or a multi-channel audio signal received from a database;
a second primary component analyzer configured to perform primary component analysis based on a third signal from the second converter; and
a second feature extractor configured to extract a second feature of the primary component signal based on the third signal from the second primary component analyzer,
wherein learning is performed based on the deep neural network model based on the feature extracted by the second feature extractor.

### Claim 20 (depends on 19)

The image display apparatus of claim 19, wherein the signal processing device further comprises a model learning predictor configured to perform learning based on the deep neural network model based on the feature extracted by the second feature extractor.
