# US 12,118,987 B2 · Dialog detector

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,118,987 B2
- Document ID: US12118987B2
- Status: Granted
- Kind: B2
- Application number: 17/604,379
- Filed: 2020-04-13
- Published: 2022-06-23
- Granted: 2024-10-15
- Earliest claimed: 2019-04-18
- Assignees: Dolby Laboratories Licensing Corporation
- Applicants: Dolby Laboratories Licensing Corporation
- Inventors: Lie Lu; Xin Liu
- Web page: https://patentgrep.com/patent/US12118987B2

## CPC

- G10L 15/197: Probabilistic grammars, e.g. word n-grams
- G10L 25/51: for comparison or discrimination
- G10L 15/02: Feature extraction for speech recognition; Selection of recognition unit
- G10L 15/04: Segmentation; Word boundary detection
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L 25/18: the extracted parameters being spectral information of each sub-band
- G10L 25/81: for discriminating voice from music
- H04S 1/007: {in which the audio signals are in digital form (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
- G10L 25/45: characterised by the type of analysis window
- H04S 2400/03: Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 (H04S2400/01 takes precedence)
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04S: STEREOPHONIC SYSTEMS

## Abstract

The present application relates to a method of extracting audio features in a dialog detector in response to an input audio signal, the method comprising dividing the input audio signal into a plurality of frames, extracting frame audio features from each frame, determining a set of context windows, each context window including a number of frames surrounding a current frame, deriving, for each context window, a relevant context audio feature for the current frame based on the frame audio features of the frames in each respective context, and concatenating each context audio feature to form a combined feature vector to represent the current frame. The context windows with the different length can improve the response speed and improve robustness.

## Claims

### Claim 1

A machine learning method of dialog detection, the method comprising:
receiving an input audio signal and dividing the received input audio signal into a plurality of frames;
extracting frame audio features from each frame;
determining first, second, and third context windows for a current frame, each of the first, second, and third context windows including a different respective numberof frames adjacent to the current frame;
deriving, for each of the first, second, and third context windows, a respective context audio feature for the current frame based on the frame audio features of the frames in each respective context window;
generating a combined feature vector to represent the current frame, the generating including concatenating the respective context audio features of the first, second, and third context windows;
obtaining a speech confidence score that represents a probability of presence of dialog in the current frame using the combined feature vector; and
sending the current frame to an audio improving device for dialog enhancement when it is determined based on the obtained speech confidence score that dialog is present.

### Claim 2 (depends on 1)

The method of claim 1,
wherein the first context window includes the current frame and a number of frames preceding and/or succeeding the current frame;
wherein the second context window includes the frames of the short term first context window and a plurality of look-back frames; and
wherein the third context window includes the frames of the second context window and a plurality of long-term history frames.

### Claim 3 (depends on 1)

The method of claim 1, wherein the respective number of frames in each of the first, second, and third context windows is predetermined.

### Claim 4 (depends on 1)

The method of claim 1, wherein the first context window contains the current frame and a set of look-ahead frames from a look-ahead buffer.

### Claim 5 (depends on 1)

The method of claim 1, wherein the respective number of frames in at least one of the first, second, and third context windows is adaptively determined based on the extracted frame audio features.

### Claim 6 (depends on 5)

The method of claim 5, wherein the respective number of frames in the at least one of the first, second, and third context windows is adaptively determined by analyzing stationarity of frame-level audio features.

### Claim 7 (depends on 6)

The method of claim 6, wherein the adaptive determination of respective the number of frames in the at least one of the first, second, and third context windows is based on information related to amplitude of the input audio signal.

### Claim 8 (depends on 6)

The method of claim 6, wherein the adaptive determination of the number of frames in the at least one of the first, second, and third context windows is based on information related to spectrum of the input audio signal.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
converting frames in a left channel and a right channel to a spectrum representation of frames;
removing uncorrelated signals in the left channel and the right channel by applying frequency dependent gains to the spectrum representation in the left channel and the right channel, respectively;
obtaining a downmixed signal from the left and right channels; and
using said downmixed signal as said input audio signal.

### Claim 10 (depends on 9)

The method of claim 9, wherein the frequency dependent gains are estimated from a covariance matrix.

### Claim 11 (depends on 1)

A method of classifying an input audio signal of a dialog detector, the method comprising:
obtaining the speech confidence score according to the method of claim 1;
determining a music confidence score based on the combined feature vector; and
combining the speech confidence score and the music confidence score to obtain a final dialog confidence score.

### Claim 12 (depends on 11)

The method of claim 11, wherein the final dialog confidence score is refined by a context-dependent parameter.

### Claim 13 (depends on 12)

The method of claim 12, wherein the context-dependent parameter is computed based on proportion of frames identified as speech or music in a history context window, that is at least ten seconds long.

### Claim 14 (depends on 1)

A dialog detector comprising means for performing the method according to claim 1.

### Claim 15 (depends on 1)

A non-transitory computer readable storage medium having stored thereon instructions that, when executed by a computer, cause the computer to perform the method according to claim 1.

### Claim 16

A method of obtaining a confidence score that represents a probability of presence of dialog in a current frame of an input audio signal, the method comprising:
dividing the input audio signal into a plurality of frames;
extracting frame audio features from each frame;
determining a set of context windows, each context window including a number of frames surrounding the current frame;
deriving, for each context window, a relevant context audio feature for the current frame based on the frame audio features of the frames in each respective context;
concatenating each context audio feature to form a combined feature vector to represent the current frame;
obtaining a speech confidence score that represents a probability of presence of dialog in the current frame using the combined feature vector;
converting frames in a left channel and a right channel to a spectrum representation of frames;
removing uncorrelated signals in the left channel and the right channel by applying frequency dependent gains to the spectrum representation in the left channel and the right channel, respectively;
obtaining a downmixed signal from the left and right channels; and
using said downmixed signal as said input audio signal.

### Claim 17 (depends on 16)

The method of claim 16, wherein the frequency dependent gains are estimated from a covariance matrix.