# US 10,872,611 B2 · Selecting channel adjustment method for inter-frame temporal shift variations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,872,611 B2
- Document ID: US10872611B2
- Status: Granted
- Kind: B2
- Application number: 16/115,166
- Filed: 2018-08-28
- Published: 2019-03-14
- Granted: 2020-12-22
- Earliest claimed: 2017-09-12
- Assignees: QUALCOMM Incorporated
- Applicants: QUALCOMM Incorporated
- Inventors: Venkata Subrahmanyam Chandra Sekhar Chebiyyam; Venkatraman Atti
- Web page: https://patentgrep.com/patent/US10872611B2

## CPC

- G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
- G10L 19/002: Dynamic bit allocation (for perceptual audio coders G10L19/032)
- G10L 19/022: Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring
- H04S 3/008: {in which the audio signals are in digital form, i.e. employing more than two discrete digital channels (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- G10L 19/005: Correction of errors induced by the transmission channel, if related to the coding algorithm
- G10L 19/22: Mode decision, i.e. based on audio signal content versus external parameters
- G10L 21/038: using band spreading techniques
- H04S 1/007: {in which the audio signals are in digital form (data reduction aspects thereof based on psychoacoustics G10L19/02)}
- H04S 2400/01: Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
- H04S 2400/03: Aspects of down-mixing multi-channel audio to configurations with lower numbers of playback channels, e.g. 7.1 -> 5.1 (H04S2400/01 takes precedence)
- H04S 2400/15: Aspects of sound capture and related signal processing for recording or reproduction
- H04S 2420/03: Application of parametric coding in stereophonic audio systems
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- H04S: STEREOPHONIC SYSTEMS

## Abstract

A method for multi-channel audio or speech signal processing includes receiving a reference channel and a target channel, determining a variation between a first mismatch value and a second mismatch value, and comparing the variation with a first threshold that may have a pre-determined value or may be adjusted based on a frame type or a smoothing factor. The method also includes adjusting a set of target samples of the target channel based on the variation and based on the comparison to generate an adjusted set of target samples. Adjusting the set of target samples includes selecting one among a first interpolation and a second interpolation based on the variation. The method further includes generating at least one encoded channel based on a set of reference samples and the adjusted set of target samples. The method also includes transmitting the at least one encoded channel to a second device.

## Claims

### Claim 1

A method for coding of multi-channel audio signals, the method comprising:
receiving, at a first device, a reference channel and a target channel, the reference channel including a set of reference samples, and the target channel including a set of target samples;
determining, at the first device, a variation between a first mismatch value and a second mismatch value, the first mismatch value indicative of an amount of temporal mismatch between a first reference sample of the set of reference samples and a first target sample of the set of target samples, the second mismatch value indicative of an amount of temporal mismatch between a second reference sample of the set of reference samples and a second target sample of the set of target samples;
selecting, at the first device, a particular adjustment technique from a plurality of adjustment techniques based on a comparison of the variation with a first threshold;
using the variation subsequent to the comparison, at the first device, to perform the particular adjustment technique to adjust the set of target samples to generate an adjusted set of target samples;
generating, at the first device, at least one encoded channel based on the set of reference samples and the adjusted set of target samples; and
transmitting the at least one encoded channel from the first device to a second device.

### Claim 2 (depends on 1)

The method of claim 1, further comprising selecting one of a first interpolation or a second interpolation as the particular adjustment technique in response to determining whether the variation exceeds the first threshold, wherein the first interpolation is different from the second interpolation.

### Claim 3 (depends on 2)

The method of claim 2, wherein performing the first interpolation comprises performing at least one among a Sinc interpolation and a Lagrange interpolation.

### Claim 4 (depends on 2)

The method of claim 2, wherein performing the first interpolation comprises performing a hybrid interpolation, the hybrid interpolation includes using both a Sinc interpolation and a Lagrange interpolation.

### Claim 5 (depends on 2)

The method of claim 2, wherein performing the second interpolation comprises performing an overlap and add interpolation.

### Claim 6 (depends on 5)

The method of claim 5, wherein performing the overlap and add interpolation is based on the first mismatch value and the second mismatch value.

### Claim 7 (depends on 6)

The method of claim 6, wherein performing the overlap and add interpolation is based on a first window function and a second window function, wherein the second window function is dependent on the first window function.

### Claim 8 (depends on 2)

The method of claim 2, wherein the first interpolation is performed on a number of samples corresponding to a spreading factor.

### Claim 9 (depends on 8)

The method of claim 8, wherein a value of the spreading factor is less than or equal to a number of samples in a frame of the target channel.

### Claim 10 (depends on 1)

The method of claim 1, further comprising determining the first threshold based on frame type of the set of target samples.

### Claim 11 (depends on 10)

The method of claim 10, wherein the frame type indicates the set of target samples corresponds to at least one among speech, music, and noise.

### Claim 12 (depends on 11)

The method of claim 11, wherein determining the first threshold based on information indicating frame type of the set of target samples comprises decreasing the first threshold in response to the determination that the frame type corresponds to music.

### Claim 13 (depends on 1)

The method of claim 1, further comprising determining the first threshold based on a smoothing factor, the smoothing factor indicates smoothness setting of cross-correlation value.

### Claim 14 (depends on 1)

The method of claim 1, further comprising:
down-sampling the reference channel to generate a reference down-sampled channel;
down-sampling the target channel to generate a target down-sampled channel; and
determining the first mismatch value and the second mismatch value based on comparisons of the reference down-sampled channel and the target down-sampled channel.

### Claim 15 (depends on 1)

The method of claim 1, further comprising determining whether to adjust the set of target samples based on one among the variation, a reference channel indicator, an energy of the reference channel and an energy of the target channel, and a transient detector.

### Claim 16 (depends on 1)

The method of claim 1, wherein a first portion of the set of target samples are time-shifted relative to a first portion of the set of reference samples by an amount that is based on the first mismatch value, and wherein a second portion of the set of target samples are time-shifted relative to a second portion of the set of reference samples by an amount that is based on the second mismatch value.

### Claim 17 (depends on 1)

The method of claim 1, wherein the first mismatch value corresponds to an amount of time delay between receipt of a frame of a first audio signal via a first microphone and receipt of a corresponding frame of a second audio signal via a second microphone, wherein the first audio signal corresponds to one of the reference channel or the target channel, and wherein the second audio signal corresponds to the other of the reference channel or the target channel.

### Claim 18 (depends on 1)

The method of claim 1, wherein the at least one encoded channel includes a mid channel, a side channel, or both.

### Claim 19 (depends on 1)

The method of claim 1, wherein a first audio signal includes one of a right channel or a left channel, and wherein a second audio signal includes the other of the right channel or the left channel, wherein the first audio signal corresponds to one of the reference channel or the target channel, and wherein the second audio signal corresponds to the other of the reference channel or the target channel.

### Claim 20 (depends on 1)

The method of claim 1, wherein the first device is integrated into a mobile device or a base station.

### Claim 21

A multi-channel audio coding device comprising
an encoder configured to:
receive a reference channel and a target channel, the reference channel including a set of reference samples, and the target channel including a set of target samples;
determine a variation between a first mismatch value and a second mismatch value, the first mismatch value indicative of an amount of temporal mismatch between a first reference sample of the set of reference samples and a first target sample of the set of target samples, the second mismatch value indicative of an amount of temporal mismatch between a second reference sample of the set of reference samples and a second target sample of the set of target samples;
select a particular adjustment technique from a plurality of adjustment techniques based on a comparison of the variation with a first threshold;
use the variation subsequent to the comparison to perform the particular adjustment technique to adjust the set of target samples to generate an adjusted set of target samples; and
generate at least one encoded channel based on the set of reference samples and the adjusted set of target samples; and
a network interface configured to transmit the at least one encoded channel.

### Claim 22 (depends on 21)

The multi-channel audio coding device of claim 21, wherein the encoder includes a sample adjuster configured to select one of a first interpolation or a second interpolation as the particular adjustment technique based on whether the variation exceeds the first threshold, and wherein the first interpolation is different from the second interpolation.

### Claim 23 (depends on 22)

The multi-channel audio coding device of claim 22, wherein the first interpolation comprises at least one among a Sinc interpolation and a Lagrange interpolation.

### Claim 24 (depends on 22)

The multi-channel audio coding device of claim 22, wherein the first interpolation comprises a hybrid interpolation, the hybrid interpolation includes both a Sinc interpolation and a Lagrange interpolation.

### Claim 25 (depends on 22)

The multi-channel audio coding device of claim 22, wherein the second interpolation comprises an overlap and add interpolation.

### Claim 26 (depends on 25)

The multi-channel audio coding device of claim 25, wherein the overlap and add interpolation is based on the first mismatch value and the second mismatch value.

### Claim 27 (depends on 25)

The multi-channel audio coding device of claim 25, wherein the overlap and add interpolation is based on a first window function and a second window function, wherein the second window function is dependent on the first window function.

### Claim 28 (depends on 21)

The multi-channel audio coding device of claim 21, further comprising a shift estimator configured to determine the first mismatch value and the second mismatch value, wherein the first mismatch value and the second mismatch value are determined based on comparisons of a reference down-sampled channel to a target down-sampled channel, wherein the reference down-sampled channel is based on the reference channel, and wherein the target down-sampled channel is based on the target channel.

### Claim 29 (depends on 21)

The multi-channel audio coding device of claim 21, further comprising:
a first input interface configured to receive a first audio signal from a first microphone; and
a second input interface configured to receive a second audio signal from a second microphone, wherein the first audio signal corresponds to one of the reference channel or the target channel, and wherein the second audio signal corresponds to the other of the reference channel or the target channel.

### Claim 30 (depends on 21)

The multi-channel audio coding device of claim 21, wherein the encoder and the network interface are integrated into a mobile device or a base station.

### Claim 31

A multi-channel audio coding apparatus comprising:
means for receiving a reference channel, the reference channel including a set of reference samples;
means for receiving a target channel, the target channel including a set of target samples;
means for determining a variation between a first mismatch value and a second mismatch value, the first mismatch value indicative of an amount of temporal mismatch between a first reference sample of the set of reference samples and a first target sample of the set of target samples, the second mismatch value indicative of an amount of temporal mismatch between a second reference sample of the set of reference samples and a second target sample of the set of target samples;
means for selecting a particular adjustment technique from a plurality of adjustment techniques based on a comparison of the variation with a first threshold;
means for using the variation subsequent to the comparison to perform the particular adjustment technique to adjust the set of target samples to generate an adjusted set of target samples;
means for generating at least one encoded channel based on the set of reference samples and the adjusted set of target samples; and
means for transmitting the at least one encoded channel.

### Claim 32 (depends on 31)

The multi-channel audio coding apparatus of claim 31, wherein means for the particular adjustment technique comprises means for selecting one of a first interpolation or a second interpolation in response to determining whether the variation exceeds the first threshold, and wherein the first interpolation is different from the second interpolation.

### Claim 33 (depends on 32)

The multi-channel audio coding apparatus of claim 32, wherein means for performing the first interpolation comprises means for performing at least one among a Sinc interpolation and a Lagrange interpolation.

### Claim 34 (depends on 32)

The multi-channel audio coding apparatus of claim 32, wherein means for performing the second interpolation comprises means for performing an overlap and add interpolation.

### Claim 35 (depends on 31)

The multi-channel audio coding apparatus of claim 31, further comprising means for determining whether to adjust the set of target samples based on one among the variation, a reference channel indicator, an energy of the reference channel and an energy of the target channel, and a transient detector.

### Claim 36 (depends on 31)

The multi-channel audio coding apparatus of claim 31, wherein a first audio signal includes one of a right channel or a left channel, and wherein a second audio signal includes the other of the right channel or the left channel, wherein the first audio signal corresponds to one of the reference channel or the target channel, and wherein the second audio signal corresponds to the other of the reference channel or the target channel.

### Claim 37

A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to perform operations comprising:
receiving, at a first device, a reference channel and a target channel, the reference channel including a set of reference samples, and the target channel including a set of target samples;
determining, at the first device, a variation between a first mismatch value and a second mismatch value, the first mismatch value indicative of an amount of temporal mismatch between a first reference sample of the set of reference samples and a first target sample of the set of target samples, the second mismatch value indicative of an amount of temporal mismatch between a second reference sample of the set of reference samples and a second target sample of the set of target samples;
selecting, at the first device, a particular adjustment technique from a plurality of adjustment techniques based on a comparison of the variation with a first threshold;
using the variation subsequent to the comparison, at the first device, to perform the particular adjustment technique to adjust the set of target samples to generate an adjusted set of target samples;
generating, at the first device, at least one encoded channel based on the set of reference samples and the adjusted set of target samples; and
transmitting the at least one encoded channel from the first device to a second device.

### Claim 38 (depends on 37)

The non-transitory computer-readable medium of claim 37, wherein the operations comprise selecting one of a first interpolation or a second interpolation as the particular adjustment technique in response to determining whether the variation exceeds the first threshold, wherein the first interpolation is different from the second interpolation.

### Claim 39 (depends on 38)

The non-transitory computer-readable medium of claim 38, wherein the first interpolation comprises at least one among a Sinc interpolation and a Lagrange interpolation.

### Claim 40 (depends on 38)

The non-transitory computer-readable medium of claim 38, wherein the first interpolation comprises a hybrid interpolation, the hybrid interpolation includes both a Sinc interpolation and a Lagrange interpolation.

### Claim 41 (depends on 38)

The non-transitory computer-readable medium of claim 38, wherein the second interpolation comprises an overlap and add interpolation.