# US 10,504,530 B2 · Switching between transforms > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,504,530 B2 - Document ID: US10504530B2 - Status: Granted - Kind: B2 - Application number: 15/277,027 - Filed: 2016-09-27 - Published: 2017-05-04 - Granted: 2019-12-10 - Earliest claimed: 2015-11-03 - Assignees: Dolby Laboratories Licensing Corporation - Applicants: Dolby Laboratories Licensing Corporation - Inventors: Dong Shi; Xuejing Sun; Glenn N. Dickins; David Gunawan - Web page: https://patentgrep.com/patent/US10504530B2 ## CPC - G10L 19/0212: {using orthogonal transformation} - H04N 19/12: Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264 - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/179: the unit being a scene or a shot - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A method of processing data comprises processing first frequency-domain audio or video data using signal processing of a first type, and transforming the processed first frequency-domain audio or video data to processed time-domain audio or video data using a transform which is the inverse of the first transform, and transforming the processed time-domain audio or video data using a second transform which is matched to a second type of signal processing. The method further comprises identifying time-domain audio or video data for which signal processing of the first type, after transformation using the second transform, would yield satisfactory results. The method further comprises transforming the identified time-domain audio or video data to frequency-domain identified audio or video data using the second transform, instead of using the first transform, and processing the identified frequency-domain audio or video data using signal processing of the first type. ## Claims ### Claim 1 Apparatus for processing audio or video data, the apparatus comprising: an input for receiving audio or video data; a first transform module configured to generate first transform-domain audio or video data by at least applying a first transform to the audio or video data; a first signal processing module connected to the first transform module and configured to generate processed first transform-domain audio or video data by at least applying first signal processing to the first transform-domain audio or video data; a second transform module configured to generate second transform-domain audio or video data by at least applying a second transform to a selected one of the audio or video data and the processed first transform-domain audio or video data, the second transform being different from the first transform; a second signal processing module selectively connected to the second transform module and configured to generate processed second transform-domain audio or video data by at least applying second signal processing to the second transform-domain audio or video data; an analysis module configured to analyze at least one of the audio or video data, the first transform-domain audio or video data, the processed first transform-domain audio or video data, the second transform-domain audio or video data or the processed second transform-domain audio or video data, and to determine at least in part therefrom whether to apply the first signal processing or the second signal processing, wherein the analysis module determines whether to apply the first signal processing or the second signal processing based on at least one of a latency criterion, a complexity criterion, and a band separation criterion; and a path selection module configured to, in response to the analysis module determining to apply the first signal processing, selectively channel the audio or video data along a first path from the input through the first transform module and the first signal processing module, thereby producing the processed first transform-domain audio or video data, and send the processed first transform-domain audio or video data to a third signal processing module without applying any further transform, and in response to the analysis module determining to apply the second signal processing, selectively channel the audio or video data along a second path from the input through the second transform module and the second signal processing module, thereby producing the processed second transform-domain audio or video data, and send the processed second transform-domain audio or video data to the third signal processing module without applying any further transform. ### Claim 2 (depends on 1) The apparatus as recited in claim 1, wherein: the first transform is matched to the first signal processing; and the second transform is matched to subsequent processing, at least a part of said subsequent processing being performed by the third signal processing module. ### Claim 3 (depends on 1) The apparatus as recited in claim 1, wherein: the determining to apply the second signal processing comprises determining that a signal to noise ratio of the time-domain audio or video data, of the first transform-domain audio or video data, of the processed first transform-domain audio or video data, of the second transform-domain audio or video data or of the processed second transform-domain audio or video data is below a predetermined threshold. ### Claim 4 (depends on 1) The apparatus as recited in claim 1, wherein the first transform comprises one of: a modulated complex lapped transform, a modified discrete sine transform, a quadrature mirror filter-bank transform, a DCT-I, II or IV transform or a variant or approximation of a Karhunen-Loève transform. ### Claim 5 (depends on 1) The apparatus as recited in claim 1, wherein the second transform comprises one of: a modified discrete cosine transform, a modified discrete Fourier transform, a complex quadrature mirror filter-bank transform, or a discrete Fourier transform. ### Claim 6 (depends on 1) The apparatus as recited in claim 1, wherein the second signal processing comprises at least one of: echo cancellation processing, noise estimation or suppression, or multi-band dynamic range control or equalization. ### Claim 7 (depends on 1) The apparatus as recited in claim 1, wherein the first signal processing comprises at least one of: echo cancellation, or complex-valued frequency domain convolution or filtering. ### Claim 8 (depends on 1) The apparatus as recited in claim 1, wherein said third signal processing module is configured to perform third signal processing, the third signal processing comprising data compression. ### Claim 9 Apparatus for processing audio or video data, the apparatus comprising: an input for receiving time-domain audio or video data; a first transform module configured to generate first transform-domain audio or video data by at least applying a first transform to the time-domain audio or video data; a first signal processing module configured to generate first processed transform-domain audio or video data by at least applying first signal processing to the first transform-domain audio or video data; a conversion module configured to convert the first transform-domain audio or video data directly into second transform-domain audio or video data, the second transform-domain audio or video data being equivalent to data generated by applying a second transform to the audio or video data, the second transform being different from the first transform; a second signal processing module connected to the conversion module and configured to generate processed second transform-domain audio or video data by at least applying second signal processing to the second transform-domain audio or video data; an analysis module configured to analyze at least one of the time-domain audio or video data, the first transform-domain audio or video data, the processed first transform-domain audio or video data, the second transform-domain audio or video data or the processed second transform-domain audio or video data, and to determine at least in part therefrom whether to apply the first signal processing or the second signal processing, wherein the analysis module determines whether to apply the first signal processing or the second signal processing based on at least one of a latency criterion, a complexity criterion, and a band separation criterion; and a path selection module configured to, in response to the analysis module determining to apply the first signal processing, selectively channel the time-domain audio or video data along a first path from the first transform module through the first signal processing module, thereby producing the first processed transform-domain audio or video data, and send the first processed transform-domain audio or video data to a third signal processing module without applying any further transform, and in response to the analysis module determining to apply the second signal processing, selectively channel the time-domain audio or video data along a second path from the first transform module through the conversion module and the second signal processing module, thereby producing the processed second transform-domain audio or video data, and send the processed second transform-domain audio or video data to the third signal processing module without applying any further transform. ### Claim 10 (depends on 9) The apparatus as recited in claim 9, wherein: the first transform is matched to the first signal processing; and the second transform is matched to subsequent processing, at least a part of said subsequent processing being performed by the third signal processing module. ### Claim 11 (depends on 9) The apparatus as recited in claim 9, wherein: the determining to apply the second signal processing comprises determining that a signal to noise ratio of the time-domain audio or video data, of the first transform-domain audio or video data, of the processed first transform-domain audio or video data, of the second transform-domain audio or video data or of the processed second transform-domain audio or video data is below a predetermined threshold. ### Claim 12 (depends on 9) The apparatus as recited in claim 9, wherein the first transform comprises one of: a modulated complex lapped transform, a modified discrete sine transform, a quadrature mirror filter-bank transform, a DCT-I, II or IV transform or a variant or approximation of a Karhunen-Loève transform. ### Claim 13 (depends on 9) The apparatus as recited in claim 9, wherein the second transform comprises one of: a modified discrete cosine transform, a modified discrete Fourier transform, a complex quadrature mirror filter-bank transform, or a discrete Fourier transform. ### Claim 14 (depends on 9) The apparatus as recited in claim 9, wherein the second signal processing comprises at least one of: echo cancellation processing, noise estimation or suppression, or multi-band dynamic range control or equalization. ### Claim 15 (depends on 9) The apparatus as recited in claim 9, wherein the first signal processing comprises at least one of: echo cancellation, or complex-valued frequency domain convolution or filtering. ### Claim 16 (depends on 9) The apparatus as recited in claim 9, wherein said third signal processing module is configured to perform third signal processing, the third signal processing comprising data compression. ### Claim 17 Apparatus for processing audio or video data, the apparatus comprising: an input for receiving first transform-domain audio or video data; a first signal processing module configured to generate processed first transform-domain audio or video data by at least applying first signal processing to the first transform-domain audio or video data; a first inverse transform module selectively connected to the first signal processing module and configured to generate audio or video data by at least applying a first inverse transform to the processed first transform-domain audio or video data, the first inverse transform being the inverse of a first transform; a first transform module connected to the first inverse transform module and configured to generate second transform-domain audio or video data by at least applying a second transform to the audio or video data, the second transform being different from the first transform; a second signal processing module connected to the first transform module and configured to generate processed second transform-domain audio or video data by at least applying second signal processing to the second transform-domain audio or video data; a third signal processing module selectively connected to the first signal processing module and configured to generate twice processed first transform-domain audio or video data by at least applying third signal processing to the processed first transform-domain audio or video data; an analysis module configured to analyze at least one of the first transform-domain audio or video data, the processed first transform-domain audio or video data, the audio or video data, the second transform-domain audio or video data or the processed second transform-domain audio or video data, and to determine at least in part therefrom whether to apply the second signal processing or the third signal processing; and a path selection module configured to, in response to the analysis module determining to apply the second signal processing, selectively channel the first transform-domain audio or video data along a first path from the input to the second signal processing module, thereby producing the processed first transform-domain audio or video data, and send the processed first transform-domain audio or video data to the first inverse transform module or a further inverse transform module configured to generate audio or video data by at least applying a first inverse transform to the first transform-domain audio or video data, and in response to the analysis module determining to apply the third signal processing, selectively channel the first transform-domain audio or video data along a second path from the input to the first signal processing module, thereby producing the processed first transform-domain audio or video data, and send the processed first transform-domain audio or video data to the third signal processing module. ### Claim 18 (depends on 17) The apparatus as recited in claim 17, wherein: the second transform is selected for its suitability for the first signal processing. ### Claim 19 Apparatus for processing audio or video data, the apparatus comprising: an input for receiving first transform-domain audio or video data; a first signal processing module configured to generate processed first transform-domain audio or video data by at least applying first signal processing to the first transform-domain audio or video data; a conversion module configured to convert the first transform-domain audio or video data directly into second transform-domain audio or video data; a second signal processing module connected to the conversion module and configured to generate processed second transform-domain audio or video data by at least applying second signal processing to the second transform-domain audio or video data; an analysis module configured to analyze at least one of the first transform-domain audio or video data, the processed first transform-domain audio or video data, the second transform-domain audio or video data or the processed second transform-domain audio or video data, and to determine at least in part therefrom whether to apply the first signal processing or the second signal processing; and a path selection module configured to, in response to the analysis module determining to apply the first signal processing, selectively channel the first transform-domain audio or video data along a first path from the input to the first signal processing module, thereby producing the processed first transform-domain audio or video data, and send the processed first transform-domain audio or video data to an inverse transform module configured to generate audio or video data by at least applying a first inverse transform to the first transform-domain audio or video data, and in response to the analysis module determining to apply the second signal processing, selectively channel the first transform-domain audio or video data along a second path from the input to the conversion module and the second signal processing module, thereby producing the processed second transform-domain audio or video data. ### Claim 20 (depends on 19) The apparatus as recited in claim 19, wherein: the second transform is selected for its suitability for the first signal processing. ### Claim 21 (depends on 1) The apparatus as recited in claim 1, wherein the latency criterion relates to a delay introduced by a given transform and an inverse of the given transform. ### Claim 22 (depends on 21) The apparatus as recited in claim 21, wherein the delay comprises a first time to accumulate a plurality of input samples and a second time to reconstruct a signal from the plurality of samples. ### Claim 23 (depends on 1) The apparatus as recited in claim 1, wherein the latency criterion relates to an acceptable latency that is compared with a current indication of signal depth from at least one module. ### Claim 24 (depends on 1) The apparatus as recited in claim 1, wherein the complexity criterion relates to one or more of a number of required operations, a number of multiplications per input sample, and a memory requirement for storing intermediate variables. ### Claim 25 (depends on 1) The apparatus as recited in claim 1, wherein the complexity criterion relates to an indication of processor availability. ### Claim 26 (depends on 1) The apparatus as recited in claim 1, wherein the band separation criterion relates to an extent to which each frequency band in a given transform domain is independent from each other frequency band in the given transform domain. ### Claim 27 (depends on 17) The apparatus as recited in claim 17, wherein the first transform comprises one of: a modulated complex lapped transform, a modified discrete sine transform, a quadrature mirror filter-bank transform, a DCT-I, II or IV transform or a variant or approximation of a Karhunen-Loève transform. ### Claim 28 (depends on 17) The apparatus as recited in claim 17, wherein the second transform comprises one of: a modified discrete cosine transform, a modified discrete Fourier transform, a complex quadrature mirror filter-bank transform, or a discrete Fourier transform. ### Claim 29 (depends on 17) The apparatus as recited in claim 17, wherein the second signal processing comprises at least one of: echo cancellation processing, noise estimation or suppression, or multi-band dynamic range control or equalization. ### Claim 30 (depends on 17) The apparatus as recited in claim 17, wherein the first signal processing comprises at least one of: echo cancellation, or complex-valued frequency domain convolution or filtering. ### Claim 31 (depends on 17) The apparatus as recited in claim 17, wherein said third signal processing module is configured to perform third signal processing, the third signal processing comprising data compression. ### Claim 32 (depends on 19) The apparatus as recited in claim 19, wherein the first transform comprises one of: a modulated complex lapped transform, a modified discrete sine transform, a quadrature mirror filter-bank transform, a DCT-I, II or IV transform or a variant or approximation of a Karhunen-Loève transform. ### Claim 33 (depends on 19) The apparatus as recited in claim 19, wherein the second transform comprises one of: a modified discrete cosine transform, a modified discrete Fourier transform, a complex quadrature mirror filter-bank transform, or a discrete Fourier transform. ### Claim 34 (depends on 19) The apparatus as recited in claim 19, wherein the second signal processing comprises at least one of: echo cancellation processing, noise estimation or suppression, or multi-band dynamic range control or equalization. ### Claim 35 (depends on 19) The apparatus as recited in claim 19, wherein the first signal processing comprises at least one of: echo cancellation, or complex-valued frequency domain convolution or filtering. ### Claim 36 (depends on 19) The apparatus as recited in claim 19, wherein said third signal processing module is configured to perform third signal processing, the third signal processing comprising data compression.