# US 10,311,892 B2 · Apparatus and method for encoding or decoding audio signal with intelligent gap filling in the spectral domain > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,311,892 B2 - Document ID: US10311892B2 - Status: Granted - Kind: B2 - Application number: 15/834,260 - Filed: 2017-12-07 - Published: 2018-04-12 - Granted: 2019-06-04 - Earliest claimed: 2013-07-22 - Assignees: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V. - Applicants: Fraunhofer-Gesellschaft zur Foerderung der angewandten Forschung e.V. - Inventors: Sascha Disch; Frederik Nagel; Ralf Geiger; Balaji Nagendran Thoshkahna; Konstantin Schmidt; Stefan Bayer; Christian Neukam; Bernd Edler; Christian Helmrich - Web page: https://patentgrep.com/patent/US10311892B2 ## CPC - G10L 19/008: Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing - G10L 19/0204: {using subband decomposition} - G10L 19/0208: {Subband vocoders} - G10L 19/0212: {using orthogonal transformation} - G10L 19/022: Blocking, i.e. grouping of samples in time; Choice of analysis windows; Overlap factoring - G10L 19/025: Detection of transients or attacks for time/frequency resolution switching - G10L 19/03: Spectral prediction for preventing pre-echo; Temporary noise shaping [TNS], e.g. in MPEG2 or MPEG4 - G10L 19/032: Quantisation or dequantisation of spectral components - G10L 19/06: Determination or coding of the spectral characteristics, e.g. of the short-term prediction coefficients - G10L 21/0388: Details of processing therefor - G10L 25/06: the extracted parameters being correlation coefficients - G10L 25/18: the extracted parameters being spectral information of each sub-band - G10L 25/21: the extracted parameters being power information - H04S 1/007: {in which the audio signals are in digital form (data reduction aspects thereof based on psychoacoustics G10L19/02)} - G10L 19/02: using spectral analysis, e.g. transform vocoders or subband vocoders - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - H04S: STEREOPHONIC SYSTEMS ## Abstract An apparatus for decoding an encoded audio signal, includes a spectral domain audio decoder for generating a first decoded representation of a first set of first spectral portions, the decoded representation having a first spectral resolution; a parametric decoder for generating a second decoded representation of a second set of second spectral portions having a second spectral resolution being lower than the first spectral resolution; a frequency regenerator for regenerating every constructed second spectral portion having the first spectral resolution using a first spectral portion and spectral envelope information for the second spectral portion; and a spectrum time converter for converting the first decoded representation and the reconstructed second spectral portion into a time representation. ## Claims ### Claim 1 Audio encoder for encoding an audio signal to obtain an encoded audio signal, comprising a processor and a memory including instructions that, when executed by the processor, cause the encoder to: convert, by a time-spectrum converter an audio signal comprising a sampling rate into a spectral representation; analyze, by a spectral analyzer, the spectral representation for determining a first set of first spectral portions to be encoded with a first spectral resolution and a different second set of second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being smaller than the first spectral resolution, wherein a first spectral portion of the first set of first spectral portions is placed, with respect to frequency, between two second spectral portions of the second set of second spectral portions; generate by, a spectral domain audio encoder, a first encoded representation of the first set of first spectral portions comprising the first spectral resolution, wherein the first encoded representation comprises an encoded representation of the first spectral portion of the first set of first spectral portions that is placed, with respect to frequency, between the two second spectral portions of the second set of second spectral portions calculate, by a parametric coder, spectral envelope information for the second set of second spectral portions, the spectral envelope information comprising the second spectral resolution, wherein the spectral envelope information comprises spectral envelope information of the two second spectral portions of the second set of second spectral portions; and transmitting, by an output interface, the encoded audio signal to a decoder, wherein the encoded audio signal comprises the first encoded representation, and the spectral envelope information, and wherein one or more of the time-spectrum converter, the spectral analyzer, the spectral domain audio encoder, and the parametric coder is implemented, at least in part, by one or more hardware elements of the audio encoder. ### Claim 2 (depends on 1) Audio encoder of claim 1, wherein the parametric coder is configured for calculating similarities between source ranges comprising first spectral portions of the first set of first spectral portions and target ranges comprising second spectral portions of the second set of second spectral portions and for determining, based on calculated similarities, for a second spectral portion of the second set of second spectral portions, a first spectral portion of the first set of first spectral portions matching with the second spectral portion of the second set of second spectral portions and for providing the matching information on the first spectral portion of the first set of first spectral portions matching with the second spectral portion of the second set of second spectral portions into an encoded representation. ### Claim 3 (depends on 2) Audio encoder of claim 2, wherein the spectral analyzer is configured for analyzing the spectral representation up to a maximum analysis frequency being at least one quarter of a sampling frequency of the audio signal. ### Claim 4 (depends on 1) Audio encoder of claim 1, wherein the time-spectrum converter is configured for windowing the audio signal with overlapping windows to acquire a sequence of windowed frames, a windowed frame comprising a first number of samples, and for converting the sequence of frames into the spectral representation to acquire spectral frames, a spectral frame comprising a second number of spectral samples, the second number being smaller than the first number. ### Claim 5 (depends on 1) Audio encoder of claim 1, wherein the spectral domain audio encoder is configured to process a sequence of frames of spectral values for quantization and entropy coding, wherein, in a frame of the sequence of frames, spectral values of the second set of second portions are set to zero or wherein, in a frame, spectral values of the first set of first spectral portions and the second set of the second spectral portions are present and wherein, during processing or subsequent to processing, spectral values in the second set of second spectral portions are set to zero. ### Claim 6 (depends on 1) Audio encoder of claim 1, wherein the spectral domain audio encoder is configured to generate the first encoded representation of the first set of first spectral portions comprising a Nyquist frequency defined by the sampling rate of the audio signal. ### Claim 7 (depends on 1) Audio encoder of claim 1, wherein the spectral domain audio encoder is configured to provide the first encoded representation so that, for a frame of a sampled audio signal, the first encoded representation comprises the first set of first spectral portions and the second set of second spectral portions, wherein the spectral values in the second set of second spectral portions are encoded as zero values or as noise values. ### Claim 8 (depends on 1) Audio encoder of claim 1, wherein the spectral analyzer is configured to analyze the spectral representation starting, with respect to frequency, with a frequency gap filling start frequency, and ending, with respect to frequency, with a maximum frequency represented by a maximum frequency comprised in the spectral representation, and wherein a spectral portion extending from a minimum frequency up to the frequency gap filling start frequency belongs to the first set of first spectral portions to be encoded by the spectral domain audio encoder. ### Claim 9 (depends on 1) Audio encoder of claim 1, wherein the spectral analyzer is configured to apply a tonal mask processing to at least a portion of the spectral representation so that tonal components and non-tonal components are separated from each other, wherein the first set of the first spectral portions comprises the tonal components and wherein the second set of the second spectral portions comprises the non-tonal components. ### Claim 10 (depends on 1) Audio encoder of claim 1, wherein the spectral domain audio encoder comprises a psycho-acoustic module for quantizing the first set of first spectral portions under consideration of a masking threshold determined in the psycho-acoustic module. ### Claim 11 (depends on 1) Audio encoder of claim 1, wherein the time-spectrum converter is configured to apply a Modified Discrete Cosine Transform. ### Claim 12 (depends on 1) Audio encoder of claim 1, wherein the spectral analyzer is configured to separate spectral portions having tonal components from spectral portions having non-tonal components in the spectral representation, wherein the spectral analyzer is configured to further analyze the spectral portions having the non-tonal components to be reconstructed by using a spectral portion from the first set of spectral portions, wherein the spectral analyzer is configured to determine noise-like spectral portions in the non-tonal components to be reconstructed by noise filling, wherein the first set of first spectral portions comprises the tonal components, wherein the second set of second spectral portions comprises the spectral portions having the non-tonal components, and wherein a third set of third spectral portions comprises the noise-like spectral portions to be reconstructed by noise filling, wherein the parametric coder is configured for introducing an energy information for the noise-like spectral portions into the second encoded representation. ### Claim 13 (depends on 1) Audio encoder of claim 1, wherein the spectral analyzer is configured to analyze the spectral representation in a frequency range starting from a frequency gap filling start frequency and extending to frequencies higher than the frequency gap filling start frequency, wherein the spectral domain audio encoder is configured to encode at least a third spectral portion in the spectral representation comprising frequencies below the frequency gap filling start frequency with a spectral resolution lower than the first spectral resolution by setting spectral values in at least the third spectral portion to zero and by calculating and encoding the spectral envelope information indicating an energy in at least the third spectral portion. ### Claim 14 Method for encoding an audio signal to obtain an encoded audio signal, comprising: converting an audio signal comprising a sampling rate into a spectral representation; analyzing the spectral representation for determining a first set of first spectral portions to be encoded with a first spectral resolution and a different second set of second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being smaller than the first spectral resolution, wherein a first spectral portion of the first set of first spectral portions is placed, with respect to frequency, between two second spectral portions of the second set of second spectral portions; generating a first encoded representation of the first set of first spectral portions comprising the first spectral resolution, wherein the first encoded representation comprises an encoded representation of the first spectral portion of the first set of first spectral portions that is placed, with respect to frequency, between the two second spectral portions of the second set of second spectral portions; calculating spectral envelope information for the second set of second spectral portions, the spectral envelope information comprising the second spectral resolution, wherein the spectral envelope information comprises spectral envelope information of the two second spectral portions of the second set of second spectral portions; and transmitting, by an output interface, the encoded audio signal to a decoder, wherein the encoded audio signal comprises the first encoded representation, and the spectral envelope information, and wherein one or more of the converting, the analyzing, the generating, and the calculating is implemented, at least in part, by one or more hardware elements of an audio signal processing device. ### Claim 15 Non-transitory digital storage medium having computer-readable code stored thereon to perform, when running on a computer or a processor, a method for encoding an audio signal to obtain an encoded audio signal, the method comprising: converting an audio signal comprising a sampling rate into a spectral representation; analyzing the spectral representation for determining a first set of first spectral portions to be encoded with a first spectral resolution and a different second set of second spectral portions to be encoded with a second spectral resolution, the second spectral resolution being smaller than the first spectral resolution, wherein a first spectral portion is placed, with respect to frequency, between two second spectral portions; generating a first encoded representation of the first set of spectral portions comprising the first spectral resolution, wherein the first encoded representation comprises an encoded representation of the first spectral portion of the first set of first spectral portions that is placed, with respect to frequency, between the two second spectral portions of the second set of second spectral portions; and calculating spectral envelope information for the second set of second spectral portions, the spectral envelope information comprising the second spectral resolution, wherein the spectral envelope information comprises spectral envelope information of the two second spectral portions of the second set of second spectral portions; and transmitting, by an output interface, the encoded audio signal to a decoder, wherein the encoded audio signal comprises the first encoded representation, and the spectral envelope information.