# US 8,543,228 B2 · Coded domain audio analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,543,228 B2
- Document ID: US8543228B2
- Status: Granted
- Kind: B2
- Application number: 11/697,584
- Filed: 2007-04-06
- Published: 2008-10-09
- Granted: 2013-09-24
- Earliest claimed: 2007-04-06
- Assignees: Nokia Corporation
- Applicants: Juha Ojanpera
- Inventors: Juha Ojanpera
- Web page: https://patentgrep.com/patent/US8543228B2

## CPC

- G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)}
- G06F 16/683: using metadata automatically derived from the content
- G10H 1/0008: {Associated control or indicating means}
- G10L 25/00: Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 (muting semiconductor-based amplifiers when some special characteristics of a signal are sensed by a speech detector, e.g. sensing when no signal is present, H03G3/34)
- G10H 2210/076: for extraction of timing, tempo; Beat detection
- G10H 2240/061: MP3, i.e. MPEG-1 or MPEG-2 Audio Layer III, lossy audio compression
- G10H 2240/141: Library retrieval matching, i.e. any of the steps of matching an inputted segment or phrase with musical database contents, e.g. query by humming, singing or playing; the steps may include, e.g. musical analysis of the input, musical feature extraction, query formulation, or details of the retrieval process
- G10H 2250/031: Spectrum envelope processing
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

The present invention provides methods, computer-readable media, and apparatuses for determining a measure a similarity of the audio content of a plurality of files. Quantized samples are converted to energy values from files having audio content with compressed formats. Energy characteristics are determined from the energy values and are matched to obtain a greatest content match for a plurality of delay instances. A selected delay value corresponds to a determined offset between the files for relating the audio content of the files. A degree of similarity is determined to relate the audio content of the files. A frequency spectrum is partitioned into a plurality of frequency bands, and a total energy value is determined for each frequency band. The total energy values are correlated for the files.

## Claims

### Claim 1

A method comprising:
accessing a first file and a second file, wherein the first file and the second file include compressed audio content;
collecting compressed audio content samples from the compressed audio content of the first file and from the compressed audio content of the second file;
converting the compressed audio content samples to energy values;
determining energy characteristics from the energy values;
matching the energy characteristics to obtain a greatest content match with respect to a plurality of delay instances; and
selecting a delay value based on the greatest content match, the selected delay value corresponding to an offset between the first file and the second file.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining a degree of similarity that relates content of the first file and the second file; and
displaying at least one of the selected delay value and the degree of similarity.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
wherein selecting the delay value includes determining the delay value from delay characteristics of the plurality of delay instances.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
determining a limited number of the compressed audio content samples to be extracted from a bit stream; and
extracting the limited number of the compressed audio content samples from the bitstream.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
de-quantizing the compressed audio content samples to obtain de-quantized samples corresponding to amplitude levels.

### Claim 6 (depends on 5)

The method of claim 5, further comprising:
combining the de-quantized samples from a right channel and a left channel of a stereo audio signal to form the energy values.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
partitioning a frequency spectrum into a plurality of frequency bands; and
determining a total energy value for each said frequency band for the first file and the second file.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
correlating the total energy values for the first file and the second file.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
replacing the first file or the second file with a reference file; and
repeating the steps of matching the energy characteristics and selecting the delay value based on the reference file.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
collecting second compressed audio content samples from compressed audio content of a third file;
converting the second compressed audio content samples to second energy values;
determining second energy characteristics from the second energy values;
matching the second energy characteristics to obtain a second greatest content match with respect to a second plurality of delay instances; and
select a second selected delay value from the second plurality of delay instances, the second selected delay value corresponding to an offset between the first file and the third file for relating content of the first file and the third file.

### Claim 11 (depends on 1)

The method of claim 1, wherein the compressed audio content of the first file and the compressed audio content of the second file both conform to an advanced audio coding (AAC) format or a Moving Picture Experts Group (MPEG)-1 Layer III (MP3) format.

### Claim 12 (depends on 1)

The method of claim 1, wherein the compressed audio content of the first file conforms to a first compression format and the compressed audio content of the second file conforms to a second compression format different than the first compression format.

### Claim 13 (depends on 1)

The method of claim 1, further comprising:
organizing a data structure that includes the first file and the second file based on a determined similarity between the first file and the second file.

### Claim 14 (depends on 1)

The method of claim 1, wherein both the first file and the second file include compressed multi-media content.

### Claim 15

An apparatus comprising:
one or more processors; and
at least one memory storing computer-executable instructions that, with the one or more processors, cause the apparatus to at least:
access a first file and a second file, wherein the first file and the second file both include compressed audio content;
collect compressed audio content samples from the compressed audio content of the first file and from the compressed audio content of the second file,
convert the compressed audio content samples to energy values,
determine energy characteristics from the energy values,
match the energy characteristics to obtain a greatest content match with respect to a plurality of time delay instances, and
select a delay value based on the greatest content match, the selected delay value corresponding to an offset between the first file and the second file.

### Claim 16 (depends on 15)

The apparatus of claim 15, wherein the at least one memory further stores computer-executable instructions that, with the one or more processors, cause the apparatus to:
determine a degree of similarity that relates content of the first file and the second file, and
display at least one of the degree of similarity and the selected delay value.

### Claim 17

A non-transitory computer-readable medium storing computer-executable instructions that, when executed, cause an apparatus to at least:
access a first file and a second file, wherein the first file and the second file include compressed audio content;
collect compressed audio content samples from the compressed audio content of the first file and from the compressed audio content of the second file;
convert the compressed audio content samples to energy values;
determine energy characteristics from the energy values for the first file and the second file;
match the energy characteristics to obtain a greatest content match with respect to a plurality of delay instances; and
select a delay value based on the greatest content match, the selected delay value corresponding to an offset between the first file and the second file.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, further storing computer-executable instructions that, when executed, cause the apparatus to:
determine a degree of similarity that relates content of the first file and the second file, and
display at least one of the degree of similarity and the selected delay value.

### Claim 19 (depends on 17)

The non-transitory computer-readable medium of claim 17, further storing computer-executable instructions that, when executed, cause the apparatus to:
partition a frequency spectrum into a plurality of frequency bands; and
determine a total energy value in each said frequency band for the first file and the second file.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, further storing computer-executable instructions that, when executed, cause the apparatus to:
correlate the total energy values for the first file and the second file.

### Claim 21 (depends on 17)

The non-transitory computer-readable medium of claim 17, further storing computer-executable instructions that, when executed, cause the apparatus to:
determine a limited number of the compressed audio content samples to be extracted from a bit stream; and
extract the limited number of the compressed audio content samples from the bitstream.

### Claim 22 (depends on 17)

The non-transitory computer-readable medium of claim 17, wherein both the first file and the second file include compressed multi-media content.

### Claim 23 (depends on 15)

The apparatus of claim 15, wherein the at least one memory further stores computer-executable instructions that, with the one or more processors, cause the apparatus to:
determine a limited number of the compressed audio content samples to be extracted from a bit stream; and
extract the limited number of the compressed audio content samples from the bitstream.

### Claim 24 (depends on 15)

The apparatus of claim 15, wherein both the first file and the second file include multi-media content.

### Claim 25 (depends on 1)

The method of claim 1, wherein collecting the compressed audio content samples includes collecting a portion of the compressed audio content samples at a rate dependent on one or more encoding properties of the first file, wherein the encoding properties include a frame size used to encode the first file and a sampling rate used to encode the first file.

### Claim 26 (depends on 1)

The method of claim 1, wherein the delay value is in a block domain based on a compression format of the first file or a compression format of the second file.