US 12,586,552 B2 · Multi-level audio segmentation using deep embeddings

Abstract

Embodiments are disclosed for generating an audio segmentation of an audio sequence using deep embeddings. In particular, in one or more embodiments, the disclosed systems and methods comprise receiving an input including an audio sequence and extracting features for each frame of the audio sequence, where each frame is associated with a beat of the audio sequence. The method may further comprise clustering frames of the audio sequence into one or more clusters based on the extracted features and generating segments of the audio sequence based on the clustered frames, where each segment includes frames of the audio sequence from a same cluster. The method may further comprise constructing a multi-level audio segmentation of the audio sequence and performing a segment fusioning process that merges shorter segments with neighboring segments based on cluster assignments.

Claims20

Dependent claims

A computer-implemented method comprising:

receiving an input including an audio sequence and a first segment value indicating a number of clusters to group frames of the audio sequence into;
extracting features for each frame of the audio sequence, each frame associated with a beat of the audio sequence;
clustering the frames of the audio sequence into one or more clusters based on the extracted features and the first segment value;
generating segments of the audio sequence based on the clustered frames, each segment of the audio sequence formed by grouping consecutive frames of the audio sequence from a same cluster of the one or more clusters;
and
generating a first representation of the audio sequence using the generated segments of the audio sequence, wherein a subset of the generated segments with durations less than a duration threshold are fused with neighboring segments based on a second representation of the audio sequence generated using a second segment value.

3 more elements

A non-transitory computer-readable storage medium including instructions stored thereon which, when executed by at least one processor, cause the at least one processor to:

receive an input including an audio sequence and a first segment value indicating a number of clusters to group frames of the audio sequence into;
extract features for each frame of the audio sequence, each frame associated with a beat of the audio sequence;
cluster the frames of the audio sequence into one or more clusters based on the extracted features and the first segment value;
generate segments of the audio sequence based on the clustered frames, each segment of the audio sequence formed by grouping consecutive frames of the audio sequence from a same cluster of the one or more clusters;
and
generate a first representation of the audio sequence using the generated segments of the audio sequence, wherein a subset of the generated segments with durations less than a duration threshold are fused with neighboring segments based on a second representation of the audio sequence generated using a second segment value.

3 more elements

A system, comprising:

a computing device including a memory and at least one processor, the computing device implementing an audio processing system,
wherein the memory includes instructions stored thereon which, when executed, cause the audio processing system to:
receive an input including an audio sequence and a first segment value indicating a number of clusters to group frames of the audio sequence into;
extract features for each frame of the audio sequence, each frame associated with a beat of the audio sequence;
cluster the frames of the audio sequence into one or more clusters based on the extracted features and the first segment value;
generate segments of the audio sequence based on the clustered frames, each segment of the audio sequence formed by grouping consecutive frames of the audio sequence from a same cluster of the one or more clusters;
and
generate a first representation of the audio sequence using the generated segments of the audio sequence, wherein a subset of the generated segments with durations less than a duration threshold are fused with neighboring segments based on a second representation of the audio sequence generated using a second segment value.

5 more elements

Dependencies are inferred from the claim text.

Multi-level audio segmentation using deep embeddings

Life of this patent

Multi-level audio segmentation using deep embeddings

Life of this patent

Abstract

Claims20