Real-time speech and music classifier
AssigneeMicrosoft Corporation
InventorsHosam Adel Khalil, Vladimir Cuperman, Tian Wang
Life of this patent
Estimating the term…
Abstract
An efficient and accurate classification method for classifying speech and music signals, or other diverse signal types, is provided. The method and system are especially, although not exclusively, suited for use in real-time applications. Long-term and short-term features are extracted relative to each frame, whereby short-term features are used to detect a potential switching point at which to switch a coder operating mode, and long-term features are used to classify each frame and validate the potential switch at the potential switch point according to the classification and a predefined criterion.
Claims17
A method of classifying a current coding frame in a sequence of audio data frames including the current frame and at least one subsequent frame in real-time for switching a multi-mode audio coding system operated in a current coding mode between different modes, the method comprising:
- recording the sequence of audio data frames, including the current frame and the at least one subsequent frame;
- extracting at least one long-term feature and at least one short-term feature relative to each of the current frame and the at least one subsequent frame, wherein the features substantially exhibit distinct values for different signal types;
- detecting a potential switch point according to the at least one short-term feature of the current frame and the current coding mode;
- and
- determining whether to switch the current coding mode of the coding system at the potential switch point based on the at least one long-term feature.
A method for switching an audio encoder between a speech mode and a music mode for coding a sequence of audio data frames including the current frame and at least one subsequent frame, the method comprising:
- recording the sequence of frames, including the current frame and the at least one subsequent frame, in a buffer;
- extracting at least one long-term feature and at least one short-term feature relative to each of the current frame and the at least one subsequent frame, wherein the features substantially exhibit distinct values for speech and music frames;
- detecting a potential switch point according to the at least one short-term feature extracted from each of the current frame and the at least one subsequent frame;
- defining a feature space by the at least one long-term feature of each of the current frame and the at least one subsequent frame;
- generating a feature point in the feature space for each frame in the buffer, wherein a set of feature points defines a feature pattern;
- classifying each of the current frame and the at least one subsequent frame via pattern recognition relative to the feature pattern;
- and
- determining whether to switch the mode of the audio encoder according to the classification and a pre-defined switching criterion.
A coder system for coding a sequence of audio frames composed of speech data frames and music data frames including the current frame and at least one subsequent frame, the coder system comprising:
- an encoder having multiple operating modes, at least one of which is for encoding speech data and another of which is for encoding music data;
- and
- an encoding classifier in communication with the encoder, wherein the encoding classifier is adapted for determining a potential switching time for the encoder to switch its operating mode based on one or more extracted short-term features of a frame, classifying each frame in the sequence, including the current frame and the at least one subsequent frame, according to one or more long-term features according to a predefined criterion, and providing a set of classification information classifying at least one frame of the frames as a speech data or music data fame.
A method of classifying a current coding frame in a sequence of audio data frames including the current frame and at least one subsequent frame in real-time for switching a multi-mode audio coding system operated in a current coding mode between different modes, the method comprising:
- recording the sequence of audio data frames, including the current frame and the at least one subsequent frame;
- extracting at least one long-term feature and at least one short-term feature relative to each audio data frame, including the current frame and the at least one subsequent frame, wherein the features substantially exhibit distinct values for different signal types;
- determining whether to switch the current coding mode of the coding system based on the at least one extracted long-term feature;
- and
- if it is determined to switch the current coding mode of the coding system, detecting a switch point according to the at least one short-term feature of the current frame and the current coding mode, at which to switch the current coding mode of the coding system.
A method of classifying a current coding frame in a sequence of audio data frames including the current frame and at least one subsequent frame in real-time for switching a multi-mode audio coding system operated in a current coding mode between different modes, the method comprising:
- recording the sequence of audio data frames, including the current frame and the at least one subsequent frame;
- extracting at least one long-term feature relative to each audio data frame, including the current frame and the at least one subsequent frame, wherein the at least one feature substantially exhibits distinct values for different signal types;
- detecting a potential switch point according to the at least one long-term feature of the current frame and the current coding mode;
- and
- determining whether to switch the current coding mode of the coding system at the potential switch point based on the at least one long-term feature.
Dependencies are inferred from the claim text.