Method and apparatus for automatically recognizing input audio and/or video streams

Apparatus for forming audio features from an input audio stream, comprising:

• a receiver for receiving the input audio stream and separating the received audio stream into a plurality of different frequency bands;
• and
• processor structure for (i) extracting energy from each of the plurality of frequency bands, (ii) summing the energy extracted from each of the plurality of frequency bands, (iii) forming multiple feature time series waveforms from the summed energy, (iv) forming overlapping time intervals of the multiple feature time series waveforms such that the overlapping time intervals encompass the entire received audio stream, (v) determining the information content of each feature from each time interval, (vi) rank-ordering each of the features of the time interval segments according to their information content, and (vii) transforming each of the rank-ordered features of the time interval segments to produce complex spectra;
• and
• a memory for storing the transformed complex spectra.

A method for recognizing an input data stream, comprising the steps of:

• receiving the input data stream;
• randomly selecting any one time interval from the received data stream;
• forming a first plurality of feature time series waveforms respectively corresponding to distinct portions of the received data stream;
• rank ordering features of the first plurality of waveforms according to their information content;
• retrieving a second plurality of feature time series waveforms;
• correlating the first plurality of feature time series waveforms with the second plurality of feature time series waveforms in an order corresponding to (i) a map of candidate patterns from the second plurality of feature time series waveforms that best match the rank ordering of the first plurality of feature time series waveforms and (ii) the rank ordering of second plurality of feature time series waveforms;
• and
• designating a recognition when a joint correlation probability value of multiple feature correlations between the first plurality of feature time series waveforms and at least one of the second plurality of feature time series waveforms indicates that random detection is not probable.

A method for forming video features from an input video stream, comprising the steps of:

• receiving an input video stream which corresponds to a video screen having a plurality of regions, the video screen comprising a plurality of pixels having luminance and chrominance;
• summing the pixel values of at least one of the intensity and the color of video signals corresponding to each of the plural areas of the video screen;
• forming a set of low rate time series feature waveforms from the pixels summed over each of the plural areas of the video screen;
• forming overlapping time intervals of the multiple feature waveforms such that the overlapping time intervals encompass the entire received video frame sequence;
• determining the most distinctive information from each time interval;
• rank-ordering the features of the time interval segments according to their information content;
• transforming each feature of the rank-ordered time interval segments to produce complex spectra;
• and
• storing the transformed complex spectra as video features.

A method for forming audio features from an audio stream, comprising the steps of:

• receiving the input audio stream and separating the received audio stream into a plurality of different frequency bands;
• extracting energy from the plurality of frequency bands;
• summing the energy extracted from each of the plurality of frequency bands;
• forming multiple feature waveforms from the summed energy;
• forming overlapping time intervals of the multiple feature waveforms such that the overlapping time intervals encompass the entire received audio stream;
• determining the most distinctive information from each time interval;
• rank-ordering features of the time interval segments according to their distinctiveness;
• transforming the rank-ordered time interval segments to produce complex spectra;
• and
• storing the transformed complex spectra as audio features.

A computer readable storage medium for storing a program which causes one or more computers to recognize an input data stream, the stored program causing the one or more computers to:

• receive the input data stream;
• randomly select any time interval of the received data stream;
• form a first plurality of feature time series waveforms from the received data stream which respectively correspond to spectrally distinct portions of the received data stream;
• store a second plurality of feature time series waveforms;
• correlate the first plurality of feature time series waveforms with the second plurality of feature time series waveforms in an order corresponding to (i) a map of candidate patterns from the second plurality of feature time series waveforms that best match the rank ordering of the first plurality of feature time series waveforms and (ii) the rank ordering of second plurality of feature time series waveforms;
• and
• designate a recognition when a joint correlation probability value of multiple feature correlations between the first plurality of feature time series waveforms and at least one of the second plurality of feature time series waveforms indicates that random detection is not probable.

A computer readable storage medium which stores a program which causes one or more computers to form video features from an input video stream, the program causing the at least one computer to:

• receive an input video stream which corresponds to a video screen having a plurality of regions, the video screen comprising a plurality of pixels having intensity and color;
• sum the pixel values of at least one of the intensity and the color of video signals corresponding to each of the plural areas of the video screen;
• form a set of low rate time series feature waveforms from the pixels summed over each of the plural areas of the video screen;
• form overlapping time intervals of the multiple feature waveforms such that the overlapping time intervals encompass the entire received video frame sequence;
• determine the most distinctive information from each time interval;
• rank-order the features of the time interval segments according to their information content;
• transform the rank-ordered features of each of the time interval segments to produce complex spectra;
• and
• store the transformed complex spectra as video features.

A computer readable storage medium which stores a program that causes one or more computers to form audio features from an audio stream, the program causing the one or more computers to:

• receive an input audio stream and separating the received audio stream into a plurality of different frequency bands;
• extract energy from the plurality of frequency bands;
• sum the energy extracted from each of the plurality of frequency bands;
• form multiple feature time series waveforms from the summed energy;
• form overlapping time intervals of the multiple feature streams such that the overlapping time intervals encompass the entire received audio stream;
• determine the most distinctive information from each time interval;
• rank-order the features of the time interval segments according to their information content;
• transform the rank-ordered time interval segments to produce complex spectra;
• and
• store the transformed complex spectra as audio features.

A method for forming recognition features from an input data stream, comprising the steps of:

• receiving the input data stream;
• forming a plurality of feature time series waveforms which respectively correspond to distinct portions of the received input data stream;
• forming multiple feature streams from the plurality of feature time series waveforms;
• forming overlapping time intervals of the multiple feature streams;
• estimating the distinctiveness of each feature in each time interval;
• rank-ordering the features according to their information content;
• transforming the feature time series waveforms to obtain complex spectra;
• and
• storing the feature complex spectra as the recognition features.

A method of using recognition features from an input data stream to achieve automatic signal identification, comprising the steps of:

• receiving the input data stream;
• forming a plurality of time series waveforms which correspond to all features of the received input data stream;
• forming multiple feature streams from the plurality of feature time series waveforms;
• correlating the most distinctive feature of plural stored candidate patterns with the multiple feature streams formed from the unknown input data stream in an order corresponding to a map of candidate patterns that best match the rank ordering of the plurality of feature time series waveforms;
• and
• designating recognition of the input data stream when a joint probability of multiple feature correlations between the input data stream and the stored candidate patterns indicates that random detection is not probable.