Non-fourier spectral analysis for editing and visual display of music
OwnerDavid C. Chuinventor
Life of this patent
Estimating the term…
Abstract
System and method for identifying tones present in a short segment of digitized music stream, and for reporting simultaneously and quantitatively their respective magnitude and phase in near real time. Also captured are pitch deviations from the nominal tones of a predetermined music scale. The resulting spectral data can be scrolled manually from frame to frame to facilitate detail music evaluation and editing. The apparatus can also operate at real time to display notes being played, or to tone-activate audio-visual music enhancement and display with automatic synchronization.
Claims16
A system for computing quantitative estimates of magnitude, phase, and pitch deviation-from-nominal for each of one or more distinct nominal pitches of a predefined music scale vector in a digital audio frame vector having a plurality of discrete samples, the system including a computer processor configured to:
- acquire a wave matrix and an inverse cross-wave matrix,
- the wave matrix having a cosine wave vector for each distinct nominal pitch, the frequency of the cosine wave being the nominal pitch, and length of the cosine wave vector the number of discrete samples, a sine wave vector for each distinct nominal pitch, the frequency of the sine wave being the nominal pitch, and length of the sine wave vector being the number of discrete samples, such that the number of rows is twice the number of distinct nominal pitches, and the number of columns equal to the number of discrete samples,
- the inverse cross-wave matrix being the inverse of the matrix multiplication of the wave matrix and the transpose of the wave matrix;
- compute a keyboard transform vector,
- the keyboard transform vector being the combination of a first scalar (dot-product) multiplication and a second scalar (dot-product) multiplication to form the keyboard transform vector such that the number of elements in the keyboard transform vector is twice the number of distinct nominal pitches, the first scalar (dot-product) multiplication being a scalar (dot-product) multiplication of the digital audio frame vector by each cosine wave vector of the wave matrix, and the second scalar (dot-product) multiplication being a scalar (dot-product) multiplication of the digital audio frame vector by each sine wave vector of the wave matrix;
- perform a matrix multiplication of the inverse cross-wave matrix by the keyboard transform vector to form a complex spectral vector such that the number of elements in the complex spectral vector is twice the number of distinct nominal pitches;
- perform a standard rectangular-to-polar conversion of complex spectral vector for generating a magnitude spectral vector and a phase spectral vector, such that the number of elements in the magnitude spectral vector is the number of distinct nominal pitches, and the number of elements in the phase spectral vector is the number of distinct nominal pitches;
- perform a pitch deviation estimate on at least one nominal pitch with prominent magnitude, based on the difference between nominal phase progression between two consecutive audio frames and the actual difference between the phase estimates of the same two frames;
- record the estimates in a non-transitory computer readable medium;
- and
- display an audio-visual representation of at least one element from the magnitude spectral vector for the user.
A method for computing quantitative estimates of magnitude, phase, and pitch deviation-from-nominal for each of one or more distinct nominal pitches of a predefined music scale vector in a digital audio frame vector comprising a plurality of discrete samples, comprising the steps of:
- computing a wave matrix and an inverse cross-wave matrix,
- the wave matrix having a cosine wave vector for each distinct nominal pitch, whereby the frequency of the cosine wave is the nominal pitch, and length of the cosine wave vector is the number of discrete samples, a sine wave vector for each distinct nominal pitch, whereby the frequency of the sine wave is the nominal pitch, and length of the sine wave vector is the number of discrete samples, such that the number of rows is twice the number of distinct nominal pitches, and the number of columns equal to the number of discrete samples,
- the inverse cross-wave matrix being the inverse of the matrix multiplication of the wave matrix and the transpose of the wave matrix;
- computing a keyboard transform vector including
- performing a first scalar (dot-product) multiplication of the digital audio frame vector by each cosine wave vector of the wave matrix,
- performing a second scalar (dot-product) multiplication of the digital audio frame vector by each sine wave vector of the wave matrix,
- combining the first scalar (dot-product) multiplication and the second scalar (dot-product) multiplication to form the keyboard transform vector such that the number of elements in the keyboard transform vector is twice the number of distinct nominal frequencies;
- performing a matrix multiplication of the inverse cross-wave matrix by the keyboard transfix vector to form a complex spectral vector such that the number of elements in the complex spectral vector is twice the number of distinct nominal frequencies;
- performing a standard rectangular-to-polar conversion of complex spectral vector for generating a magnitude spectral vector and a phase spectral vector, such that the number of elements in the magnitude spectral vector is the number of distinct nominal pitches, and the number of elements in the phase spectral vector is the number of distinct nominal pitches;
- perform a pitch deviation estimate on at least one nominal pitch with prominent magnitude, based on the difference between nominal phase progression between two consecutive audio frames and the actual difference between the phase estimates of the same two frames;
- record the estimates in a non-transitory computer readable medium;
- and
- display an audio-visual representation of at least one element from the magnitude spectral vector for the user.
A system for computing quantitative estimates of magnitude, phase, and pitch deviation-from-nominal for each of one or more distinct nominal pitches of a predefined music scale vector in a digital audio frame vector a plurality of discrete samples, the system including a computer processor configured to:
- acquire a wave matrix and a square cross-wave matrix,
- the wave matrix having a cosine wave vector for each distinct nominal pitch, the frequency of the cosine wave being the nominal pitch, and length of the cosine wave vector being the number of discrete samples, a sine wave vector for each distinct nominal pitch, the frequency of the sine wave being the nominal pitch, and length of the sine wave vector is the number of discrete samples, such that the number of rows is twice the number of distinct nominal frequencies, and the number of columns equal to the number of discrete samples,
- the square cross-wave matrix being the matrix multiplication of the wave matrix and the transpose of the wave matrix;
- compute a keyboard transform vector,
- the keyboard transform vector being the combination of a first scalar (dot-product) multiplication and a second scalar (dot-product) multiplication to form the keyboard transform vector such that the number of elements in the keyboard transform vector is twice the number of distinct nominal pitches, the first scalar (dot-product) multiplication being a scalar (dot-product) multiplication of the digital audio frame vector by each cosine wave vector of the wave matrix, and the second scalar (dot-product) multiplication being a scalar (dot-product) multiplication of the digital audio frame vector by each sine wave vector of the wave matrix;
- compute a squared magnitude keyboard transform vector by summing the square of a first rectangular component and a second rectangular component for each of the distinct nominal frequencies;
- compute a decimated keyboard transform vector by selecting only elements from the complex keyboard transform vector with corresponding to d elements of the squared magnitude keyboard transform having the greatest magnitudes, where d is an integer between one and the number of distinct nominal frequencies, inclusive;
- compute a decimated cross-wave matrix by selecting only rows and columns from the square cross-wave matrix corresponding to the d elements of the squared magnitude keyboard transform vector selected in the previous step;
- perform a matrix inversion to the decimated cross-wave matrix to form an inverse decimated cross-wave matrix;
- perform a matrix multiplication of the inverse decimated cross-wave matrix by the decimated keyboard transform vector to form a decimated complex spectral vector such that the number of elements in the decimated complex spectral vector is twice d;
- perform a standard rectangular-to-polar conversion of the decimated complex spectral vector for generating a decimated magnitude spectral vector and a decimated phase spectral vector, such that the number of elements in the magnitude spectral vector is d, and the number of elements in the phase spectral vector is d;
- compute a complete magnitude spectral vector by placing elements of the magnitude of the decimated magnitude spectral vector in their respective tonal position and assign zero to all other tonal positions;
- compute a complete phase spectral vector by placing elements of the phase of the decimated phase spectral vector in their respective tonal position and assign zero to all other tonal positions;
- perform a pitch deviation estimate on at least one nominal pitch with prominent magnitude, based on the difference between nominal phase progression between two consecutive audio frames and the actual difference between the phase estimates of the same two frames;
- record the estimates in a non-transitory computer readable medium;
- and
- display an audio-visual representation of at least one element from the magnitude spectral vector for the user.
A method for computing quantitative estimates of magnitude, phase, and pitch deviation-from-nominal for each of one or more distinct nominal pitches of a predefined music scale vector in a digital audio frame vector a plurality of discrete samples, the system comprising a computer processor configured to:
- acquire a wave matrix and a square cross-wave matrix,
- the wave matrix having a cosine wave vector for each distinct nominal pitch, the frequency of the cosine wave being the nominal pitch, and length of the cosine wave vector being the number of discrete samples, a sine wave vector for each distinct nominal pitch, the frequency of the sine wave being the nominal pitch, and length of the sine wave vector is the number of discrete samples, such that the number of rows is twice the number of distinct nominal frequencies, and the number of columns equal to the number of discrete samples;
- the square cross-wave matrix being the matrix multiplication of the wave matrix and the transpose of the wave matrix;
- compute a keyboard transform vector,
- the keyboard transform vector being the combination of a first scalar (dot-product) multiplication and a second scalar (dot-product) multiplication to form the keyboard transform vector such that the number of elements in the keyboard transform vector is twice the number of distinct nominal pitches, the first scalar (dot-product) multiplication being a scalar (dot-product) multiplication of the digital audio frame vector by each cosine wave vector of the wave matrix, and the second scalar (dot-product) multiplication being a scalar (dot-product) multiplication of the digital audio frame vector by each sine wave vector of the wave matrix;
- compute a squared magnitude keyboard transform vector by summing the square of a first rectangular component and a second rectangular component for each of the distinct nominal frequencies;
- compute a decimated keyboard transform vector by selecting only elements from the complex keyboard transform vector with corresponding to d elements of the squared magnitude keyboard transform having the greatest magnitudes, where d is an integer between one and the number of distinct nominal frequencies, inclusive;
- compute a decimated cross-wave matrix by selecting only rows and columns from the square cross-wave matrix corresponding to the d elements of the squared magnitude keyboard transform vector selected in the previous step;
- perform a matrix inversion to the decimated cross-wave matrix to form an inverse decimated cross-wave matrix;
- perform a matrix multiplication of the inverse decimated cross-wave matrix by the decimated keyboard transform vector to form a decimated complex spectral vector such that the number of elements in the decimated complex spectral vector is twice d;
- perform a standard rectangular-to-polar conversion of the decimated complex spectral vector for generating a decimated magnitude spectral vector and a decimated phase spectral vector, such that the number of elements in the magnitude spectral vector is d, and the number of elements in the phase spectral vector is d;
- compute a complete magnitude spectral vector by placing elements of the magnitude of the decimated magnitude spectral vector in their respective tonal position and assign zero to all other tonal positions;
- compute a complete phase spectral vector by placing elements of the phase of the decimated phase spectral vector in their respective tonal position and assign zero to all other tonal positions;
- perform a pitch deviation estimate on at least one nominal pitch with prominent magnitude, based on the difference between nominal phase progression between two consecutive audio frames and the actual difference between the phase estimates of the same two frames;
- record the estimates in a non-transitory computer readable medium;
- and
- display an audio-visual representation of at least one element from the complete magnitude spectral vector for the user.
Dependencies are inferred from the claim text.