# US 12,119,023 B2 · Audio onset detection method and apparatus > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,119,023 B2 - Document ID: US12119023B2 - Status: Granted - Kind: B2 - Application number: 17/434,628 - Filed: 2020-02-27 - Published: 2022-11-10 - Granted: 2024-10-15 - Earliest claimed: 2019-02-28 - Assignees: Beijing Bytedance Network Technology Co., Ltd. - Applicants: Beijing Bytedance Network Technology Co., Ltd. - Inventors: Wei Li - Web page: https://patentgrep.com/patent/US12119023B2 ## CPC - G10L 15/04: Segmentation; Word boundary detection - G10L 25/81: for discriminating voice from music - G10H 1/40: Rhythm - G10L 25/18: the extracted parameters being spectral information of each sub-band - G10L 25/51: for comparison or discrimination - G10L 25/87: Detection of discrete points within a voice signal - G10H 2210/051: for extraction or detection of onsets of musical sounds or notes, i.e. note attack timings - G10H 2210/071: for rhythm pattern analysis or rhythm style recognition - G10H 2250/455: Gensound singing voices, i.e. generation of human voices for musical applications, vocal singing sounds or intelligible words at a desired pitch or with desired vocal effects, e.g. by phoneme synthesis - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G10H: ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE ## Abstract An audio onset detection method and apparatus, an electronic device, and a computer readable storage medium. The audio onset detection method comprises: determining a first voice frequency spectrum parameter corresponding to each frequency band according to a frequency domain signal corresponding to an audio signal of an audio; for each frequency band, determining a second voice frequency spectrum parameter of a current frequency band according to the first voice frequency spectrum parameter of the current frequency band and the first voice frequency spectrum parameters of frequency bands positioned before the current frequency band according to a time sequence; and determining one or more onset positions of notes and syllables in the audio according to the second voice frequency spectrum parameters corresponding to the frequency bands. ## Claims ### Claim 1 An audio onset detection method, comprising: dividing an audio signal of an audio into a plurality of sub audio signals, and converting the plurality of sub audio signals respectively into frequency domain signals, wherein each of the plurality of sub audio signals corresponds to one of a plurality of frequency bands; determining a plurality of first speech spectrum parameters corresponding to the plurality of frequency bands, respectively; determining, for each of the plurality of frequency bands, a second speech spectrum parameter of a current frequency band based on a first speech spectrum parameter of the current frequency band and a first speech spectrum parameter of a frequency band chronologically before the current frequency band, wherein the determining, for each of the plurality of frequency bands, a second speech spectrum parameter of a current frequency band based on a first speech spectrum parameter of the current frequency band and a first speech spectrum parameter of a frequency band chronologically before the current frequency band further comprises: with respect to each of the plurality of frequency bands, determining a mean value of the first speech spectrum parameter of the current frequency band based on the first speech spectrum parameter of the current frequency band and the first speech spectrum parameter of the frequency band chronologically before the current frequency band, and determining the second speech spectrum parameter of the current frequency band based on the determined mean value; and determining one or more onsets of a note and a syllable in the audio based on second speech spectrum parameters of the frequency bands. ### Claim 2 (depends on 1) The audio onset detection method according to claim 1, further comprising: identifying the determined mean value as the second speech spectrum parameter of the current frequency band. ### Claim 3 (depends on 1) The audio onset detection method according to claim 1, further comprising: determining the second speech spectrum parameter of the current frequency band based on the first speech spectrum parameter of the current frequency band and the determined mean value. ### Claim 4 (depends on 3) The audio onset detection method according to claim 3, wherein the determining the second speech spectrum parameter of the current frequency band based on the first speech spectrum parameter of the current frequency band and the determined mean value comprises: calculating a difference between the first speech spectrum parameter of the current frequency band and the determined mean value; and determining the second speech spectrum parameter of the current frequency band based on the difference. ### Claim 5 (depends on 4) The audio onset detection method according to claim 4, wherein the determining the second speech spectrum parameter of the current frequency band based on the difference comprises: determining a mean value of the difference according to a difference corresponding to the current frequency band and a difference corresponding to the frequency band chronologically before the current frequency band; and identifying the determined mean value of the difference as the second speech spectrum parameter of the current frequency band. ### Claim 6 (depends on 1) The audio onset detection method according to claim 1, wherein the determining one or more onsets of a note and a syllable in the audio based on second speech spectrum parameters of the frequency bands comprises: drawing a speech spectrum parameter curve based on the second speech spectrum parameters of the frequency bands; and determining a local highest point according to the speech spectrum parameter curve, and determining one or more onsets of the note and the syllable in the audio based on a second speech spectrum parameter corresponding to the local highest point. ### Claim 7 An audio onset detection device, comprising: at least one processor; and at least one memory communicatively coupled to the at least one processor and storing instructions that upon execution by the at least one processor cause the at least one processor to perform operations comprising: dividing an audio signal of an audio into a plurality of sub audio signals, and converting the plurality of sub audio signals respectively into frequency domain signals, wherein each of the plurality of sub audio signals corresponds to one of a plurality of frequency bands; determining a plurality of first speech spectrum parameters corresponding to the plurality of frequency bands, respectively; determining, for each of the frequency bands, a second speech spectrum parameter of a current frequency band based on a first speech spectrum parameter of the current frequency band and a first speech spectrum parameter of a frequency band chronologically before the current frequency band, wherein the determining, for each of the plurality of frequency bands, a second speech spectrum parameter of a current frequency band based on a first speech spectrum parameter of the current frequency band and a first speech spectrum parameter of a frequency band chronologically before the current frequency band further comprises: with respect to each of the plurality of frequency bands, determining a mean value of the first speech spectrum parameter of the current frequency band based on the first speech spectrum parameter of the current frequency band and the first speech spectrum parameter of the frequency band chronologically before the current frequency band, and determining the second speech spectrum parameter of the current frequency band based on the determined mean value; and determining one or more onsets of a note and a syllable in the audio based on second speech spectrum parameters of the frequency bands. ### Claim 8 (depends on 7) The device of claim 7, wherein the operations further comprise: identifying the determined mean value as the second speech spectrum parameter of the current frequency band. ### Claim 9 (depends on 7) The device of claim 7, wherein the operations further comprise: determining the second speech spectrum parameter of the current frequency band based on the first speech spectrum parameter of the current frequency band and the determined mean value. ### Claim 10 (depends on 9) The device of claim 9, wherein the operations further comprise: calculating a difference between the first speech spectrum parameter of the current frequency band and the determined mean value; and determining the second speech spectrum parameter of the current frequency band based on the difference. ### Claim 11 (depends on 10) The device of claim 10, wherein the operations further comprise: determining a mean value of the difference according to a difference corresponding to the current frequency band and a difference corresponding to the frequency band chronologically before the current frequency band; and identifying the determined mean value of the difference as the second speech spectrum parameter of the current frequency band. ### Claim 12 (depends on 7) The device of claim 7, wherein the operations further comprise: drawing a speech spectrum parameter curve based on the second speech spectrum parameters of the frequency bands; and determining a local highest point according to the speech spectrum parameter curve; and determining one or more onsets of the note and the syllable in the audio based on a second speech spectrum parameter corresponding to the local highest point. ### Claim 13 A non-transitory computer readable storage medium, storing non-transitory computer readable instructions that, when executed by a computer, causes the computer to perform operations comprising: dividing an audio signal of an audio into a plurality of sub audio signals, and converting the plurality of sub audio signals respectively into frequency domain signals, wherein each of the plurality of sub audio signals corresponds to one of a plurality of frequency bands; determining a plurality of first speech spectrum parameters corresponding to the plurality of frequency bands, respectively; determining, for each of the frequency bands, a second speech spectrum parameter of a current frequency band based on a first speech spectrum parameter of the current frequency band and a first speech spectrum parameter of a frequency band chronologically before the current frequency band, wherein the determining, for each of the plurality of frequency bands, a second speech spectrum parameter of a current frequency band based on a first speech spectrum parameter of the current frequency band and a first speech spectrum parameter of a frequency band chronologically before the current frequency band further comprises: with respect to each of the plurality of frequency bands, determining a mean value of the first speech spectrum parameter of the current frequency band based on the first speech spectrum parameter of the current frequency band and the first speech spectrum parameter of the frequency band chronologically before the current frequency band, and determining the second speech spectrum parameter of the current frequency band based on the determined mean value; and determining one or more onsets of a note and a syllable in the audio based on second speech spectrum parameters of the frequency bands. ### Claim 14 (depends on 13) The non-transitory computer readable storage medium according to claim 13, wherein the operations further comprise: identifying the determined mean value as the second speech spectrum parameter of the current frequency band. ### Claim 15 (depends on 13) The non-transitory computer readable storage medium according to claim 13, wherein the operations further comprise: determining the second speech spectrum parameter of the current frequency band based on the first speech spectrum parameter of the current frequency band and the determined mean value. ### Claim 16 (depends on 15) The non-transitory computer readable storage medium according to claim 15, wherein the operations further comprise: calculating a difference between the first speech spectrum parameter of the current frequency band and the determined mean value; and determining the second speech spectrum parameter of the current frequency band based on the difference. ### Claim 17 (depends on 16) The non-transitory computer readable storage medium according to claim 16, wherein the operations further comprise: determining a mean value of the difference according to a difference corresponding to the current frequency band and a difference corresponding to the frequency band chronologically before the current frequency band; and identifying the determined mean value of the difference as the second speech spectrum parameter of the current frequency band. ### Claim 18 (depends on 13) The non-transitory computer readable storage medium according to claim 13, wherein the operations further comprise: drawing a speech spectrum parameter curve based on the second speech spectrum parameters of the frequency bands; and determining a local highest point according to the speech spectrum parameter curve; and determining one or more onsets of the note and the syllable in the audio based on a second speech spectrum parameter corresponding to the local highest point.