Text synchronization with audio
AssigneeAmazon Technologies, Inc.
InventorsBrandon Scott Durham, Darren Levi Malek, Toby Ray Latin-Stoermer, Abhishek Mishra, Jason Christopher Hall
Life of this patent
Estimating the term…
Abstract
A technology for synchronizing text with audio includes analyzing the audio to identify voice segments in the audio where a human voice is present and to identify non-voice segments in proximity to the voice segments. Segmented text associated with the audio, having text segments, may be identified and synchronized to the voice segments.
Claims18
A computing device that is configured to synchronize lyrics with music, comprising:
- a processor;
- a memory in electronic communication with the processor;
- instructions stored in the memory, the instructions being executable by the processor to:
- identify a marker for singing segments in the music where a person is singing using a machine learning model;
- identify a marker for break segments in proximity to the singing segments where the person is not singing using the machine learning model;
- identify lyric segments in lyrics associated with the music, the lyric segments being divided by lyric breaks;
- synchronize one of the lyric breaks with a marker of one of the break segments;
- and
- synchronize at least one of the lyric segments to a marker of one of the singing segments.
A computer-implemented method, comprising:
- analyzing audio, using a processor, to extract features from the audio and identify voice segments in the audio where a human voice is present and to identify non-voice segments in proximity to the voice segments based on the extracted features;
- identifying segmented text associated with the audio, the segmented text having text segments;
- synchronizing the text segments to the voice segments using the processor;
- and
- soliciting group-sourced corrections to correct the synchronizing of the text segments to the voice segments.
A non-transitory computer-readable medium comprising computer-executable instructions which, when executed by a processor, implement a system, comprising:
- an audio analysis module configured to analyze audio to identify a voice segment in the audio where a human voice is present;
- a text analysis module configured to identify segments in text associated with the audio and identify the voice segment as trained using other audio;
- a correlation module configured to determine a number of the segments of the text to associate with the voice segment;
- and
- a synchronization module to associate the number of the segments of the text with the voice segment.
Dependencies are inferred from the claim text.