Back to results
US 11,232,787 B2Granted
Media composition with phonetic matching and waveform alignment
AssigneeAvid Technology, Inc.
InventorChristopher M. Winsor
Life of this patent
Estimating the term…
Abstract
When a portion of an original audio track is unsatisfactory, alternate portions are searched for using phonetic matching within phonetically indexed alternate files. The alternates may correspond to recordings captured in a different take from the original where timecode matching is unavailable. An editor may preview one or more candidate alternates and select a preferred one to be used to replace the original. A media editing application, such as a digital audio workstation, automatically aligns and matches the preferred alternative to the original using waveform matching and optional gain matching.
Claims16
A method of editing an audio composition using an audio editing application, the method comprising:
- receiving a first audio file that includes a recorded first take of at least one of spoken dialog and song, wherein the first audio file is indexed by a first phonetic index comprising an ordered list of phonemes, each of the phonemes being associated with a temporal offset within the first audio file at which the phoneme occurs in the at least one of spoken dialog and song;
- receiving a second audio file that includes a recorded second take of the at least one of spoken dialog and song, wherein the second audio file is indexed by a second phonetic index comprising an ordered list of phonemes, each of the phonemes being associated with a temporal offset within the second audio file at which the phoneme occurs in the at least one of spoken dialog and song;
- and
- in response to a user of the audio editing application specifying a portion of the first audio file that is to be replaced, the audio editing application automatically:
- using the first phonetic index to identify a phoneme sequence corresponding to the specified portion of the first audio file;
- searching the second phonetic index for an occurrence of the phoneme sequence in the second phonetic index;
- and
- in response to locating an occurrence of the phoneme sequence in the second phonetic index:
- identifying a matching portion of the second audio file corresponding to the located occurrence of the phoneme sequence;
- and
- in response to the user of the audio editing application selecting the matching portion of the second audio file:
- temporally aligning the matching portion of the second audio file with the specified portion of the first audio file by matching a waveform of the matching portion of the second audio file with a waveform of the specified portion of the first audio file;
- and
- replacing the specified portion of the first audio file with the temporally aligned matching portion of the second audio file.
11 more elements
A method of editing an audio composition using an audio editing application, the method comprising:
- receiving a first audio file that includes a recording of at least one of spoken dialog and song;
- receiving a second audio file that includes an alternate recording of the at least one of spoken dialog and song, wherein the second audio file is indexed by a phonetic index comprising an ordered list of phonemes, each of the phonemes being associated with a temporal offset within the second audio file at which the phoneme occurs in the at least one of spoken dialog and song;
- and
- in response to a user of the audio editing application specifying a text string representing speech contained within a portion of the first audio file that is to be replaced, the audio editing application automatically:
- converting the text string into a corresponding phoneme sequence;
- searching the phonetic index of the second audio file for an occurrence of the phoneme sequence;
- and
- in response to locating an occurrence of the phoneme sequence in the phonetic index:
- identifying a matching portion of the second audio file corresponding to the located occurrence of the phoneme sequence;
- and
- in response to the user of the audio editing application selecting the matching portion of the second audio file:
- enabling the user to align the matching portion of the second audio file with the portion of the first audio file that is to be replaced;
- and
- replacing the portion of the first audio file that is to be replaced with the temporally aligned matching portion of the second audio file.
11 more elements
A computer program product comprising:
- a non-transitory computer-readable medium with computer-readable instructions encoded thereon, wherein the computer-readable instructions, when processed by a processing device instruct the processing device to perform a method of editing an audio composition using an audio editing application, the method comprising:
- receiving a first audio file that includes a recorded first take of at least one of spoken dialog and song, wherein the first audio file is indexed by a first phonetic index comprising an ordered list of phonemes, each of the phonemes being associated with a temporal offset within the first audio file at which the phoneme occurs in the at least one of spoken dialog and song;
- receiving a second audio file that includes a recorded second take of the at least one of spoken dialog and song, wherein the second audio file is indexed by a second phonetic index comprising an ordered list of phonemes, each of the phonemes being associated with a temporal offset within the second audio file at which the phoneme occurs in the at least one of spoken dialog and song;
- and
- in response to a user of the audio editing application specifying a portion of the first audio file that is to be replaced, the audio editing application automatically:
- using the first phonetic index to identify a phoneme sequence corresponding to the specified portion of the first audio file;
- searching the second phonetic index for an occurrence of the phoneme sequence in the second phonetic index;
- and
- in response to locating an occurrence of the phoneme sequence in the second phonetic index:
- identifying a matching portion of the second audio file corresponding to the located occurrence of the phoneme sequence;
- and
- in response to the user of the audio editing application selecting the matching portion of the second audio file:
- temporally aligning the matching portion of the second audio file with the specified portion of the first audio file by matching a waveform of the matching portion of the second audio file with a waveform of the specified portion of the first audio file;
- and
- replacing the specified portion of the first audio file with the temporally aligned matching portion of the second audio file.
12 more elements
A method of editing an audio composition using an audio editing application, the method comprising:
- receiving a first audio file that includes a first recording of audio content captured by a first recorder, wherein the first audio file is indexed by a first phonetic index comprising an ordered list of phonemes, each of the phonemes being associated with a temporal offset within the first audio file at which the phoneme occurs in the audio content;
- receiving a second audio file that includes a second recording of the audio content captured by a second recorder simultaneously with the capture of the audio content by the first recording, wherein the second audio file is indexed by a second phonetic index comprising an ordered list of phonemes, each of the phonemes being associated with a temporal offset within the second audio file at which the phoneme occurs in the audio content;
- and
- in response to a user of the audio editing application specifying a portion of the first audio file that is to be replaced, the audio editing application automatically:
- using the first phonetic index to identify a phoneme sequence corresponding to the specified portion of the first audio file;
- searching the second phonetic index for an occurrence of the phoneme sequence in the second phonetic index;
- and
- in response to locating an occurrence of the phoneme sequence in the second phonetic index:
- identifying a matching portion of the second audio file corresponding to the located occurrence of the phoneme sequence;
- and
- in response to the user of the audio editing application selecting the matching portion of the second audio file:
- temporally aligning the matching portion of the second audio file with the specified portion of the first audio file by matching a waveform of the matching portion of the second audio file with a waveform of the specified portion of the first audio file;
- and
- replacing the specified portion of the first audio file with the temporally aligned matching portion of the second audio file.
11 more elements
Dependencies are inferred from the claim text.