# US 2014/0207449 A1 · Using speech to text for detecting commercials and aligning edited episodes with transcripts > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0207449 A1 - Document ID: US20140207449A1 - Status: Published application - Kind: A1 - Application number: 13/744,585 - Filed: 2013-01-18 - Published: 2014-07-24 - Earliest claimed: 2013-01-18 - Assignees: Ramp Holdings, Inc. - Inventors: R Paul Johnson; Raymond Lau - Web page: https://patentgrep.com/patent/US20140207449A1 ## CPC - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G11B 27/00: Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel - G11B 27/034: on discs (G11B27/036, G11B27/038 take precedence) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract Methods and apparatus, including computer program products, for using speech to text for detecting commercials and aligning edited episodes with transcripts. A method includes, receiving an original video or audio having a transcript, receiving an edited video or audio of the original video or audio, applying a speech-to-text process to the received original video or audio having a transcript, applying a speech-to-text process to the received edited video or audio, and applying an alignment to determine locations of the edits. ## Claims ### Claim 1 A method comprising: in a computer system having a processor and a memory, receiving an original video or audio having a transcript; receiving an edited video or audio of the original video or audio; applying a speech-to-text process to the received original video or audio having a transcript; applying a speech-to-text process to the received edited video or audio; and applying an alignment to determine locations of the edits. ### Claim 2 (depends on 1) The method of claim 1 further comprising: aligning an output of the speech-to-text process to the received original video or audio having a transcript and the output of the speech-to-text process to the received edited video or audio; examining alignments for large sections that exist only in the original and not in the edited version; and for each pair of segments of the remaining segments aside from the commercials in the original, and the corresponding segments in the edited version, executing a closed-caption alignment process. ### Claim 3 (depends on 1) The method of claim 1 wherein the output of the speech-to-text process to the received original video or audio having a transcript and the output of the speech-to-text process to the received edited video or audio include timestamps. ### Claim 4 (depends on 1) The method of claim 1 wherein the transcript is derived from closed-captioning. ### Claim 5 (depends on 1) The method of claim 1 wherein the transcript is derived from metadata associated with the video or audio. ### Claim 6 (depends on 5) The method of claim 5 wherein the metadata comprises director's commentary. ### Claim 7 (depends on 5) The method of claim 5 wherein the metadata comprises scene descriptions. ### Claim 8 (depends on 1) The method of claim 1 wherein the original video or audio is selected from the group consisting of broadcast media, cable media and satellite media. ### Claim 9 (depends on 2) The method of claim 2 wherein aligning comprises performing a match on words with dynamic programming. ### Claim 10 (depends on 2) The method of claim 2 wherein aligning comprises performing a match on words minimizing an error metric. ### Claim 11 (depends on 10) The method of claim 10 wherein the error metric comprises a number of substitutions, insertions and deletions. ### Claim 12 (depends on 2) The method of claim 2 wherein examining comprises considering each position as a putative commercial boundary and looking at an accuracy within a window before/after the putative commercial boundary. ### Claim 13 (depends on 12) The method of claim 12 wherein an accuracy is derived from a combination of one or more of the number of correct, substitutions, insertions, and deletions. ### Claim 14 (depends on 2) The method of claim 2 wherein examining comprises using multiple window sizes. ### Claim 15 (depends on 2) The method of claim 2 wherein examining comprises using finite state machine logic. ### Claim 16 (depends on 2) The method of claim 2 wherein examining comprises using statistical classification or other machine learning. ### Claim 17 (depends on 2) The method of claim 2 wherein examining comprises imposing minimum or maximum length restrictions. ### Claim 18 (depends on 2) The method of claim 2 wherein examining comprises using confidence scores. ### Claim 19 (depends on 2) The method of claim 2 wherein examining comprises looking at time stamps to a left and a right. ### Claim 20 (depends on 2) The method of claim 2 further comprising performing closed-captioning alignment using the detected commercials to cut the original and edited media into segments to be aligned. ### Claim 21 (depends on 1) The method of claim 1 wherein applying a speech-to-text process to the received original video or audio having the transcript is replaced with audio fingerprinting the received original video or audio having the transcript. ### Claim 22 (depends on 1) The method of claim 1 wherein applying a speech-to-text process to the received edited video or audio is replaced with audio fingerprinting the received edited video or audio.