# US 2024/0126994 A1 · Transcript paragraph segmentation and visualization of transcript paragraphs

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0126994 A1
- Document ID: US20240126994A1
- Status: Published application
- Kind: A1
- Application number: 17/967,562
- Filed: 2022-10-17
- Published: 2024-04-18
- Earliest claimed: 2022-10-17
- Assignees: Adobe Inc.
- Applicants: Adobe Inc.
- Inventors: Hanieh DEILAMSALEHY; Aseem Omprakash AGARWALA; Haoran CAI; Hijung SHIN; Joel Richard BRANDT; Lubomira Assenova DONTCHEVA
- Web page: https://patentgrep.com/patent/US20240126994A1

## CPC

- G06F 40/30: Semantic analysis
- G06F 40/205: Parsing
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- H04N 5/9305: {involving the mixing of the reproduced video signal with a non-recorded signal, e.g. a text signal}
- G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes}
- G10L 15/04: Segmentation; Word boundary detection
- G10L 25/78: Detection of presence or absence of voice signals (switching of direction of transmission by voice frequency in two-way loud-speaking telephone systems H04M9/10)
- G10L 25/87: Detection of discrete points within a voice signal
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

Embodiments of the present invention provide systems, methods, and computer storage media for segmenting a transcript into paragraphs. In an example embodiment, a transcript is segmented to start a new paragraph whenever there is a change in speaker and/or a long pause in speech. If any remaining paragraphs are longer than a designated length or duration (e.g., 50 or 100 words), each of those paragraphs is segmented using dynamic programming to minimize a cost function that penalizes candidate paragraphs based on divergence from a target paragraph length and/or that rewards candidate paragraphs that group semantically similar sentences. As such, the transcript is visualized, segmented at the identified paragraphs.

## Claims

### Claim 1

One or more computer storage media storing computer-useable instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations comprising:
causing generation of a representation of a paragraph segmentation of a transcript based on paragraph length and semantic coherency of text within one or more paragraphs of the paragraph segmentation; and
causing a user interface to present a visualization of the paragraph segmentation of the transcript.

### Claim 2 (depends on 1)

The one or more computer storage media of claim 1, wherein the transcript is of an audio track, and the generation of the representation of the paragraph segmentation is based on speech pauses detected in the audio track using an audio classifier.

### Claim 3 (depends on 1)

The one or more computer storage media of claim 1, wherein the generation of the representation of the paragraph segmentation is based on speech pauses detected using differences in word or sentence timings extracted from the transcript.

### Claim 4 (depends on 1)

The one or more computer storage media of claim 1, wherein the generation of the representation of the paragraph segmentation comprises identifying paragraphs that are longer than a designated length or duration and, for each identified paragraph, segmenting the identified paragraph using dynamic programming to minimize a cost function.

### Claim 5 (depends on 1)

The one or more computer storage media of claim 1, wherein the generation of the representation of the paragraph segmentation comprises using dynamic programming to minimize a cost function that penalizes candidate segmentations that include candidate paragraphs with detected speech pauses longer than a target length or duration.

### Claim 6 (depends on 1)

The one or more computer storage media of claim 1, wherein the generation of the representation of the paragraph segmentation comprises using dynamic programming to minimize a cost function that rewards candidate paragraphs that group semantically similar sentences.

### Claim 7 (depends on 1)

The one or more computer storage media of claim 1, wherein the generation of the representation of the paragraph segmentation comprises quantifying the semantic coherency by generating a measure of similarity of each pair of sentences in a candidate paragraph and combining the measure of similarity for each pair to generate a measure of paragraph similarity for the candidate paragraph.

### Claim 8 (depends on 1)

The one or more computer storage media of claim 1, wherein the paragraph segmentation segments the transcript at a subset of sentence boundaries identified from the transcript.

### Claim 9 (depends on 1)

The one or more computer storage media of claim 1, wherein the transcript is of an audio track of a video, the user interface is a transcript interface of a video editing application, and the transcript interface is configured to interpret input selecting transcript text, via an interaction with the visualization of the paragraph segmentation of the transcript, as an instruction to select a corresponding video segment of the video.

### Claim 10

A method comprising:
triggering generation of a representation of a paragraph segmentation of a transcript based semantic coherency; and
causing a user interface to present transcript text of the transcript using the paragraph segmentation.

### Claim 11 (depends on 10)

The method of claim 10, wherein the transcript is of an audio track, and the generation of the representation of the paragraph segmentation is based on speech pauses detected in the audio track using an audio classifier.

### Claim 12 (depends on 10)

The method of claim 10, wherein the generation of the representation of the paragraph segmentation is based on speech pauses detected using differences in word or sentence timings extracted from the transcript.

### Claim 13 (depends on 10)

The method of claim 10, wherein the generation of the representation of the paragraph segmentation comprises identifying paragraphs that are longer than a designated length or duration and, for each identified paragraph, segmenting the identified paragraph using dynamic programming to minimize a cost function.

### Claim 14 (depends on 10)

The method of claim 10, wherein the generation of the representation of the paragraph segmentation comprises using dynamic programming to minimize a cost function that penalizes candidate segmentations that include candidate paragraphs with detected speech pauses longer than a designated length or duration.

### Claim 15 (depends on 10)

The method of claim 10, wherein the generation of the representation of the paragraph segmentation comprises using dynamic programming to minimize a cost function that rewards candidate paragraphs that group semantically similar sentences.

### Claim 16 (depends on 10)

The method of claim 10, wherein the generation of the representation of the paragraph segmentation comprises quantifying the semantic coherency by generating a measure of similarity of each pair of sentences in a candidate paragraph and combining the measure of similarity for each pair to generate a measure of paragraph similarity for the candidate paragraph.

### Claim 17 (depends on 10)

The method of claim 10, wherein the paragraph segmentation segments the transcript at a subset of sentence boundaries identified from the transcript.

### Claim 18 (depends on 10)

The method of claim 10, wherein the transcript is of an audio track of a video, the user interface is a transcript interface of a video editing application, and the transcript interface is configured to interpret input selecting a portion of the transcript text as an instruction to select a corresponding video segment of the video.

### Claim 19

A computer system comprising one or more processors and memory configured to provide computer program instructions to the one or more processors, the computer program instructions comprising:
a video interaction engine configured to trigger generation of a representation of a paragraph segmentation of a transcript of a video based on paragraph length and semantic coherency; and
a transcript tool configured to cause presentation of transcript text of the transcript using the paragraph segmentation.

### Claim 20 (depends on 19)

The computer system of claim 19, wherein the video interaction engine and the transcript tool are part of a video editing application, and the transcript tool is configured to interpret input selecting a portion of the transcript text as an instruction to select a corresponding video segment of the video.
