# US 2025/0157103 A1 · Media stream storyboard generation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0157103 A1
- Document ID: US20250157103A1
- Status: Published application
- Kind: A1
- Application number: 18/389,537
- Filed: 2023-11-14
- Published: 2025-05-15
- Earliest claimed: 2023-11-14
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Kori Marie INKPEN; Nicolai MARQUARDT; John C. TANG; Asta Jane ROSEWAY; Erzhen HU; Sasa JUNUZOVIC; Nathalie M. RICHE
- Web page: https://patentgrep.com/patent/US20250157103A1

## CPC

- G06F 16/739: {in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames}
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 40/134: Hyperlinking
- G06F 40/166: Editing, e.g. inserting or deleting
- G06F 40/30: Semantic analysis
- G06F 40/35: Discourse or dialogue representation
- G06T 11/60: Creating or editing images; Combining images with text
- G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G06N 3/0475: Generative networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for generating storyboards is described. An extraction prompt is provided to a first generative neural network model. The extraction prompt is a text-based prompt that instructs the first generative neural network model how to identify timestamps of segments having related content within transcripts according to dialog within the transcripts. A transcript of a meeting is provided as an input to the first generative neural network model. Segment timestamps for identified segments within the meeting are received from the first generative neural network model based on the extraction prompt and the transcript. Segment images for the identified segments are generated using a second generative neural network model, wherein each of the segment images represents segment content within a corresponding identified segment.

## Claims

### Claim 1

A method for generating storyboards, comprising:
providing an extraction prompt to a first generative neural network model, wherein the extraction prompt is a text-based prompt that instructs the first generative neural network model how to identify timestamps of segments having related content within transcripts according to dialog within the transcripts:
providing a transcript of a meeting as an input to the first generative neural network model:
receiving, from the first generative neural network model, segment timestamps for identified segments within the meeting based on the extraction prompt and the transcript; and
generating segment images for the identified segments using a second generative neural network model, wherein each of the segment images represents segment content within a corresponding identified segment.

### Claim 2 (depends on 1)

The method of claim 1, wherein the extraction prompt further instructs the first generative neural network model how to identify segment labels according to content within identified segments, the method further comprising:
receiving, from the first generative neural network model, segment labels for the identified segments based on the extraction prompt and the transcript; and
labeling the segment images with a corresponding segment label.

### Claim 3 (depends on 1)

The method of claim 1, wherein generating the segment images comprises generating, within a segment image for an identified segment, an avatar for a source user of dialog within the identified segment.

### Claim 4 (depends on 3)

The method of claim 3, further comprising augmenting the segment image for the identified segment with text from the dialog within the identified segment.

### Claim 5 (depends on 4)

The method of claim 4, wherein the text from the dialog within the identified segment is depicted within a dialog bubble for the avatar.

### Claim 6 (depends on 5)

The method of claim 5, wherein the avatar is a captured image portion from a media stream of the meeting.

### Claim 7 (depends on 5)

The method of claim 5, wherein the avatar is generated based on a likeness of the source user.

### Claim 8 (depends on 4)

The method of claim 4, further comprising:
generating a link for playback of a media stream of the meeting at a timestamp corresponding to the text from the dialog within the identified segment; and
augmenting the segment image to include the link.

### Claim 9 (depends on 1)

The method of claim 1, wherein the second generative neural network model is trained to generate images from text.

### Claim 10 (depends on 1)

The method of claim 1, wherein the first generative neural network model is a generative large language model.

### Claim 11 (depends on 1)

The method of claim 1, wherein the transcript of the meeting is extracted from a media stream of the meeting that comprises audio and video.

### Claim 12

A system for generating storyboards, the system comprising:
at least one processor, and
at least one memory storing computer-executable instructions that when executed by the at least one processor cause the at least one processor to:
provide an extraction prompt to a first generative neural network model, wherein the extraction prompt is a text-based prompt that instructs the first generative neural network model how to identify timestamps of segments having related content within transcripts according to dialog within the transcripts;
provide a transcript of a meeting as an input to the first generative neural network model;
receive, from the first generative neural network model, segment timestamps for identified segments within the meeting based on the extraction prompt and the transcript;
generate segment images for the identified segments using a second generative neural network model, wherein each of the segment images represents segment content within a corresponding identified segment.

### Claim 13 (depends on 12)

The system of claim 12, wherein the extraction prompt further instructs the first generative neural network model how to identify segment labels according to content within the identified segments, wherein the computer-executable instructions cause the at least one processor to:
receive, from the first generative neural network model, segment labels for the identified segments based on the extraction prompt and the transcript; and
label the segment images with a corresponding segment label.

### Claim 14 (depends on 12)

The system of claim 12, wherein the computer-executable instructions cause the at least one processor to generate, within a segment image for an identified segment, an avatar for a source user of dialog within the identified segment.

### Claim 15 (depends on 14)

The system of claim 14, wherein the computer-executable instructions cause the at least one processor to augment the segment image for the identified segment with text from the dialog within the identified segment so that the text from the dialog within the identified segment is depicted within a dialog bubble for the avatar.

### Claim 16 (depends on 14)

The system of claim 14, wherein the avatar is a captured image portion from a media stream of the meeting.

### Claim 17

A method for generating storyboards, comprising:
identifying one or more segments within a media stream according to content within the one or more segments, including providing an extraction prompt and a transcript of the media stream to a large language model, wherein the content comprises dialog from at least one user and the extraction prompt is a text-based prompt that instructs the large language model how to identify the one or more segments;
generating segment labels for the one or more segments according to the content within the one or more segments using the large language model; and
generating segment images, for the one or more segments, for a storyboard of the media stream, wherein each of the segment images represents segment content within a corresponding segment and sources of the segment content.

### Claim 18 (depends on 17)

The method of claim 17, wherein generating the segment images comprises extracting images of a source user of the at least one user from the media stream.

### Claim 19 (depends on 17)

The method of claim 17, wherein generating the segment images comprises generating an avatar of a source user of the at least one user using a generative neural network model.

### Claim 20 (depends on 17)

The method of claim 17, further comprising augmenting a segment image with text from dialog within the corresponding segment.
