# US 2025/0097544 A1 · Generating expanded content in a video stream based on predicting viewer behavior

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0097544 A1
- Document ID: US20250097544A1
- Status: Published application
- Kind: A1
- Application number: 18/468,321
- Filed: 2023-09-15
- Published: 2025-03-20
- Earliest claimed: 2023-09-15
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Sarbajit K. Rakshit; Sudhanshu Sekher Sar; Satyam Jakkula; Sudheesh S. Kairali
- Web page: https://patentgrep.com/patent/US20250097544A1

## CPC

- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- G06Q 30/0255: {based on user history}
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/44218: {Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV programme (methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45)}
- H04N 21/454: Content {or additional data} filtering, e.g. blocking advertisements
- H04N 21/4665: {involving classification methods, e.g. Decision trees}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR

## Abstract

Provided are a computer program product, system, and method for generating expanded content in a video stream based on predicting viewer behavior. A determination is made of promoted content rendered in the video. In response to determining the promoted content, invoking at least one machine learning model to perform determining whether a viewer of the video is interested in the promoted content and generating expanded content for a duration in response to determining that the viewer of the video is interested in the promoted content. The rendering of the video is paused and the expanded content is rendered for the duration while pausing the rendering of the video. The rendering of the video is continued in response to completing the rendering of the expanded content.

## Claims

### Claim 1

A computer program product for expanding media content in a video during rendering of the video, the computer program product comprising a computer readable storage medium having computer readable program code embodied therein that is executable to perform operations, the operations comprising:
determining promoted content rendered in the video;
in response to determining the promoted content, invoking at least one machine learning model to perform:
determining whether a viewer of the video is interested in the promoted content; and
generating expanded content for a duration in response to determining that the viewer of the video is interested in the promoted content;
pausing the rendering of the video;
rendering the expanded content for the duration while pausing the rendering of the video; and
continuing the rendering of the video in response to completing the rendering of the expanded content.

### Claim 2 (depends on 1)

The computer program product of claim 1, wherein the at least one machine learning model further determines the duration to render expanded content in response to determining that the viewer of the video is interested in the promoted content.

### Claim 3 (depends on 1)

The computer program product of claim 1, wherein the operations further comprise:
determining whether the duration exceeds a remaining expanded content time, wherein the generating the expanded content is performed in response to determining that the duration is less than the remaining expanded content time,
wherein the duration is set to the remaining expanded content time in response to the remaining expanded content time is greater than a minimum content time and less than the duration; and
decrementing the remaining expanded content time by the duration in response to determining that the duration is less than the remaining expanded content time.

### Claim 4 (depends on 1)

The computer program product of claim 1, wherein the determining whether the viewer of the video is interested in the promoted content comprises:
inputting, into a machine learning model classifier, information on at least a plurality of: the promoted content; eye movement tracking of at least one viewer indicating whether the at least one viewer was gazing on the promoted content; interaction history of at least one viewer with respect to the promoted content when viewing video; and interaction history of the at least one viewer with respect to the promoted content and content similar to the promoted content, wherein output from the machine learning model classifier indicates whether the at least one viewer observing the promoted content is likely interested in further interacting with the promoted content.

### Claim 5 (depends on 4)

The computer program product of claim 4, wherein the promoted content represents a product or service, wherein the interaction history of the at least one viewer with respect to the promoted content comprises a purchase history with respect to the promoted content and browsing history with respect to the promoted content, wherein the operations further comprise:
accessing, from an account of the at least one viewer at an e-commerce web site, the purchase history and the browsing history with respect to the promoted content to provide to the machine learning model classifier.

### Claim 6 (depends on 1)

The computer program product of claim 1, wherein the operations further comprise:
inputting, into a first machine learning model classifier, information on at least one viewer observing the video including the promoted content and the promoted content to output indication whether the at least one viewer of the video is likely interested in further interaction with the promoted content; and
inputting into a second machine learning model information on the at least one viewer and the promoted content to output the duration to render the expanded content in response to the first machine learning model classifier outputting indication that the at least one viewer of the video is likely interested in further interaction with the promoted content.

### Claim 7 (depends on 1)

The computer program product of claim 1, wherein the generating the expanded content comprises:
processing information on a context of the video including the promoted content to generate expanded content in the context of video.

### Claim 8 (depends on 1)

The computer program product of claim 1, wherein the generating the expanded content comprises:
receiving, by a generative adversarial neural network, input comprising frames of the video including a scene in which the promoted content appears, and wherein the expanded content comprises a continuance of the scene in the video for the duration.

### Claim 9 (depends on 8)

The computer program product of claim 8, wherein the input, received by the generative adversarial neural network, further includes information on supplement promoted content to include in the expanded content, wherein the expanded content includes rendering of the supplemental promoted content.

### Claim 10 (depends on 1)

The computer program product of claim 1, wherein the generating the expanded content comprises:
receiving, by a first generative adversarial network, input comprising a script of the content of the video including the promoted content to generate expanded content script for the duration; and
inputting, to a second generative adversarial network, the expanded content script to generate expanded content comprising video to render while the video is paused.

### Claim 11 (depends on 1)

The computer program product of claim 1, wherein the operations further comprise:
determining that the viewer has completed interacting with the promoted content before the expanded content has completed rendering for the duration; and
ending the pausing to continue rendering the video from where paused in response to determining that the viewer has completed interacting with the promoted content.

### Claim 12

A system program product for expanding media content in a video during rendering of the video, comprising:
a processor; and
a computer readable storage medium having computer readable program code embodied therein that when executed by the processor performs operations, the operations comprising:
determining promoted content rendered in the video;
in response to determining the promoted content, invoking at least one machine learning model to perform:
determining whether a viewer of the video is interested in the promoted content; and
generating expanded content for a duration in response to determining that the viewer of the video is interested in the promoted content;
pausing the rendering of the video;
rendering the expanded content for the duration while pausing the rendering of the video; and
continuing the rendering of the video in response to completing the rendering of the expanded content.

### Claim 13 (depends on 12)

The system of claim 12, wherein the determining whether the viewer of the video is interested in the promoted content comprises:
inputting, into a machine learning model classifier, information on at least a plurality of: the promoted content; eye movement tracking of at least one viewer indicating whether the at least one viewer was gazing on the promoted content; interaction history of at least one viewer with respect to the promoted content when viewing video; and interaction history of the at least one viewer with respect to the promoted content and content similar to the promoted content, wherein output from the machine learning model classifier indicates whether the at least one viewer observing the promoted content is likely interested in further interacting with the promoted content.

### Claim 14 (depends on 12)

The system of claim 12, wherein the operations performed in response to executing the computer readable program code further comprise:
inputting, into a first machine learning model classifier, information on at least one viewer observing the video including the promoted content and the promoted content to output indication whether the at least one viewer of the video is likely interested in further interaction with the promoted content; and
inputting into a second machine learning model information on the at least one viewer and the promoted content to output the duration to render the expanded content in response to the first machine learning model classifier outputting indication that the at least one viewer of the video is likely interested in further interaction with the promoted content.

### Claim 15 (depends on 12)

The system of claim 12, wherein the generating the expanded content comprises:
processing information on a context of the video including the promoted content to generate expanded content in the context of video.

### Claim 16 (depends on 12)

The system of claim 12, wherein the generating the expanded content comprises:
receiving, by a generative adversarial neural network, input comprising frames of the video including a scene in which the promoted content appears, and wherein the expanded content comprises a continuance of the scene in the video for the duration.

### Claim 17

A method for expanding media content in a video during rendering of the video, comprising:
determining promoted content rendered in the video;
in response to determining the promoted content, invoking at least one machine learning model to perform:
determining whether a viewer of the video is interested in the promoted content; and
generating expanded content for a duration in response to determining that the viewer of the video is interested in the promoted content;
pausing the rendering of the video;
rendering the expanded content for the duration while pausing the rendering of the video; and
continuing the rendering of the video in response to completing the rendering of the expanded content.

### Claim 18 (depends on 17)

The method of claim 17, wherein the determining whether the viewer of the video is interested in the promoted content comprises:
inputting, into a machine learning model classifier, information on at least a plurality of: the promoted content; eye movement tracking of at least one viewer indicating whether the at least one viewer was gazing on the promoted content; interaction history of at least one viewer with respect to the promoted content when viewing video; and interaction history of the at least one viewer with respect to the promoted content and content similar to the promoted content, wherein output from the machine learning model classifier indicates whether the at least one viewer observing the promoted content is likely interested in further interacting with the promoted content.

### Claim 19 (depends on 17)

The method of claim 17, wherein the operations further comprise:
inputting, into a first machine learning model classifier, information on at least one viewer observing the video including the promoted content and the promoted content to output indication whether the at least one viewer of the video is likely interested in further interaction with the promoted content; and
inputting into a second machine learning model information on the at least one viewer and the promoted content to output the duration to render the expanded content in response to the first machine learning model classifier outputting indication that the at least one viewer of the video is likely interested in further interaction with the promoted content.

### Claim 20 (depends on 17)

The method of claim 17, wherein the generating the expanded content comprises:
processing information on a context of the video including the promoted content to generate expanded content in the context of video.