# US 2024/0267573 A1 · Livestream with synthetic scene insertion

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0267573 A1
- Document ID: US20240267573A1
- Status: Published application
- Kind: A1
- Application number: 18/430,720
- Filed: 2024-02-02
- Published: 2024-08-08
- Earliest claimed: 2023-02-03
- Assignees: Loop Now Technologies, Inc.
- Applicants: Loop Now Technologies, Inc.
- Inventors: Hong-Ming Tseng; Edwin Chiu; Wu-Hsi Li; Jerry Ting Kwan Luk; Jeremiah Kevin Tu
- Web page: https://patentgrep.com/patent/US20240267573A1

## CPC

- H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement}
- G06T 7/194: involving foreground-background segmentation
- G10L 15/1822: {Parsing for meaning understanding}
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- H04N 21/2187: Live feed
- H04N 21/44016: {involving splicing one content stream with another content stream, e.g. for substituting a video clip}
- H04N 21/812: {involving advertisement data (advertising per se G06Q30/02)}
- G10L 25/57: for processing of video signals
- G10L 2015/088: {Word spotting}
- H04N 21/2668: Creating a channel for a dedicated end-user group, e.g. insertion of targeted commercials based on end-user profiles {(information retrieval from the Internet by querying with filtering and personalisation G06F16/9535; arrangements for replacing or switching information during the broadcast H04H20/10; push services over packet-switching network H04L12/1859; adaptation of message content in packet-switching networks H04L51/063)}
- H04N 21/816: {involving special video data, e.g 3D video}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)

## Abstract

Disclosed embodiments provide techniques for livestream with synthetic scene insertion. A prerecorded livestream featuring a host is rendered to one or more viewers. An operator accesses a video segment related to the prerecorded livestream. The related video segment includes a performance by an individual. The operator retrieves an image of the host of the prerecorded livestream and creates from the related video segment a synthesized video segment that includes the performance of the individual accomplished by the host. One or more insertion points are determined within the prerecorded livestream for the insertion of the synthesized video segment. The operator inserts the synthesized video segment into the prerecorded livestream at the one or more determined insertion points. The insertion is accomplished dynamically and appears seamless to the viewer. The remainder of the prerecorded video is rendered to the one or more viewers after the insertion point.

## Claims

### Claim 1

A computer-implemented method for video analysis comprising:
rendering a prerecorded livestream, wherein the prerecorded livestream features a host and is viewed by one or more viewers;
accessing, by an operator, a video segment that is related to the prerecorded livestream, wherein the video segment includes a performance by an individual;
retrieving an image of the host;
creating, from the video segment that was accessed, a synthesized video segment, wherein the synthesized video segment includes the performance accomplished by the host;
determining at least one insertion point, within the prerecorded livestream, for the synthesized video segment;
inserting, by the operator, the synthesized video segment into the prerecorded livestream at the at least one insertion point, wherein the inserting is accomplished dynamically and wherein the inserting appears seamless to a viewer; and
rendering a remainder of the prerecorded livestream after the at least one insertion point.

### Claim 2 (depends on 1)

The method of claim 1 wherein the determining at least one insertion point further comprises forming a response to an interaction by the one or more viewers of the prerecorded livestream.

### Claim 3 (depends on 2)

The method of claim 2 wherein the inserting the synthesized video segment comprises the response to the interaction by the one or more viewers.

### Claim 4 (depends on 1)

The method of claim 1 wherein the determining at least one insertion point further comprises analyzing the prerecorded livestream.

### Claim 5 (depends on 4)

The method of claim 4 wherein the analyzing includes detecting one or more words spoken by the host.

### Claim 6 (depends on 4)

The method of claim 4 wherein the analyzing includes detecting one or more actions of the host.

### Claim 7 (depends on 4)

The method of claim 4 wherein the analyzing includes detecting one or more voice inflections of the host.

### Claim 8 (depends on 4)

The method of claim 4 wherein the analyzing includes detecting one or more subject matters discussed by the host.

### Claim 9 (depends on 1)

The method of claim 1 wherein the at least one insertion point is stored with metadata associated with the prerecorded livestream.

### Claim 10 (depends on 1)

The method of claim 1 wherein the accessing includes accessing a second video segment that is related to the prerecorded livestream, wherein the second video segment includes a second performance by the individual.

### Claim 11 (depends on 10)

The method of claim 10 wherein the second video segment includes a performance by a second individual.

### Claim 12 (depends on 10)

The method of claim 10 wherein the creating further comprises generating, from the second video segment, a second synthesized video segment, wherein the second synthesized video segment includes the second performance accomplished by the host.

### Claim 13 (depends on 12)

The method of claim 12 wherein the inserting includes the second synthesized video segment.

### Claim 14 (depends on 1)

The method of claim 1 wherein the determining at least one insertion point further comprises assessing a body position.

### Claim 15 (depends on 1)

The method of claim 1 wherein the inserting the synthesized video segment further comprises stitching the synthesized video segment into the prerecorded livestream at the at least one insertion point.

### Claim 16 (depends on 15)

The method of claim 15 wherein the stitching occurs at one or more boundary frames at the at least one insertion point between the synthesized video segment and the prerecorded livestream.

### Claim 17 (depends on 16)

The method of claim 16 wherein the stitching further comprises differentiating an object from a background.

### Claim 18 (depends on 17)

The method of claim 17 further comprising removing the object from the synthesized video segment or the prerecorded livestream.

### Claim 19 (depends on 16)

The method of claim 16 wherein the stitching further comprises restoring a corrupt video frame.

### Claim 20 (depends on 19)

The method of claim 19 wherein the restoring includes evaluating one or more video frames before and after the corrupt video frame.

### Claim 21 (depends on 16)

The method of claim 16 wherein the stitching further comprises deleting a frame of the prerecorded livestream.

### Claim 22 (depends on 16)

The method of claim 16 wherein the stitching further comprises assessing a body position.

### Claim 23 (depends on 1)

The method of claim 1 wherein the synthesized video segment includes synthesized audio.

### Claim 24 (depends on 1)

The method of claim 1 further comprising enabling an ecommerce purchase, within the prerecorded livestream, of a product for sale.

### Claim 25

A computer program product embodied in a non-transitory computer readable medium for video analysis, the computer program product comprising code which causes one or more processors to perform operations of:
rendering a prerecorded livestream, wherein the prerecorded livestream features a host and is viewed by one or more viewers;
accessing, by an operator, a video segment that is related to the prerecorded livestream, wherein the video segment includes a performance by an individual;
retrieving an image of the host;
creating, from the video segment that was accessed, a synthesized video segment, wherein the synthesized video segment includes the performance accomplished by the host;
determining at least one insertion point, within the prerecorded livestream, for the synthesized video segment;
inserting, by the operator, the synthesized video segment into the prerecorded livestream at the at least one insertion point, wherein the inserting is accomplished dynamically and wherein the inserting appears seamless to a viewer; and
rendering a remainder of the prerecorded livestream after the at least one insertion point.

### Claim 26

A computer system for video analysis comprising:
a memory which stores instructions;
one or more processors coupled to the memory wherein the one or more processors, when executing the instructions which are stored, are configured to:
render a prerecorded livestream, wherein the prerecorded livestream features a host and is viewed by one or more viewers;
access, by an operator, a video segment that is related to the prerecorded livestream, wherein the video segment includes a performance by an individual;
retrieve an image of the host;
create, from the video segment that was accessed, a synthesized video segment, wherein the synthesized video segment includes the performance accomplished by the host;
determine at least one insertion point, within the prerecorded livestream, for the synthesized video segment;
insert, by the operator, the synthesized video segment into the prerecorded livestream at the at least one insertion point, wherein inserting is accomplished dynamically and wherein the inserting appears seamless to a viewer; and
render a remainder of the prerecorded livestream after the at least one insertion point.
