# US 2022/0068315 A1 · System and methods for concatenating video sequences using face detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0068315 A1
- Document ID: US20220068315A1
- Status: Published application
- Kind: A1
- Application number: 17/523,368
- Filed: 2021-11-10
- Published: 2022-03-03
- Earliest claimed: 2017-02-16
- Assignees: Fusit, Inc.
- Applicants: Fusit, Inc.
- Inventors: Michal Shafir Nir; Liat Sade-Sternberg; Rhona Horiner Rosen; Tamar Raviv
- Web page: https://patentgrep.com/patent/US20220068315A1

## CPC

- G11B 27/036: Insert-editing
- G06V 20/49: {Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes}
- G06V 40/161: {Detection; Localisation; Normalisation}
- G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals
- G11B 27/28: by using information signals recorded by the same method as the main recording {(G11B27/22 takes precedence)}
- H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/23424: {involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement}
- H04N 21/234363: {by altering the spatial resolution, e.g. for clients with a lower screen resolution}
- H04N 21/234381: {by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping}
- H04N 21/2365: Multiplexing of several video streams
- H04N 21/2665: Gathering content from different sources, e.g. Internet and satellite
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/440263: {by altering the spatial resolution, e.g. for displaying on a connected PDA}
- H04N 21/440281: {by altering the temporal resolution, e.g. by frame skipping (television signal recording using magnetic recording on tape for reproducing at a rate different from the recording rate H04N5/783)}
- H04N 21/47205: {for manipulating displayed content, e.g. interacting with MPEG-4 objects, editing locally}
- H04N 21/4788: communicating with other users, e.g. chatting {(arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104)}
- H04N 21/854: Content authoring
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

There are provided methods and devices for media processing, comprising: providing at least one media asset source selected from a media asset sources library, the at least one media asset source comprising at least one source video, via a network or client device; receiving via the network or the client device a media recording comprising a client video recorded by a user of the client device; parsing the client video and the source video, respectively, to a plurality of client video frames and a plurality of source video frames; identifying at least one face in at least one frame of the plurality of source video frames and at least another face in at least one frame of the plurality of client video frames by face detection; superposing one or more markers on the identified at least one face of the plurality of source video frames; processing said client video frames to fit the size or shape of said source video frames by using said one or more markers; concatenating said processed client video frames with said source video frames, wherein said concatenation comprises matching the frame rate and resolution of the processed client video frames to the frame rate and resolution of the plurality of client video frames to yield a mixed media asset.

## Claims

### Claim 1

A method for media processing, comprising:
providing at least one media asset source selected from a media asset sources library, the at least one media asset source comprising at least one source video, via a network or client device, wherein the at least one source video comprises a plurality of source video frames;
receiving via the network or the client device a media recording comprising a client video, wherein the client video comprises a plurality of client video frames and wherein the frame rate or bit rate of the at least one source video is different from the frame rate or bit rate of the client video;
parsing the client video and the source video, respectively, to the plurality of client video frames and the plurality of source video frames;
identifying at least one face image in at least one frame of the plurality of source video frames and at least other face image in at least one frame of the plurality of client video frames using a face detection method;
superposing one or more markers on the identified at least one face image in the at least one frame of the plurality of source video frames;
processing said client video frames and said source video frames using an editing module and an analyzer module to fit the size or shape of elements in said client video frames to the size or shape of elements in said source video frames;
mixing said processed client video frames with said source video frames, using a mixing module, to yield a mixed media asset, wherein said mixing comprising:
filtering the new mixed media asset using a filter generator module, wherein said filtering comprises matching the frame rate or the bit rate of the processed client video frames to the frame rate or bit rate of the plurality of the source video frames; and
grouping the processed client video frames of the plurality of client video frames to yield a mixed and coherent media asset.

### Claim 2 (depends on 1)

The method of claim 1, wherein said processing comprises:
extracting said one or more markers from said at least identified one face image and superposing the extracted one or more markers on the other at least one identified face; and
resizing the other at least one identified face to match the size or shape of the at least one identified face in the at least one frame of the plurality of source video frames.

### Claim 3 (depends on 2)

The method of claim 2, wherein said processing further comprises cropping or scaling the at least one other identified face image in the at least one frame of the plurality of client video frames to match the size or shape of said identified at least one face image in the source video frames.

### Claim 4 (depends on 1)

The method of claim 1, comprising filtering the new mixed media asset using a filter generator module, wherein said filtering comprises one or more of the following filtering procedures:
fast forward filtering; slow motion filtering; color filtering; black and white filtering.

### Claim 5 (depends on 1)

The method of claim 1, wherein said face detection method comprise identifying the position of one or more face elements in at least one frame of the plurality of video source frames and other one or more face elements in the at least one client video frames.

### Claim 6 (depends on 1)

The method of claim 1, wherein said media asset source is in a form selected from the group consisting of:
graphics interchange format (GIF); or MP4; VP8; m4v; mov; avi; fav; mpg; wmv; h265, and the client video form is in a different form.

### Claim 7 (depends on 1)

The method of claim 1, wherein said face detection are selected from a face detection algorithms group consisting of:
SMQT Features and SNOW Classifier Method (SFSC),
Efficient and Rank Deficient Face Detection Method (ERDFD),
Gabor-Feature Extraction and Neural Network Method (GFENN),
an efficient face candidates selector Features Method (EFCSF).

### Claim 8 (depends on 1)

The method of claim 1, comprising analyzing and editing simultaneously said client video frames and said source video frames using the editing module and the analyzer module to fit the size or shape of elements in said client video frames to the size or shape of elements in said source video frames, by analyzing successively each of the frames of said client video frames and said source video frames to identify one or more faces at said client video frames and said source video frames, and editing the identified face at each frame of said client video frames according to the location of the identified face in the at least one frame of the plurality of source video frames.

### Claim 9 (depends on 1)

The method of claim 1, wherein the resolution of the at least one source video is different from the resolution of the client video.

### Claim 10

An apparatus for media processing, comprising:
a memory which is configured to hold one or more source media videos, wherein each of the one or more source media videos comprises a plurality of source video frames; and
a processor which is configured to:
transmit the one or more source media videos to a client device;
receive, via the network or the client device, a media recording comprising a client video recorded by a user of the client device, wherein the client video comprises a plurality of client video frames and wherein the frame rate or bit rate of the one or more source media videos is different from the frame rate or bit rate of the client video;
parse the client video and the source video, respectively, to the plurality of client video frames and the plurality of source video frames;
identify at least one face image in at least one frame of the plurality of source video frames and at least other face in at least one frame of the plurality of client video frames using face detection method;
superpose one or more markers on the identified at least one face image in the at least one frame of the plurality of source video frames;
process said client video frames and said source video frames using an editing module and an analyzer module to fit the size or shape of elements in said client video frames to the size or shape of elements in said source video frames;
mix said processed client video frames with said source video frames, using a mixing module, to yield a new mixed media asset, wherein said mixing comprises:
matching the frame rate or bit rate or resolution of the processed client video frames to the frame rate or bit rate or resolution of the plurality of source video frames.

### Claim 11 (depends on 10)

The apparatus of claim 10, wherein said processing comprises:
extracting said one or more markers from said identified at least one face image and superposing the extracted one or more markers on the other at least one identified face; and
resizing the other at least one identified face to match the size and shape of the at least one identified face in the at least one frame of the plurality of source video frames.

### Claim 12 (depends on 10)

The apparatus of claim 10, wherein said processing further comprises cropping or scaling the at least one other identified face image in the at least one frame of the plurality of client video frames to match the size or shape of said identified at least one face image in the source video frames.

### Claim 13 (depends on 10)

The apparatus of claim 10, wherein the resolution of the at least one source video is different from the resolution of the client video.

### Claim 14 (depends on 10)

The apparatus of claim 10, wherein the editing module and the analyzer module are configured to simultaneously fit the size or shape of elements in said client video frames to the size or shape of elements in said source video frames by analyzing successively each of the frames of said client video frames and said source video frames to identify one or more faces at said client video frames and said source video frames, and editing the identified face at each frame of said client video frames according to the location of the identified face in the at least one frame of the plurality of source video frames.

### Claim 15

A computer software product, comprising a non-transitory computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to provide at least one media asset source selected from a media asset sources library, the at least one media asset source comprising at least one source video, via a network to a client device, wherein the at least one source video comprises a plurality of source video frames;
receive via the network from the client device a media recording comprising a client video, wherein the client video comprises a plurality of client video frames and wherein the frame rate or bit rate of the one or more source media videos is different from the frame rate or bit rate of the client video;
parse the client video and the source video, respectively, to the plurality of client video frames and the plurality of source video frames;
identify at least one face image in at least one frame of the plurality of source video frames and at least other face in at least one frame of the plurality of client video frames using face detection method;
superpose one or more markers on the identified at least one face image in the at least one frame of the plurality of source video frames;
simultaneously process and edit said client video frames and said source video frames using an editing module and an analyzer module to fit the size or shape of elements in said client video frames to the size or shape of elements in said source video frames, said processing and editing comprises:
extracting said one or more markers from said identified at least one face image in the at least one frame of the plurality of source video frames; superposing the extracted one or more markers on the at least other face image; and
resizing the at least other face image to match the size or shape of the identified at least one face image, using the superposed one or more markers;
mixing said processed client video frames with said source video frames, using a mixing module, to yield a new mixed media asset
wherein said mixing comprises:
matching the frame rate or bit rate of the processed client video frames to the frame rate or bit rate of the plurality of source video frames.
