US 10,757,346 B1Granted

Systems and methods for video extraction and insertion

AssigneeFlixbay Technologies, Inc.

InventorBrian Eric Adams

Life of this patent

Estimating the term…

Abstract

Systems and methods for extracting objects from a first video and inserting said objects into a second new background. In an embodiment, at least one hardware processor is used to receive a first video depicting a first scene comprising one or more objects and a background; identify the one or more objects in the first video; generating a first video layer by extracting the identified one or more objects from the background of the first video; receive a second video layer depicting a second scene; and merge the first video layer and second video layer to generate a composite scene, wherein the one or more objects of the first video layer overlaid on the second scene such that the one or more objects appear as part of the second scene.

Claims16

Dependent claims

A method comprising:

using at least one hardware processor to:
receive a plurality of image frames of a first video depicting a first scene comprising one or more objects and a background;
build a reference frame based on at least a subset of image frames of the plurality of image frames of the first video, one or more image frames of the subset of image frames comprising the one or more objects and the background;
identify the one or more objects in the first video based, in part, on a comparison of the plurality of image frames of the first video against the reference frame;
adjust the first video using edge identification and tracking to compensate for movement in the first video due to at least one of translational, tilt, and rotational movement in a device used to acquire the first video;
generate a first video layer by extracting the identified one or more objects from the background of the first video;
receive a second video layer depicting a second scene;
and
merge the first video layer and second video layer to generate a composite scene, wherein the one or more objects of the first video layer overlaid on the second scene such that the one or more objects appear as part of the second scene,
wherein each image frame of the plurality of image frames comprises a hierarchical data structure of pixels arranged in a plurality of abstraction levels comprising at least one or more first areas, a plurality of second areas, and a plurality of third areas, the one or more first areas including a first plurality of pixels, each one or more first areas comprising the plurality of second areas including a second plurality of pixels fewer than the first plurality of pixels, and each second area of the plurality of second areas comprising the plurality of third areas including a third plurality of pixels fewer than the second plurality of pixels, and wherein identifying the one or more objects is based, in part, on sequentially analyzing each image frame based on the hierarchical data structure from the one or more first areas to the plurality of third areas.

7 more elements

A mobile device comprising:

at least one hardware processor;
and
one or more software modules that, when executed by the at least one hardware processor,
receive a plurality of image frames of a first video depicting a first scene comprising one or more objects and a background;
build a reference frame based on at least a subset of image frames of the plurality of image frames of the first video, one or more image frames of the subset of image frames comprising the one or more objects and the background;
identify the one or more objects in the first video based, in part, on a comparison of the plurality of image frames of the first video against the reference frame;
adjust the first video using edge identification and tracking to compensate for movement in the first video due to at least one of translational, tilt, and rotational movement in a device used to acquire the first video;
generate a first video layer by extracting the identified one or more objects from the background of the first video;
receive a second video layer depicting a second scene;
and
merge the first video layer and second video layer to generate a composite scene, wherein the one or more objects of the first video layer overlaid on the second scene such that the one or more objects appear as part of the second scene,
wherein each image frame of the plurality of image frames comprises a hierarchical data structure of pixels arranged in a plurality of abstraction levels comprising at least one or more first areas, a plurality of second areas, and a plurality of third areas, the one or more first areas including a first plurality of pixels, each one or more first areas comprising the plurality of second areas including a second plurality of pixels fewer than the first plurality of pixels, and each second area of the plurality of second areas comprising the plurality of third areas including a third plurality of pixels fewer than the second plurality of pixels, and wherein identifying the one or more objects is based, in part, on sequentially analyzing each image frame based on the hierarchical data structure from the one or more first areas to the plurality of third areas.

9 more elements

A non-transitory computer-readable medium having instructions stored therein, wherein the instructions, when executed by a processor, cause the processor to:

receive a plurality of image frames of a first video depicting a first scene comprising one or more objects and a background;
build a reference frame based on at least a subset of image frames of the plurality of image frames of the first video, one or more image frames of the subset of image frames comprising the one or more objects and the background;
identify the one or more objects in the first video based, in part, on a comparison of the plurality of image frames of the first video against the reference frame;
adjust the first video using edge identification and tracking to compensate for movement in the first video due to at least one of translational, tilt, and rotational movement in a device used to acquire the first video;
generate a first video layer by extracting the identified one or more objects from the background of the first video;
receive a second video layer depicting a second scene;
and
merge the first video layer and second video layer to generate a composite scene, wherein the one or more objects of the first video layer overlaid on the second scene such that the one or more objects appear as part of the second scene,
wherein each image frame of the plurality of image frames comprises a hierarchical data structure of pixels arranged in a plurality of abstraction levels comprising at least one or more first areas, a plurality of second areas, and a plurality of third areas, the one or more first areas including a first plurality of pixels, each one or more first areas comprising the plurality of second areas including a second plurality of pixels fewer than the first plurality of pixels, and each second area of the plurality of second areas comprising the plurality of third areas including a third plurality of pixels fewer than the second plurality of pixels, and wherein identifying the one or more objects is based, in part, on sequentially analyzing each image frame based on the hierarchical data structure from the one or more first areas to the plurality of third areas.

6 more elements

Dependencies are inferred from the claim text.

patentgrep

Loading patent…

Abstract

A method comprising:

using at least one hardware processor to:
receive a plurality of image frames of a first video depicting a first scene comprising one or more objects and a background;
build a reference frame based on at least a subset of image frames of the plurality of image frames of the first video, one or more image frames of the subset of image frames comprising the one or more objects and the background;
identify the one or more objects in the first video based, in part, on a comparison of the plurality of image frames of the first video against the reference frame;
adjust the first video using edge identification and tracking to compensate for movement in the first video due to at least one of translational, tilt, and rotational movement in a device used to acquire the first video;
generate a first video layer by extracting the identified one or more objects from the background of the first video;
receive a second video layer depicting a second scene;
and
merge the first video layer and second video layer to generate a composite scene, wherein the one or more objects of the first video layer overlaid on the second scene such that the one or more objects appear as part of the second scene,
wherein each image frame of the plurality of image frames comprises a hierarchical data structure of pixels arranged in a plurality of abstraction levels comprising at least one or more first areas, a plurality of second areas, and a plurality of third areas, the one or more first areas including a first plurality of pixels, each one or more first areas comprising the plurality of second areas including a second plurality of pixels fewer than the first plurality of pixels, and each second area of the plurality of second areas comprising the plurality of third areas including a third plurality of pixels fewer than the second plurality of pixels, and wherein identifying the one or more objects is based, in part, on sequentially analyzing each image frame based on the hierarchical data structure from the one or more first areas to the plurality of third areas.

7 more elements