Automatic composition of composite images or videos from frames captured with moving camera
AssigneeGoPro, Inc.
InventorsBalineedu Chowdary Adsumilli, Ryan Lustig
Life of this patent
Estimating the term…
Abstract
A processing device generates composite images from a sequence of images. The composite images may be used as frames of video. A foreground/background segmentation is performed at selected frames to extract a plurality of foreground object images depicting a foreground object at different locations as it moves across a scene. The foreground object images are stored to a foreground object list. The foreground object images in the foreground object list are overlaid onto subsequent video frames that follow the respective frames from which they were extracted, thereby generating a composite video.
Claims20
A method for generating a composite image of an object from a plurality of image frames obtained via a camera experiencing motion, the method comprising:
- generating foreground object images of the object from the plurality of image frames based on a foreground-background segmentation;
- obtaining motion information of the camera for the plurality of image frames;
- determining relative positions of the camera for the plurality of image frames with respect to a current position of the camera for a current frame based on the motion information of the camera;
- generating transformed foreground object images by transforming the foreground object images based on the relative positions of the camera for the plurality of image frames with respect to the current position of the camera for the current frame, wherein the transforming of the foreground object images comprises applying a partial transparency to at least one of the foreground object images;
- and
- generating the composite output image based an overlay of the transformed foreground object images onto the current frame.
A system for generating a composite image of an object from a plurality of image frames obtained via a camera experiencing motion, the system comprising:
- one or more processors;
- and
- a non-transitory computer-readable storage medium storing instructions therein that when executed cause the one or more processors to perform:
- generating foreground object images of the object from the plurality of image frames based on a foreground-background segmentation;
- obtaining motion information of the camera for the plurality of image frames;
- determining relative positions of the camera for the plurality of image frames with respect to a current position of the camera for a current frame based on the motion information of the camera;
- generating transformed foreground object images by transforming the foreground object images based on the relative positions of the camera for the plurality of image frames with respect to the current position of the camera for the current frame, wherein the transforming of the foreground object images comprises applying a partial transparency to at least one of the foreground object images;
- and
- generating the composite output image based an overlay of the transformed foreground object images onto the current frame.
A system for generating a composite image of an object from a plurality of image frames obtained via a camera experiencing motion, the system comprising:
- one or more processors;
- and
- a non-transitory computer-readable storage medium storing instructions therein that when executed cause the one or more processors to perform:
- generating foreground object images of the object from the plurality of image frames based on a foreground-background segmentation;
- obtaining motion information of the camera for the plurality of image frames;
- determining relative positions of the camera for the plurality of image frames with respect to a current position of the camera for a current frame based on the motion information of the camera;
- generating transformed foreground object images by transforming the foreground object images based on the relative positions of the camera for the plurality of image frames with respect to the current position of the camera for the current frame, wherein the transforming of the foreground object images comprises applying a partial transparency to at least one of the foreground object images;
- and
- generating the composite output image based an overlay of the transformed foreground object images onto the current frame;
- wherein the partial transparency is applied to a given foreground object image:
- based on an overlap of the given foreground object image with one or more foreground object images within the current frame such that the partial transparency is applied to a first foreground object image based on the first foreground object image overlapping with a second foreground object image within the current frame and the partial transparency is not applied to a third foreground object image based on the third foreground object image not overlapping with any foreground object image within the current frame;
- and
- based on the given foreground object image taking up more than a threshold amount of the current frame such that the partial transparency is applied to a fourth foreground object based on the fourth foreground object taking up more than the threshold amount of the current frame and the partial transparency is not applied to a fifth foreground object based on the fifth foreground object not taking up more than the threshold amount of the current frame.
Dependencies are inferred from the claim text.