Back to results
US 2015/0317520 A1Published application
Method and apparatus for extraction of static scene photo from sequence of images
AssigneeSony Corporation
InventorSabri Gurbuz
Life of this patent
Estimating the term…
Abstract
An apparatus and method for extracting a static background image from a non-static image sequence or video sequence having at least three spatially overlapping frames is presented. Obscured static background areas are filled, according to the disclosure, with actual content as the background area becomes visible over time as non-static objects move with respect to the background. Consecutive image frames are stored in tracking buffers from which alignment is performed and absolute differences determined. Object contours are found in the difference image and bounding boxes determined as object masks. The background is then filled from areas outside these object masks to arrive at a static background image.
Claims20
An apparatus for extracting a static scene image from a sequence of images containing moving objects, comprising:
- (a) at least three tracking buffers configured for retaining downsized images from a sequence of images;
- (b) at least two application buffers configured for retaining full size images from the sequence of images;
- (c) a computer processor operably coupled for controlling said tracking buffers and said application buffers in the processing of the sequence of images;
- and
- (d) programming in a non-transitory computer readable medium and executable on the computer processor for performing steps comprising:
- (i) downsizing images from a sequence of images, and storing at least three sequential images in said tracking buffers;
- (ii) storing at least two images in said application buffers for extracting a static scene image;
- (iii) performing a moving object extraction process to determine information about moving objects in said sequential image in said tracking buffers;
- (iv) upsizing moving object information into full size object masks which can be applied to said application buffers;
- (v) forming a static scene image in a first buffer of said application buffers, using static scene image formation, in response to collecting portions of a scene from a second buffer of said application buffers, which exclude the locations of full size object masks of objects determined to be moving;
- and
- (vi) processing additional images in the sequence of images until a full static background image is available in said first image buffer in which all excluded locations have been filled from other images in said sequence of images.
9 more elements
An apparatus for extracting a static scene image from a sequence of images containing moving objects, comprising:
- (a) at least three tracking buffers configured for retaining downsized images from a sequence of images, said tracking buffers arranged as a circular buffer accessed through pointers whereby receipt of successive images does not require copying images from one buffer to another;
- (b) at least two application buffers configured for retaining full size images from the sequence of images;
- (c) a computer processor operably coupled for controlling said tracking buffers and said application buffers in processing the sequence of images;
- and
- (d) programming in a non-transitory computer readable medium and executable on the computer processor for performing steps comprising:
- (i) downsizing images from a sequence of images, and storing at least three sequential images in said tracking buffers;
- (ii) storing at least two images in said application buffers for extracting a static scene image;
- (iii) performing a moving object extraction process to determine information about moving objects in said sequential image in said tracking buffers;
- (iv) upsizing moving object information into full size object masks which can be applied to said application buffers;
- (v) forming a static scene image in a first buffer of said application buffers, using static scene image formation, in response to collecting portions of a scene from a second buffer of said application buffers, which exclude the locations of full size object masks of objects determined to be moving, so that as a previously formed background image is aligned with a current image, and the current image excluding object mask regions are copied to this previously formed background image, pixels in the previously formed background image are retained corresponding to current object mask regions;
- and
- (vi) processing additional images in the sequence of images until a full static background image is available in said first image buffer in which all excluded locations have been filled from other images in said sequence of images.
9 more elements
A method of extracting a static background scene from a sequence of images containing moving objects, comprising the steps of:
- (a) downsizing images from a sequence of images, and storing at least three sequential images in at least three tracking buffers configured for retaining downsized images from a sequence of images;
- (b) storing at least two images in at least two application buffers configured for retaining full size images from the sequence of images when extracting a static scene image;
- (c) extracting moving objects in a process to determine information about moving objects in said sequential image in said tracking buffers;
- (d) upsizing moving object information into full size object masks which can be applied to said application buffers;
- (e) forming a static scene image in a first buffer of said application buffers, using static scene image formation, in response to collecting portions of a scene from a second buffer of said application buffers, which exclude the locations of full size object masks of objects determined to be moving;
- and
- (f) processing additional images in the sequence of images until a full static background image is available in said first image buffer in which all excluded locations have been filled from other images in said sequence of images.
4 more elements
Dependencies are inferred from the claim text.