# US 9,519,983 B2 · Image processing device, image processing method and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,983 B2 - Document ID: US9519983B2 - Status: Granted - Kind: B2 - Application number: 14/063,512 - Filed: 2013-10-25 - Published: 2014-05-15 - Granted: 2016-12-13 - Earliest claimed: 2012-11-12 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Masae Koyama - Web page: https://patentgrep.com/patent/US9519983B2 ## CPC - G06T 13/80: Two-dimensional [2D] animation, e.g. using sprites - G06T 3/4038: Image mosaicing, e.g. composing plane images from plane sub-images - G06T 7/11: Region-based segmentation - G06T 7/194: involving foreground-background segmentation - G06T 11/60: Creating or editing images; Combining images with text - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - H04N 23/80: Camera processing pipelines; Components thereof - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - G06T 2207/10016: Video; Image sequence - G06T 2207/20221: Image fusion; Image merging - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract According to an illustrative embodiment, an image processing device is provided. The image processing device includes a foreground selection processing circuit to select at least one foreground image that has been separated from a source image; a background selection circuit to select at least two display background images from at least one background image that has been separated from the source image; and a combination circuit to combine the at least one selected foreground image with the at least two display background images to generate a plurality of combined images, wherein at least one of the plurality of combined images does not appear in the source image. ## Claims ### Claim 1 An image processing device, comprising: one or more central processing units (CPU) configured to: extract a foreground image corresponding to a target object from a target frame among a plurality of input frames of an input motion image; output a first output frame of an output motion image, in case the target object is imaged in a first current frame of the input motion image, wherein the first output frame includes the target object associated with the foreground image extracted from the target frame, at least a part of the first current frame; output a second output frame of the output motion image, in case the target object is not imaged in a second current frame, wherein the second output frame is generated by combining the foreground image extracted from the target frame, and at least a part of the second current frame; and output a plurality of output frames of the output motion image including the first output frame and the second output frame, wherein each of the plurality of input frames correspond to each of the plurality of output frames. ### Claim 2 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs are configured to extract the at least part of the first current frame or the second current frame of the input motion image from the target frame among the plurality of input frames of the input motion image. ### Claim 3 (depends on 1) The device as recited in claim 1, wherein the source image is a still image. ### Claim 4 (depends on 1) The device as recited in claim 1, wherein the source image is a still image formed by a plurality of images. ### Claim 5 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs are configured to output at least one frame from the plurality of output frames as an output frame of the output motion image for the target frame among the plurality of input frames of the input motion image. ### Claim 6 (depends on 1) The device as recited in claim 1, wherein the at least two display background images are each a portion of a still background image. ### Claim 7 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs are configured to extract a most recently extracted foreground image corresponding to the target object as a currently extracted foreground image. ### Claim 8 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs selects a best foreground image as a currently selected foreground image. ### Claim 9 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs are configured to extract the foreground image corresponding to the target object based on a user input. ### Claim 10 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs are configured to extract the foreground image corresponding to the target object from the target frame among the plurality of input frames of the input motion image. ### Claim 11 (depends on 10) The device as recited in claim 10, wherein the one or more CPUs selects at least one foreground image based on at least one criteria selected from the group consisting of whether or not the foreground image is blurred, whether or not a subject of the foreground image is smiling, whether or not a subject of the foreground image has closed eyes, and the brightness of the foreground image. ### Claim 12 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs are configured to determine at least one of a size and a position of a display region for use in extracting the foreground image corresponding to the target object. ### Claim 13 (depends on 1) The device as recited in claim 1, further comprising a memory for storing the plurality of output frames of the output motion image. ### Claim 14 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs are configured to determine based on a predetermined number, at least one of a size and a position of a display region for use in selecting a display background image from the at least one background image. ### Claim 15 (depends on 1) The device as recited in claim 1, wherein the device is incorporated in a camera, the camera comprising an imaging circuit and a display. ### Claim 16 (depends on 1) The device as recited in claim 1, wherein the source image is a moving image, at least two foreground images are selected, and the plurality of combined images make up a moving image in which the at least two selected foreground images make up a foreground moving image and the at least two display background images make up a background moving image, and in which, at least one of the foreground moving image and the background moving image is reproduced at a speed that is different from a reproduction speed of the source image. ### Claim 17 (depends on 16) The device as recited in claim 16, wherein a reproduction frame rate of the foreground moving image is different from a reproduction frame rate of the background moving image. ### Claim 18 (depends on 1) The device as recited in claim 1, wherein the first current frame is a frame preceding the second current frame in the input motion image. ### Claim 19 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs are further configured to extract more than one foreground image corresponding to the target object from the target frame among the plurality of input frames of the input motion image. ### Claim 20 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs are further configured to display a foreground display region associated with the foreground image, wherein the foreground image is detected based on motion detection corresponding to the target object in the target frame among the plurality of input frames of the input motion image. ### Claim 21 (depends on 20) The device as recited in claim 20, wherein the one or more CPUs are further configured to modify the foreground display region of the input motion image based on a user operation. ### Claim 22 (depends on 1) The device as recited in claim 1, wherein the one or more CPUs are further configured to determine a first frame and a last frame among the plurality of input frames of the input motion image based on a user operation. ### Claim 23 (depends on 22) The device as recited in claim 22, wherein the target frame is either the first frame or the last frame of the input motion image. ### Claim 24 An image processing method, comprising: extracting by an image processing device, a foreground image corresponding to a target object from a target frame among a plurality of input frames of an input motion image; outputting by said image processing device, a first output frame of an output motion image, in case the target object is imaged in a first current frame of the input motion image, wherein the first output frame includes the target object associated with the foreground image extracted from the target frame, and at least a part of a first current frame; outputting by said image processing device, a second output frame of an output motion image, in case the target object is not imaged in a second current frame, wherein the second output frame is generated by combining the foreground image extracted from the target frame, and at least a part of a second current frame; and outputting by said image processing device, a plurality of output frames of the output motion image including the first output frame and the second output frame, wherein each of the plurality input frames correspond to each of the plurality of output frames. ### Claim 25 A non-transitory computer-readable medium storing a set of computer-readable instructions for causing a computer to perform steps comprising: extracting a foreground image corresponding to a target object from a target frame among a plurality of input frames of an input motion image; outputting a first output frame of an output motion image, in case the target object is imaged in a first current frame of the input motion image, wherein the first output frame includes the target object associated with the foreground image extracted from the target frame, and at least a part of a first current frame; outputting a second output frame of an output motion image, in case the target object is not imaged in a second current frame, wherein the second output frame is generated by combining the foreground image extracted from the target frame, at least a part of a second current frame; and output a plurality of output frames of the output motion image including the first output frame and the second output frame wherein each of the plurality input frames correspond to each of the plurality of output frames.