US 11,113,801 B1Granted

Robust image motion detection using scene analysis and image frame pairs

AssigneeApple Inc.

InventorWu Cheng

Life of this patent

Estimating the term…

Loading patent…

Abstract

Devices, methods, and computer-readable media describing an adaptive approach to reference image selection are disclosed herein, e.g., to generate fused images with reduced motion distortion. More particularly, an incoming image stream may be obtained from an image capture device, which image stream may comprise a variety of different image captures, e.g., including “image frame pairs” (IFPs) that are captured consecutively, wherein the images in a given IFP are captured with differing exposure settings. When a capture request is received at the image capture device, the image capture device may select two or more images from the incoming image stream for fusion, e.g., including at least one IFP. In some embodiments, one of the images from the at least one IFP will be designated as the reference image for a fusion operation, e.g., based on a robust motion detection analysis process performed on the images of the at least one IFP.

Claims20

Dependent claims

A device, comprising:

a memory;
one or more image capture devices;
a display;
a user interface;
and
one or more processors operatively coupled to the memory, wherein the one or more processors are configured to execute instructions causing the one or more processors to:
obtain an incoming image stream from the one or more image capture devices;
select two or more images from the incoming image stream, wherein the selected two or more images comprise at least one image frame pair (IFP), wherein the IFP comprises a first image frame and a second image frame that were captured consecutively and with differing exposure settings;
determine one of the first image frame or the second image frame to be designated as a reference image, wherein the determination is based, at least in part, on a comparison of an amount of motion in the first image frame to an amount of motion in the second image frame;
determine one or more of the selected two or more images not designated as a reference image to be designated as candidate images;
register the one or more candidate images with the reference image;
fuse the registered one or more candidate images and the reference image, resulting in a fused image;
and
store the fused image in the memory.

11 more elements

A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:

obtain an incoming image stream from one or more image capture devices;
select two or more images from the incoming image stream, wherein the selected two or more images comprise at least one image frame pair (IFP), wherein the IFP comprises a first image frame and a second image frame that were captured consecutively and with differing exposure settings;
determine one of the first image frame or the second image frame to be designated as a reference image, wherein the determination is based, at least in part, on a comparison of an amount of motion in the first image frame to an amount of motion in the second image frame;
determine one or more of the selected two or more images not designated as a reference image to be designated as candidate images;
register the one or more candidate images with the reference image;
fuse the registered one or more candidate images and the reference image, resulting in a fused image;
and
store the fused image in a memory.

5 more elements

An image processing method, comprising:

obtaining an incoming image stream from one or more image capture devices;
selecting two or more images from the incoming image stream, wherein the selected two or more images comprise at least one image frame pair (IFP), wherein the IFP comprises a first image frame and a second image frame that were captured consecutively and with differing exposure settings;
determining one of the first image frame or the second image frame to be designated as a reference image, wherein the determination is based, at least in part, on a comparison of an amount of motion in the first image frame to an amount of motion in the second image frame;
determining one or more of the selected two or more images not designated as a reference image to be designated as candidate images;
registering the one or more candidate images with the reference image;
fusing the registered one or more candidate images and the reference image, resulting in a fused image;
and
storing the fused image in a memory.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/563,327Filed 6 Sep 2019 · us provisional application
Priority: 11 Sep 2018
Granted: 7 Sep 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification8 codes

G06T 5/50
Image enhancement or restoration: using two or more images, e.g. averaging or subtraction
G06T 7/20
Image analysis: analysis of motion
G06T 7/248
Using feature-based methods, e.g. the tracking of corners or segments: involving reference images or patches
G06T 7/269
Analysis of motion: using gradient-based methods