US 8,009,932 B2Granted

Automatic alignment of video frames for image processing

AssigneeProvidence Engineering and Environmental Group LLC

InventorsLiming Zhou, Yousheng Zeng

Life of this patent

Estimating the term…

Loading patent…

Abstract

The provided is a method that can automatically align image frames in recorded video clips. Individual frames in video may shift and rotate due to shaking or vibration of a video camera. Unaligned frames make some imaging processing techniques difficult or infeasible. One example of image processing techniques is to isolate, recognize, and/or quantitatively analyze vapor plume images captured by an Infrared (IR) camera. Such techniques have a great potential to be used to automatically detect volatile organic compounds (VOC) leaked from process equipment at refineries and chemical plants. This invention is a technique for various subsequent image processing techniques. The invention uses spatially based Fast Fourier Transforms (FFT) to determine amount of shift, rotation, and scaling to align image frames, and uses a digital filtering technique to improve the alignment.

Claims19

Dependent claims

A method for aligning image frames, comprising steps of:

selecting a reference image;
selecting a sample image to be aligned to the reference image;
Fourier-transforming the reference image to obtain a Fourier transform of the reference image;
Fourier-transforming the sample image to obtain a Fourier transform of the sample image;
coordinate-transforming an absolute value of the Fourier transform of the reference image to obtain a coordinate-transformed Fourier transform of the reference image;
coordinate-transforming an absolute value of the Fourier transform of the sample image to obtain a coordinate-transformed Fourier transform of the sample image;
obtaining a first phase shift from the coordinate-transformed Fourier transform of the reference image and the coordinate-transformed Fourier transform of the sample image;
inverse-Fourier-transforming the first phase shift to obtain an inverse Fourier transform of the first phase shift;
finding a first transformation factor from the inverse Fourier transform of the first phase shift;
transforming the sample image by the first transformation factor to obtain a first-transformed sample image;
Fourier-transforming the first-transformed sample image to obtain a Fourier transform of the first-transformed sample image;
obtaining a second phase shift of the Fourier transform of the reference image and the Fourier transform of the first-transformed sample image;
inverse-Fourier-transforming the second phase shift to obtain an inverse Fourier transform of the second phase shift;
finding a second transformation factor from the inverse Fourier transform of the second phase shift;
and
transforming the first-transformed sample image by the second transformation factor.

13 more elements

A non-transitory machine readable storage medium providing instructions that cause the machine to perform operations to align image frames, the operations comprising:

selecting a reference image that is represented in Cartesian coordinates;
selecting a sample image to be aligned to the reference image, the sample image being represented in Cartesian coordinates;
Fourier-transforming the reference image to obtain a Fourier transform of the reference image;
Fourier-transforming the sample image to obtain a Fourier transform of the sample image;
coordinate-transforming an absolute value of the Fourier transform of the reference image from Cartesian coordinates to log-polar coordinates to obtain a coordinate-transformed Fourier transform of the reference image;
coordinate-transforming an absolute value of the Fourier transform of the sample image from Cartesian coordinates to log-polar coordinates to obtain a coordinate-transformed Fourier transform of the sample image;
obtaining a rotational phase shift from the coordinate-transformed Fourier transform of the reference image and the coordinate-transformed Fourier transform of the sample image;
inverse-Fourier-transforming the rotational phase shift to obtain an inverse Fourier transform of the rotational phase shift;
finding a rotational shift from the inverse Fourier transform of the rotational phase shift;
rotating the sample image by the rotational shift to obtain a rotated sample image;
Fourier-transforming the rotated sample image to obtain a Fourier transform of the rotated sample image;
obtaining a translational phase shift of the Fourier transform of the reference image and the Fourier transform of the rotated sample image;
inverse-Fourier-transforming the translational phase shift to obtain an inverse Fourier transform of the translational phase shift;
finding a translational shift from the inverse Fourier transform of the translational phase shift;
and
translating the rotated sample image by the translational shift.

13 more elements

Dependencies are inferred from the claim text.

Facts

Application: 11/898,219Filed 10 Sep 2007 · us provisional application
Priority: 13 Sep 2006
Published: US 2008/0063298 A113 Mar 2008
Granted: 30 Aug 2011
Term ends: Not estimatedThe source records don’t support an estimate.

Classification5 codes

G06T 7/37
Determination of transform parameters for the alignment of images, i.e. image registration: using transform domain methods
G06V 10/243
Aligning, centring, orientation detection or correction of the image: by compensating for image skew or non-uniform image deformations
G06T 2207/10048
Image acquisition modality: infrared image
G06T
Image data processing or generation, in general