US 5,657,402 A · Method of creating a high resolution still image using a plurality of images and apparatus for practice of the method

Abstract

The invention is a method for generating a still image, comprising the steps of producing a plurality of images, each of the plurality having been produced at a distinct focal length, scaling each of the images to a common focal length and combining each of the scaled images to a final image of a single focal length, portions of which are of a relatively high resolution, as compared to the images of the original sequence. The invention also includes combining a sequence of still images of varying fields of view into a panoramic image of an overall field of view, which overall field of view includes all of the fields of view of the sequence. In addition to combining images generated at different fields of view, the method of the invention can be used to combine images generated with respect to different fields of view of an overall scene, such as a panoramic scene into a combined panoramic field of view. This aspect of the invention may also be combined with the varying focal length aspect. Even without varying the focal length or the field of view, the invention can be used to produce a composite image of enhanced resolution relative to the resolution of any of the images of the original sequence. The invention is also an apparatus for generating a still image, comprising means for producing a plurality of images, each of the plurality having been produced at a distinct focal length, the focal lengths differing from each other, means for scaling each of the plurality of images to a common focal length and means for combining each of the scaled images into a single image of a single focal length. The apparatus of the invention also includes apparatus to combine images generated with respect to different fields of view of an overall scene into a combined panoramic field of view.

Claims17

Dependent claims

A method for generating a signal corresponding to a still, perceptible image representing a physical situation, comprising the steps of:

a. using electromagnetic radiation, capturing a plurality of at least three images of said situation, each of said plurality having been captured at a distinct focal length, said focal lengths differing from each other and each of at least three overlapping images of said plurality sharing an overlap region that corresponds to the same portion of said situation:
b. generating, for each of said plurality of images, an electromagnetic signal representing said image, resultina in at least three electromagnetic signals, each signal designated an overlap signal, each one of said overlap signals representing said overlap region of one of said three overlapping images:
c. transforming each overlap signal so that it represents the respective image scaled to a common focal length and aligned to a common field of view, said transforming step comprising the steps of, for each said overlap signal:
i. applying to said signal at least one affine transformation comprising the steps of:
A. ordering said plurality of overlap signals in a sequence;
B. for each sequentially adjacent pair of overlap signals in said sequence, determining a set of affine parameters substantially defining a transformation of the image represented by a first of said pair to the image represented by the second of said pair;
C. for at least one of said plurality of overlap signals, combining a plurality of said sets of affine parameters into a composite set of affine parameters;
and
D. applying an affine transformation to said at least one overlap signal using said respective composite set of affine parameters;
and
ii. generating a signal that represents said transformed overlap signal;
and
d. combining each of said transformed overlap signals into a resultant signal that represents the combination of each of said scaled images into a single image of said situation of a single focal length using an aspect of each of said at least three overlap signals.

10 more elements

A method for generating a signal corresponding to a still, perceptible image representing a physical situation, comprising the steps of:

a. using electromagnetic radiation, capturing a plurality of at least three images of said situation, each of said plurality having been captured at a distinct focal length, said focal lengths differing from each other and each of at least three overlapping images of said plurality sharing an overlap region that corresoonds to the same portion of said situation:
b. generating, for each of said plurality of images, an electromagnetic signal representing said image, resulting in at least three electromagnetic signals, each signal designated an overlap signal, each one of said overlap signals representing said overlap region of one of said three overlapping images;
c. transforming each overlap signal so that it represents the respective image, scaled to a common focal length and aligned to a common field of view;
and
d. identifying differences between pairs of signals representing pairs of images, which signal differences are due to relative motions between pairs of images that are due to causes other than the fact that the two images were produced at different focal lengths said step of identifying differences between pairs of signals that are due to relative motions comprising the steps of:
A. estimating a first relative motion of a first pattern portion of both signals of a pair;
A. using said estimated first motion to determine a second relative motion of a second pattern portion of both signals;
C. repeating the following steps until a desired resolution of relative motion is achieved:
.alpha.. using said second relative motion to specify more precisely said first relative motion of said first pattern portion;
and
.beta.. using said more precise specification of said first relative motion to specify more precisely said second relative motion of said second pattern portion;
and
e. combining each of said transformed overlap signals into a resultant signal that represents the combination of each of said scaled images into a single image of said situation of a single focal length using an aspect of each of said at least three overlap signals.

10 more elements

A method for generating a signal corresponding to a still, perceptible image of a physical situation, comprising the steps of:

a. using electromagnetic radiation, capturing a plurality of images of said situation, each of said plurality having been produced covering a distinct field of view, said fields of view differing from each other and being members of an overall field of view, each field of view overlapping at least one other field of view, each pair of overlapping images of said plurality sharing an overlap region that corresponds to the same portion of said situation, and said overall field of view corresponding to a greater extent of said situation than any single image of said plurality;
b. generating, for each of an overlapping pair of said plurality of images, an electromagnetic signal representing said image, each signal designated an overlap signal;
c. without reference to the locations of features reletive to said overall field of view or said physical situation, transforming each overlap signal so that it represents the respective image aligned to said overall field of view, said transformation being conducted by the steps of, for each said overlap signal:
i. applying to said overlap signal at least one affine transformation by conducting the steps of;
A. ordering said plurality of overlap signals in a sequence;
B. for each sequentially adjacent pair of overlap signals in said sequence, determining a set of affine parameters substantially defining a transformation of said image represented by a first of said pair of overlap signals to the image represented by the second of said pair of signals;
C. for at least one of said plurality of overlap signals, combining a plurality of said sets of affine parameters into a composite set of affine parameters;
and
D. applying an affine transformation to said at least one overlap signal using said respective composite set of affine parameters;
and
ii. generating a signal that represents said transformed overlap signal;
and
d. combining each of said transformed overlap signals into a resultant signal that represents the combination of each of said aligned images into a single image of said situation of said overall field of view.

10 more elements

A method for generating a signal that represents a still, perceptible image of a physical situation comprising the steps of:

a. establishing a plurality of different sampling lattices bearing no predetermined spatial relationship to one another;
b. using electromagnetic radiation, capturing a plurality of images of said situation, each of said plurality having been captured at a distinct time with a different of said plurality of different sampling lattices, a region of each of said plurality of images constituting an image of the same portion of said situation as is captured by at least one other of said images;
c. generating, for each of said images, an electromagnetic signal representative of said image;
d. transforming each signal so that it represents the respective image, aligned to a common field of view;
and
e. combining at least two of said transformed signals, using a sampling lattice of higher resolution than any sampling lattice of any image of said plurality into a resultant signal that represents the combination of at least two of said images into a single image having an enhanced resolution over any of said original images.

3 more elements

A method for generating a signal corresoonding to a still, oerceotible image representing a physical situation, comprising the steps of:

a. using electromagnetic radiation, capturing a plurality of at least three images of said situation, each of said plurality having been captured at a distinct focal length, said focal lengths differing from each other and each of at least three overlapping images of said plurality sharing an overlap region that corresponds to the same portion of said situation;
b. .circle-w/dot. generating, for each of said plurality of images, an electromagnetic signal representing said image, resulting in at least three electromagnetic signals, each signal designated an overlap signal, each one of said overlap signals representing said overlap region of one of said three overlapping images;
c. transforming each overlap signal so that it represents the resoective image, scaled to a common focal length and aligned to a common field of view, said transforming step comprising the steps of:
i. ordering said plurality of overlap signals into a sequence;
ii. for at least two of said overlap signals, applying to said at least two signals at least one affine transformation;
iii. wherein for at least one of said at least two of said overlap signals, said at least one affine transformation transforms said signal so that it represents the respective image aligned to a common field of view with an image of said sequence that is distant from said respective image in said sequence;
and
iv. generating a signal that represents said transformed overlap signal;
and
d. combining each of said transformed overlap signals into a resultant signal that represents the combination of each of said scaled images into a single image of said situation of a single focal length, said single image being arranged according to a combination sampling lattice that defines a plurality of pixels, using an aspect of each of said at least three overlap signals at each pixel of said single image that represents said combined overlap signals.

7 more elements

A method for generating a signal corresponding to a still, perceptible image representing a physical situation, comprising the steps of:

a. using electromagnetic radiation, capturing a plurality of at least three images of said situation, each of said plurality having been captured at a distinct focal length, said focal lengths differing from each other and each of at least three overlapping images of said plurality sharing an overlap region that corresponds to the same portion of said situation:
b. generating, for each of said plurality of images, an electromagnetic signal representing said image, resulting in at least three electromagnetic signals, each signal designated an overlap signal, each one of said overlap signals representing said overlap region of one of said three overlapping images;
c. transformina each overlap signal so that it represents the respective image, scaled to a common focal length and aligned to a common field of view, said transforming step comprising the steps of, for each said overlap signal;
i. applying to said signal at least one affine transformation comprising the steps of:
A. ordering said plurality of overlap signals in a sequence;
B. for each sequentially adjacent pair of overlap signals in said sequence, determining a set of affine parameters substantially defining a transformation of the image represented by a first of said pair to the image represented by the second of said pair;
C. for at least one of said plurality of overlap signals, applying a fixst affine transformation to said at least one overlap signal using said affine parameters determined with respect to said overlap signal as the first of a pair and an adjacent overlap signal as the second of said pair to generate a first transformed overlap signal;
and
D. applying to said first transformed overlap signal a second affine transformation using said affine parameters determined with respect to a second pair of overlap signals that comprise:
.alpha.. said adjacent overlap signal as the first signal of said second pair;
and
.beta.. another overlap signal, different from said at least one overlap signal, as the second signal of said second pair;
and
ii. generating a signal that represents said transformed overlap signal;
and
d. combining each of said transformer overlap signals into a resultant signal that represents the combination of each of said scaled images into a sinale image of said situation of a single focal length using an aspect of each of said at least three overlap signals.

13 more elements

Dependencies are inferred from the claim text.

Method of creating a high resolution still image using a plurality of images and apparatus for practice of the method

Life of this patent

Method of creating a high resolution still image using a plurality of images and apparatus for practice of the method

Life of this patent

Abstract

Claims17