Loading patent…

Back to results

US 4,970,666 AGranted

Computerized video imaging system for creating a realistic depiction of a simulated object in an actual environment

AssigneeLand Development Laboratory, Inc.

InventorsWilliam T. Welsh, Kenneth B. Pittman

Life of this patent

Estimating the term…

Download PDF Find prior artPrior art Similar

Abstract Drawings Claims Description

Abstract

A system and method for producing highly realistic video images which depict the appearance of a simulated structure in an actual environment, and provides for accurate placement of the structure in the environment and matching of the perspective of the structure with that of the environment so that a highly realistic result is achieved. The system includes a video input means, such as a video camera and video recorder, by which a video image of the actual environment may be captured. A graphics processor unit receives the video image from the video input means and stores it in rasterized form. Field data input means is provided for receiving field location data regarding the precise location of the captured video image and field perspective data regarding the perspective of the captured image. Object data input means is also provided for receiving data, such as CAD data for example, for a three-dimensional model of a simulated object which is proposed to be included in the environment. From the three-dimensional model of the object, a two-dimensional perspective representation of the object is generated, which accurately matches the perspective of the captured video image. The thus generated two-dimensional perspective representation is then merged with the rasterized video image and accurately positioned at its proper location in the environment.

Claims35

Dependent claims

Claim 1A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 226 words8 dependent Claim 10A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 303 words2 dependent Claim 13A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 356 wordsNo dependents Claim 14A system for producing a video image realistically depicting how a simulated object would appear in an actual environment · 296 words1 dependent Claim 16A method of producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 168 words4 dependent Claim 21A method of producing a video simulation image realistically depicting how a simulated object would appear in an actual environment · 208 words4 dependent Claim 26A method of producing a video image realistically depicting how a simulated object would appear in an actual environment · 304 wordsNo dependents Claim 27A method for producing a video image realistically depicting how a simulated object would appear in an actual environment · 248 wordsNo dependents Claim 28A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 237 wordsNo dependents Claim 29A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 267 wordsNo dependents Claim 30A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 346 wordsNo dependents Claim 31A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 360 wordsNo dependents Claim 32A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 285 wordsNo dependents Claim 33A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 357 wordsNo dependents Claim 34A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 325 wordsNo dependents Claim 35A method of producing a video image simulation realistically depicting how a simulated object would appear in an actual environment · 186 wordsNo dependents

A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) video input means for receiving a captured video image of an actual environment, the image having been obtained at a particular location in the actual environment and the image having a particular perspective;
(b) graphics processor means for receiving the video image from said video input means and for storing the video image in rasterized form;
(c) field data input means for receiving field location data regarding the location of the captured video image and field perspective data regarding the perspective of the captured image;
(d) object data input means for receiving data for a three-dimensional model of a simulated object which is proposed to be included in the environment;
(e) means for generating from the three-dimensional model of said object, through the use of said field perspective data, a two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured video image of the actual environment;
and
(f) means for merging the thus generated two-dimensional perspective representation of the object with said rasterized video image of the environment and for positioning the object, through the use of said field location data, so that the object is precisely positioned at the proper location in its environment.

4 more elements

A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) video input means for receiving a captured video image of an actual environment, the image having been obtained at a particular location in the actual environment and the image having a particular perspective;
(b) graphics processor means for receiving the video image from said video input means and for storing the video image in rasterized form;
(c) field data input means for receiving field location data regarding the location of the captured video image in relation to a known coordinate system and field perspective data regarding the perspective of the captured image;
(d) object data input means for receiving CAD data for a three-dimensional CAD model of a simulated object which is proposed to be included in the environment;
(e) object reference point input means for defining reference points representing known locations on the three-dimensional model and for assigning to said reference points coordinates in said known coordinate system where the reference points on the object are to be located;
(f) means for generating from the CAD data for the three-dimensional CAD model of said object, through the use of said field perspective data, a two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured video image of the actual environment;
and
(g) means for merging the thus generated two-dimensional perspective representation of the object with said rasterized video image of the environment and for positioning the object, through the use of said field location data, so that the defined reference points on the object are located at the proper coordinates in said known coordinate system of the actual environment so that the object is precisely positioned with respect to its environment.

5 more elements

A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) video input means for receiving a captured video image of an actual environment;
(b) graphics processor means for receiving the video image from said video input means and for storing the video image in rasterized form;
(c) field data input means for receiving field location data including the coordinates of a camera location which captured the video image in relation to a known coordinate system, the coordinates of at least one other known location in the field of view of the camera, and data relating to the optical characteristics of the camera which captured the video image;
(d) object data input means for receiving CAD data for a three-dimensional CAD model of a simulated object which is proposed to be included in the environment;
(e) object reference point input means for defining reference points representing known locations on the three-dimensional model and for assigning to said reference points coordinates in said known coordinate system where the reference points on the object are to be located;
(f) means for rotating, translating and scaling the three-dimensional CAD model so that the reference points on the three-dimensional model input by said object reference point input means coincide with the proper coordinate locations in the coordinate system of the actual environment,
(g) means for generating from the three-dimensional model of said object, through the use of said field perspective data, a rasterized two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured video image of the actual environment;
(h) means for superimposing the thus generated rasterized two-dimensional perspective representation of the object with said rasterized video image of the environment at corresponding coordinate locations in said coordinate system, so that the object is precisely positioned at the proper location in its environment, and to thereby produce a video image simulation accurately depicting the object as it would appear in the actual environment, and
(i) video output means for producing a video display of the thus produced video image simulation.

6 more elements

A system for producing a video image realistically depicting how a simulated object would appear in an actual environment, comprising:

(a) video input means for capturing a video image of the actual environment, the image having been obtained at a particular point of observation in the actual environment and with a particular point of view;
(b) graphics processor means for receiving said video image and forming therefrom a rasterized video image;
(c) object data input means, including means for storing data representing a three-dimensional CAD model of a simulated object which is proposed to be included in the environment;
(d) object data conversion means, including
(1) means for receiving perspective data indicating the point of observation and the direction of view of the captured video image of the actual environment;
(2) means for receiving reference points representing known locations on the three-dimensional CAD model and their proper coordinate positions in the coordinate system of the actual environment;
(3) means for rotating, translating and scaling said three-dimensional CAD model so that said reference points on the three-dimensional CAD model coincide with the proper coordinate position in the coordinate system of the actual environment;
(4) means for generating from said three-dimensional CAD model, a two-dimensional rasterized image providing a perspective representation of the three-dimensional image with the point of observation and direction of view represented by said perspective data;
(5) means for superimposing the thus produced two-dimensional rasterized image of said object onto the captured rasterized video image of the actual environment at the corresponding coordinate locations in said coordinate system to thereby produce a synthesized composite video image depicting the object as it would appear in the actual environment;
and
(e) video output means for producing a video display of the thus produced synthesized composite video image.

8 more elements

A method of producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) capturing an image of the actual environment,
(b) converting the captured image into rasterized form,
(c) obtaining field location data regarding a location of the captured image and field perspective data regarding the perspective of the captured image,
(d) providing object data defining a three-dimensional model of a simulated object which is proposed to be included in the environment,
(e) generating from the three-dimensional model of the object, through the use of the field perspective data, a rasterized two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured image of the actual environment, and
(f) merging the thus generated rasterized two-dimensional perspective representation of the object with said rasterized image of the environment while positioning the object, through the use of said field location data, so that the object is precisely positioned at the proper location in the environment.

3 more elements

A method of producing a video simulation image realistically depicting how a simulated object would appear in an actual environment, comprising

(a) capturing with a camera and storing in a storage medium an image of the actual environment;
(b) obtaining field location data accurately identifying a point of observation and direction of view of the captured image in relation to a known coordinate system;
(c) providing CAD data for a three-dimensional model of a simulated object which is proposed to be included in the environment;
(d) defining within the CAD model reference points representing known locations on the three-dimensional model and assigning to said reference points the coordinates in said known coordinate system where the reference points of the object are to be located;
(e) generating from the three-dimensional CAD model of said object a two-dimensional image providing a perspective representation of the three-dimensional object using the same point of observation and direction of view as used in the captured image of the actual environment;
and
(f) superimposing the thus produced two-dimensional image of said object onto the captured image of the actual environment at corresponding coordinates in said coordinate system to thereby produce a video image depicting the object as it would appear in the actual environment.

4 more elements

A method of producing a video image realistically depicting how a simulated object would appear in an actual environment, comprising

(a) capturing and storing an image of the actual environment;
(b) obtaining site data accurately identifying a point of observation and direction of view of the captured image in relation to a known coordinate system;
(c) selecting a landmark in the captured image of the actual environment;
(d) identifying from a map or survey, coordinates defining the location of said landmark in relation to said known coordinate system;
(e) creating a CAD verification model of the selected landmark and assigning to said model said coordinates in said known coordinate system which define the location of the landmark;
(f) superimposing the CAD verification model of said landmark onto said captured image at corresponding coordinates in said coordinate system to thereby ascertain the accuracy of the site data obtained in step (b), and correcting the site data, if necessary, to ensure that the CAD verification model of said landmark is in registration with the image of the landmark;
(g) providing a three-dimensional CAD model of a simulated object which is proposed to be included in the environment;
(h) establishing within said CAD model reference points representing known locations on the model and assigning to said reference points coordinates in said known coordinate system;
(i) generating from the three-dimensional CAD model of the object a two-dimensional image providing a perspective representation of the three-dimensional object based upon said site data so that the perspective view of the object matches that of the actual environment;
and
(j) superimposing the thus produced two-dimensional image of said object onto the captured image of the actual environment at corresponding coordinates in said coordinate system to thereby produce a video image depicting the object as it would appear in the actual environment.

8 more elements

A method for producing a video image realistically depicting how a simulated object would appear in an actual environment, comprising

(a) positioning a camera at a predetermined location in the actual environment and obtaining therefrom an image of the actual environment;
(b) obtaining and storing data accurately identifying the location of the camera in the actual environment and its direction of view with respect to a known coordinate system;
(c) rasterizing the image of the actual environment obtained from said camera and storing the rasterized image;
(d) providing data representing a three-dimensional CAD model of a simulated object which is proposed to be included in the environment, said model being comprised of data defining a vector representation of the object;
(e) establishing within the three-dimensional model, reference points representing known locations on the model, and assigning to said reference points coordinate positions in the coordinate system of the actual environment, and rotating, translating and scaling the model so that known locations on the model are positioned at their proper locations in the coordinate system of the actual environment;
(f) converting the vector representation of said model into a two-dimensional rasterized representation of the object in a three-dimensional perspective corresponding with the perspective of the image of the environment captured by the camera;
and
(g) overlaying the thus produced rasterized representation of the object with the rasterized image of the actual environment to thereby produce a video image depicting the object as it would appear in the actual environment.

5 more elements

A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) video input means for receiving a captured video image of an actual environment;
(b) graphics processor means for receiving the video image from said video input means and for storing the video image in rasterized form;
(c) field data input means for receiving field location data regarding the precise location of the captured video image and field perspective data regarding the perspective of the captured image;
(d) object data input means for receiving CAD data for a three-dimensional CAD model of a simulated object which is proposed to be included in the environment;
(e) means operatively associated with said object data input means for translating the input CAD data from one CAD data format to another standard CAD data format;
(f) means for generating from the three-dimensional CAD data for said object, through the use of said field perspective data, a two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured video image of the actual environment;
and
(g) means for merging the thus generated two-dimensional perspective representation of the object with said rasterized video image of the environment and for positioning the object, through the use of said field location data, so that the object is precisely positioned at the proper location in its environment.

5 more elements

A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) equipment calibration means for generating calibration data relating to characteristics of particular pieces of video input equipment which may be used to capture the video image which is input to said video input means, and
(b) an equipment calibration library for storing the equipment calibration data which is generated by said equipment calibration means,
(c) video input means for receiving a captured video image of an actual environment;
(d) graphics processor means for receiving the video image from said video input means and for storing the video image in rasterized form;
(e) field data input means for receiving field location data regarding the precise location of the captured video image and field perspective data regarding the perspective of the captured image;
(f) object data input means for receiving data for a three-dimensional model of a simulated object which is proposed to be included in the environment;
(g) means for generating from the three-dimensional model of said object, through the use of said field perspective data, a two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured video image of the actual environment;
and
(h) means for merging the thus generated two-dimensional perspective representation of the object with said rasterized video image of the environment and for positioning the object, through the use of said field location data and said equipment calibration data, so that the object is precisely positioned at the proper location in its environment.

6 more elements

A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) video input means for receiving a captured video image of an actual environment, the image having been obtained at a particular location in the actual environment and the image having a particular perspective;
(b) graphics processor means for receiving the video image from said video input means and for storing the video image in rasterized form;
(c) field data input means for receiving field location data regarding the location of the captured video image in relation to a known coordinate system and field perspective data regarding the perspective of the captured image;
(d) object data input means for receiving CAD data for a three-dimensional CAD model of a simulated object which is proposed to be included in the environment;
(e) object reference point input means for defining reference points representing known locations on the three-dimensional model and for assigning to said reference points coordinates in said known coordinate system where the reference points on the object are to be located;
(f) means for generating from the CAD data for the three-dimensional CAD model of said object, through the use of said field perspective data, a two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured video image of the actual environment, said means for generating a two-dimensional perspective representation including means for rotating and translating the three-dimensional CAD mode with the use of said reference points so that the reference points coincide with the proper coordinate position in the coordinate system of the environment;
and
(g) means for merging the thus generated two-dimensional perspective representation of the object with said rasterized video image of the environment and for positioning the object through the use of said field location data, so that the defined reference points on the object are located at the proper coordinates in said known coordinate system of the actual environment so that the object is precisely positioned with respect for its environment.

5 more elements

A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) video input means for receiving a captured video image of an actual environment, the image having been obtained at a particular location in the actual environment and the image having a particular perspective;
(b) graphics processor means for receiving the video image from said video input means and for storing the video image in rasterized form;
(c) field data input means for receiving field location data regarding the location of the captured video image in relation to a known coordinate system and field perspective data regarding the perspective of the captured image, the field perspective data including data defining a location of the camera and a center of view of the video image seen by the camera;
(d) object data input means for receiving CAD data for a three-dimensional CAD model of a simulated object which is proposed to be included in the environment;
(e) object reference point input means for defining reference points representing known locations on the three-dimensional model and for assigning to said reference points coordinates in said known coordinate system where the reference points on the object are to be located;
(f) means for generating from the CAD data for the three-dimensional CAD model of said object, through the use of said field perspective data, a two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured video image of the actual environment, said means for generating a two-dimensional perspective including means for transforming the perspective of the two-dimensional representation in accordance with said field perspective data defining the camera location and center of view;
and
(g) means for merging the thus generated two-dimensional perspective representation of the object with said rasterized video image of the environment and for positioning the object, through the use of said field location data, so that the defined reference points on the object are located at the proper coordinates in said known coordinate system of the actual environment so that the object is precisely positioned with respect to its environment.

5 more elements

A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) video input means for receiving a captured video image of an actual environment;
(b) graphics processor means for receiving the video image from said video input means and for storing the video image in rasterized form;
(c) field data input means for receiving field location data regarding a precise location of the captured video image in relation to a known coordinate system and field perspective data regarding the perspective of the captured image;
(d) object data input means for receiving CAD data for a three-dimensional CAD model of a simulated object which is proposed to be included in the environment;
(e) object reference point input means for defining reference points representing known locations on the three-dimensional model and for assigning to said reference points coordinates in said known coordinate system where the reference points on the object are to be located;
(f) means for generating from the CAD data for the three-dimensional CAD model of said object, through the use of said field perspective data, a two-dimensional perspective representation of the object in a spherical perspective to closely match any image distortion of the captured video image of the actual environment;
(g) means for merging the thus generated two-dimensional perspective representation of the object with said rasterized video image of the environment and for positioning the object, through the use of said field location data, so that the defined reference points on the object are located at the proper coordinates in said known coordinate system of the actual environment so that the object is precisely positioned with respect to its environment.

4 more elements

A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) video input means for receiving a captured video image of an actual environment, the image having been obtained at a particular location in the actual environment and the image having a particular perspective;
(b) graphics processor means for receiving the video image from said video input means and for storing the video image in rasterized form;
(c) field data input means for receiving field location data regarding the location of the captured video image in relation to a known coordinate system and field perspective data regarding the perspective of the captured image;
(d) object data input means for receiving CAD data for a three-dimensional CAD model of a simulated object which is proposed to be included in the environment;
(e) object reference point input means for defining reference points representing known locations on the three-dimensional model and for assigning to said reference points coordinates in said known coordinate system where the reference points on the object are to be located;
(f) means for generating from the CAD data for the three-dimensional CAD model of said object, through the use of said field perspective data, a two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured video image of the actual environment, said means for generating a two-dimensional perspective representation including line preprocessor means for segmenting long lines of the three-dimensional model into shorter segments before conversion to two-dimensional perspective so that long straight lines present in the object will be distorted in a way similar to distortion present in the captured video image of the environment;
and
(g) means for merging the thus generated two-dimensional perspective representation of the object with said rasterized video image of the environment and for positioning the object, through the use of said field location data, so that the defined reference points on the object are located the proper coordinates in said known coordinate system of the actual environment so that the object is precisely positioned with respect to its environment.

5 more elements

A video imaging system for producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) video input means for receiving a captured video image of an actual environment, the image having been obtained at a particular location in the actual environment and the image having a particular perspective;
(b) graphics processor means for receiving the video image from said video input means and for storing the video image in rasterized form;
(c) field data input means for receiving field location data regarding the location of the captured video image in relation to a known coordinate system and field perspective data regarding the perspective of the captured image;
(d) object data input means for receiving CAD data for a three-dimensional CAD model of a simulated object which is proposed to be included in the environment;
(e) object reference point input means for defining reference points representing known locations on the three-dimensional model and for assigning to said reference points coordinates in said known coordinate system where the reference points on the object are to be located;
(f) means for generating from the CAD data for the three-dimensional CAD model of said object, through the use of said field perspective data, a two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured video image of the actual environment, said means for generating a two-dimensional perspective including means for compensating for camera lens distortion to thereby provide a more accurate simulation;
and
(g) means for merging the thus generated two-dimensional perspective representation of the object with said rasterized video image of the environment and for positioning the object, through the use of said field location data, so that the defined reference points on the object are located at a proper coordinates in said known coordinate system of the actual environment so that the object is precisely positioned with respect to its environment.

5 more elements

A method of producing a video image simulation realistically depicting how a simulated object would appear in an actual environment, comprising

(a) capturing an image of the actual environment;
(b) converting the captured image into rasterized form;
(c) obtaining field location data regarding the precise location of the captured image and field perspective data regarding the perspective of the captured image;
(d) providing CAD data defining a three-dimensional model of a simulated object which is proposed to be included in the environment;
(e) translating the input CAD data from one CAD data format to another standard CAD data format;
(f) generating from the three-dimensional model of the object, through the use of the field perspective data, a rasterized two-dimensional perspective representation of the object in a perspective which accurately matches the perspective of the captured image of the actual environment;
and
(g) merging the thus generated rasterized two-dimensional perspective representation of the object with said rasterized image of the environment while positioning the object, through the use of said field location data, so that the object is precisely positioned at the proper location in the environment.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 07/175,158Filed 30 Mar 1988
Granted: 13 Nov 1990
Term ends: Not estimatedThe source records don’t support an estimate.

Classification4 codes

G06T 15/20
Geometric effects: perspective computation
G06T 19/006
Manipulating three-dimensional [3D] models or images for computer graphics: mixed reality
G06T 2215/16
Indexing scheme for image rendering: using real world measurements to influence rendering
G06T
Image data processing or generation, in general

Family

No related US applications are recorded in the indexed filing history.

Timeline

Citations

Open elsewhere

USPTO full text and PDF
Patent Center file history
USPTO assignment records
Espacenet
Google Patents