Intelligent method and system for producing and displaying stereoscopically-multiplexed images of three-dimensional objects for use in realistic stereoscopic viewing thereof in interactive virtual reality display environments

A stereoscopic image display subsystem for displaying and viewing images of virtual 3-D objects represented in image display space, comprising:

• a display surface having a first coordinate reference frame symbolically embedded therein;
• a head/eye position and orientation tracking subsystem including
• miniature head/eye imaging cameras for capturing images of a viewer's eyes,
• an image processing computer for processing said captured images on a real-time basis and producing a first information set representative of position and orientation of the eyes and head of said viewer relative to said first coordinate reference frame;
• and
• a display surface position and orientation tracking subsystem, operably associates with said image processing computer, for producing on a real-time basis, a second information set representative of the position and orientation of said display surface relative to a second coordinate reference frame symbolically embedded within said image display space.

An interactive-based system for producing and displaying stereoscopically-multiplexed images, comprising:

• (A) object representation subsystem for representing an object M relative to a first coordinate reference frame pm;
• (B) a stereoscopic image display subsystem including
• (1) a display structure having a display surface positioned relative to a viewer and supporting an array of pixels located at a set of coordinates specified by a coordinate set sd referenced relative to a second coordinate reference frame pd symbolically embedded in said display structure, said display surface being provided for displaying stereoscopically-multiplexed images Is of a virtual object M' corresponding to the object M, and being referenced relative to a third coordinate reference frame pw so that the eyes in the head of said viewer can view said stereoscopically-multiplexed images Is displayed on said display surface;
• (2) an eye/head position and orientation tracking subsystem including
• means for tracking the position and orientation of the head and eyes of the viewer relative to said display surface and producing head and eye position and orientation parameters,
• means for producing a left eye viewing transformation Tvel parameterized by said head and eye position and orientation parameters transformed to said third coordinate reference frame pw,
• means for producing a right eye viewing transformation Tver parameterized by said head and eye position and orientation parameters transformed to said third coordinate reference frame pw, and
• means for producing a head-display viewing transformation Tdv parameterized by said head and eye position and orientation parameters transformed to said third coordinate reference frame pw;
• (3) a display surface position and orientation tracking subsystem including
• means for tracking the position and orientation of said display surface relative to said third coordinate reference frame pw and producing display surface position and orientation parameters,
• means for producing a head/eye viewing transformation Twd parameterized by said display surface position and orientation parameters transformed to said second coordinate reference frame pd and relating said second coordinate reference frame pd to said third coordinate reference frame pw,
• means for providing a set of display surface parameters sd representing the size and aspect ratio of said display surface, and
• means for providing a set of image generation-display mapping parameters msd representing a process of mapping the pixels in the stereoscopically-multiplexed image Is onto pixels of a mapped stereoscopically-multiplexed image Id being displayed on said display surface;
• (C) stereoscopic image-pair production subsystem for producing stereoscopic image-pairs, consisting left and right perspective images of said object M, including
• means for geometrically defining said object M relative to said first coordinate reference frame pm,
• means for providing a left image acquisition surface scl on which said left perspective image of said object M is acquired and referenced relative to a fourth coordinate reference frame pcl, and a right image acquisition surface scr on which a right perspective image of said object M is acquired and referenced relative to a fifth coordinate reference frame pcr, wherein said left and right object surfaces scl and scr being defined relative to a sixth coordinate reference frame pq,
• means for producing an object-camera viewing transformation Tmq parameterized by said transformations Twd, Tdv, Tvel, Tver, and said first coordinate reference frame pm, for specifying the position and orientation placement of said left and right image acquisition surfaces scl and scr relative to said first coordinate reference frame pm,
• means for producing a left image acquisition surface transformation Tqcl parameterized by said Twd, Tdv, Tvel, and Tver, for describing the position and orientation placement of the left image acquisition surface scl relative to said sixth coordinate reference frame pq,
• means for producing a right image acquisition surface transformation Tqcr parameterized by said Twd, Tdv, Tvel and Tver, for describing the position and orientation placement of the right image acquisition surface scr relative to said sixth coordinate reference frame pq,
• means for mapping said left perspective image of said object M onto said left image acquisition surface scl using, as parameters, said transformations Tmq, Tqcl, Tdv and Tvel, and said parameters sd and msd, and
• means for mapping said right perspective image of said object M onto said right image acquisition surface using, as parameters, said transformations Tmq, Tqcr, Tdv and Tver, and said parameters sd and msd;
• and
• (D) stereoscopically-multiplexed image subsystem including
• means for producing a stereoscopically multiplexed image from each of said stereoscopic image-pairs, for mapping onto said display surface by way of said mapping process, and
• means for producing stereoscopically-multiplexed image parameters ss for representing the resolution and aspect ratio of the stereoscopically multiplexed image.

An process for producing and displaying stereoscopically-multiplexed images, comprising the steps:

• (1A) representing an object M relative to a first coordinate reference frame pm;
• (1B) providing a stereoscopic image display subsystem including a display structure having a display surface positioned relative to a viewer and supporting an array of pixels located at a set of coordinates specified by a coordinate set sd referenced relative to a second coordinate reference frame pd symbolically embedded in said display structure, said display surface being provided for displaying stereoscopically-multiplexed images Is of a virtual object M' corresponding to said object M, and being referenced relative to a third coordinate reference frame pw so that the eyes in the head of said viewer can view said stereoscopically-multiplexed images Is displayed on said display surface;
• (2A) tracking the position and orientation of the head and eyes of the viewer relative to said display surface and producing head and eye position and orientation parameters;
• (2B) producing a left eye viewing transformation Tvel parameterized by said head and eye position and orientation parameters transformed to said third coordinate reference frame pw;
• (2C) producing a right eye viewing transformation Tver parameterized by said head and eye position and orientation parameters transformed to said third coordinate reference frame pw;
• and
• (2D) producing a head-display viewing transformation Tdv parameterized by said head and eye position and orientation parameters transformed to said third coordinate reference frame pw;
• (3A) tracking the position and orientation of said display surface relative to said third coordinate reference frame pw and producing display surface position and orientation parameters;
• (3B) producing a head/eye viewing transformation Twd parameterized by said display surface position and orientation parameters transformed to said second coordinate reference frame pd and relating said second coordinate reference frame pd to said third coordinate reference frame pw;
• (3C) providing a set of display surface parameters sd representing the size and aspect ratio of said display surface;
• and
• (3D) providing a set of image generation-display mapping parameters msd representing a process of mapping the pixels in the stereoscopically-multiplexed image Is onto pixels of a mapped stereoscopically-multiplexed image Id being displayed on said display surface;
• (4A) geometrically defining said object M relative to said first coordinate reference frame pm;
• (4B) providing a left image acquisition surface scl on which a left perspective image of said object M is acquired and referenced relative to a fourth coordinate reference frame pcl, and a right image acquisition surface scr on which a right perspective image of said object M is acquired and referenced relative to a fifth coordinate reference frame pcr, wherein said left and right object surfaces scl and scr being defined relative to a sixth coordinate reference frame pq;
• (4C) producing an object-camera viewing transformation Tmq parameterized by said transformations Twd, Tdv, Tvel, Tver, and said first coordinate reference frame pm, for specifying the position and orientation placement of said left and right image acquisition surfaces scr and scl relative to said first coordinate reference frame pm;
• (4D) producing a left image acquisition surface transformation Tqcl parameterized by said Twd, Tdv, Tvel, and Tver, for describing the position and orientation placement of the left image acquisition surface scl relative to said sixth coordinate reference frame pq;
• (4E) producing a right image acquisition surface transformation Tqcr parameterized by said Twd, Tdv, Tvel and Tver, for describing the position and orientation placement of the right image acquisition surface relative scr to said sixth coordinate reference frame pq;
• (4F) mapping said left perspective image of said object M onto said left image acquisition surface scl using, as parameters, said transformations Tmq, Tqcl, Tdv and Tvel, and said parameters sd and msd;
• and
• (4G) mapping said right perspective image of said object M onto said right image acquisition surface scr using, as parameters, said transformations Tmq, Tqcr, Tdv and Tver, and said parameters sd and msd;
• and
• (5A) producing a stereoscopically multiplexed image from said left and right perspective images mapped onto said left and right image acquisition surfaces scl and scr, respectively, for mapping onto said display surface by way of said mapping process;
• and
• (5B) producing stereoscopically-multiplexed image parameters ss for representing the resolution and aspect ratio of the stereoscopically multiplexed image(s).