US 7,479,967 B2 · System for combining virtual and real-time environments

Abstract

The present invention relates to a method and an apparatus for combining virtual reality and real-time environment. The present invention provides a system that combines captured real-time video data and real-time 3D environment rendering to create a fused (combined) environment. The system captures video imagery and processes it to determine which areas should be made transparent (or have other color modifications made), based on sensed cultural features and/or sensor line-of-sight. Sensed features can include electromagnetic radiation characteristics (i.e. color, infra-red, ultra-violet light). Cultural features can include patterns of these characteristics (i.e. object recognition using edge detection). This processed image is then overlaid on a 3D environment to combine the two data sources into a single scene. This creates an effect where a user can look through ‘windows’ in the video image into a 3D simulated world, and/or see other enhanced or reprocessed features of the captured image.

Claims20

Dependent claims

A virtual reality system comprising:

a camera adapted to receive images of a physical environment and to produce a frame of the physical environment that contains pixels representing a plurality of colors;
a predetermined target color;
a means for processing the frame and rendering pixels of the target color of the frame to be transparent pixels;
a means for generating a virtual image;
a means for combining the virtual image with the frame of the physical environment to form a combined image whereby the virtual image is visible at all positions of the transparent pixels;
and,
a means for displaying the combined image.

4 more elements

A computer-implemented system for combining a virtual reality environment and a physical environment for a user comprising:

a computer;
a camera operatively connected to the computer and adapted to provide to the computer real-time physical environment video images in digital pixels and adapted to be mounted on the head of a user at a location from which the camera has a view substantially similar to the view of the user's eyes;
a physical object operatively connected to the computer, and adapted to interact with the user and to provide input to the computer in response to interaction with the user;
a virtual image generator operatively connected to the computer and adapted to provide to the computer virtual video images in digital pixels;
a position detector operatively connected to the computer, and adapted to be mounted on the head of the user and to provide to the computer 3D spatial information about the location and direction of the user's head;
an image display operatively connected to the computer, adapted to be mounted on the head of the user and adapted to receive video images from the computer;
a real-time physical environment;
a pre-determined target masking color covering a predesignated area of the real-time physical environment;
all areas of the real-time physical environment video images in which the color of the pixels of the real-time physical environment video images match the target color being transparent;
the computer adapted to change the virtual video images in response to the input to the computer from the physical object;
the real-time physical environment video images overlaid onto the virtual video images to form combined video images;
and,
the combined video images provided to the image display.

10 more elements

Dependencies are inferred from the claim text.

System for combining virtual and real-time environments

Life of this patent

System for combining virtual and real-time environments

Life of this patent

Abstract

Claims20