System and method for creating a navigable, three-dimensional virtual reality environment having ultra-wide field of view

A system for creating a navigable, panoramic three-dimensional virtual reality environment with stereoscopic perception of three-dimensional depth to a user, comprising:

• a storage device for storing first and second video image streams corresponding to two different stereoscopic viewpoints of a scene, respectively, each stream comprising image frames;
• a tracking device configured to track the movement of a user;
• a position detector disposed in communication with the tracking device to receive tracking data from the tracking device and configured to determine a direction of view of the user;
• an image renderer disposed in communication with the storage device to receive the first and second video image streams and disposed in communication with the position detector to receive the direction of view, the image renderer configured to create an output stream of merged stereo image frames corresponding to the direction of view of the user, the merged frames comprising image frames from the first video stream which are temporally matched to respective image frames from the second video stream;
• and
• a head mounted display disposed in communication with the image renderer to receive output stream and configured to display the output stream to the user, whereby a navigable, panoramic three-dimensional virtual reality environment having stereoscopic perception of three-dimensional depth is created.