Virtual reality streaming system and method

A receiving virtual reality streaming system, comprising:

• a receiver adapted to receive a stream comprising a series of images captured by a wide field of view camera, and at least orientation information of the wide field of view camera as extrinsic camera parameters;
• a head mounted display ‘HMD’ having a field of view smaller than that of the wide field of view camera, the head mounted display comprising an inertial measurement unit operably to indicate at least the orientation of the head mounted display;
• and
• a display alignment processor;
• wherein
• the images captured by the wide field of view camera respectively comprise a predetermined main area of interest;
• and
• the display alignment processor is adapted to align the main area of interest in the received images with a viewpoint corresponding to a predetermined orientation of the HMD, based at least on the received orientation information of the camera and the indicated orientation of the HMD, and display the result on the HMD.

A VR streaming system comprising:

• a transmitting virtual reality streaming system, including:
• a wide field of view camera having a field of view that is wider than a field of view subsequently displayed to the recipient, the wide field of view camera comprising in turn:
• (i) one or more contributing cameras;
• and (ii) a status monitor, and the status monitor comprising in turn:
• (i) an inertial measurement unit operable to capture at least orientation information of the camera as extrinsic camera parameters;
• and (ii) a streaming encoder adapted to stream successive images along with corresponding extrinsic camera parameters;
• and
• a receiving virtual reality streaming system, including:
• a receiver adapted to receive a stream comprising a series of images captured by a wide field of view camera, and at least orientation information of the wide field of view camera as extrinsic camera parameters;
• a head mounted display ‘HMD’ having a field of view smaller than that of the wide field of view camera, the head mounted display comprising an inertial measurement unit operably to indicate at least the orientation of the head mounted display;
• and a display alignment processor;
• wherein (i) the images captured by the wide field of view camera respectively comprise a predetermined main area of interest;
• and (ii) the display alignment processor is adapted to align the main area of interest in the received images with a viewpoint corresponding to a predetermined orientation of the HMD, based at least on the received orientation information of the camera and the indicated orientation of the HMD, and display the result on the HMD.

A method of transmitting a virtual reality stream, comprising the steps of:

• capturing with a camera a wide field of view image having a field of view that is wider than a field of view subsequently displayed to the recipient,
• capturing at least orientation information of the camera as extrinsic camera parameters;
• and
• streaming successive images along with corresponding extrinsic camera parameters.

A method of receiving a virtual reality stream, comprising the steps of:

• receiving a stream comprising successive wide field of view images captured by a camera, and at least corresponding orientation information of the camera;
• receiving indications of at least an orientation of a head mounted display ‘HMD’ having a field of view smaller than the wide field of view images;
• aligning a respective predetermined main area of interest in the received images with a viewpoint corresponding at least to a predetermined orientation of the HMD, based on at least the received orientation information of the camera and the indicated orientation of the HMD;
• and
• displaying the result on the HMD.

A non-transitory, computer readable storage medium containing a computer program comprising computer executable instructions adapted to cause a computer system to perform a method of transmitting a virtual reality stream, comprising the steps of:

• capturing with a camera a wide field of view image having a field of view that is wider than a field of view subsequently displayed to the recipient,
• capturing at least orientation information of the camera as extrinsic camera parameters;
• and
• streaming successive images along with corresponding extrinsic camera parameters.

A non-transitory, computer readable storage medium containing a computer program comprising computer executable instructions adapted to cause a computer system to perform a method of receiving a virtual reality stream, comprising the steps of:

• receiving a stream comprising successive wide field of view images captured by a camera, and at least corresponding orientation information of the camera;
• receiving indications of at least an orientation of a head mounted display ‘HMD’ having a field of view smaller than the wide field of view images;
• aligning a respective predetermined main area of interest in the received images with a viewpoint corresponding at least to a predetermined orientation of the HMD, based on at least the received orientation information of the camera and the indicated orientation of the HMD;
• and
• displaying the result on the HMD.