Efficient canvas view generation from intermediate views

#This disclosure relates generally to the generation of canvas views for a virtual reality headset, and more particularly to canvas view generation from images captured by cameras. A canvas view represents a panoramic wide-angle view to recreate a scene in virtual reality, and can be paired with other canvas views to give a 3D stereoscopic effect of the scene. Existing techniques for canvas view generation can operate slowly, for example requiring manual stitching or other input from a user, and can encounter problems when dealing with discrepancies in the source camera views, such as different brightness or color between camera views.

#An embodiment of an invention can generate a canvas view of a scene based on a set of original camera views or images depicting the scene, for example captured by cameras of an image capture system and depicting a scene captured by a plurality of cameras. Canvas views can be generated based on a first mapping associating each region of the canvas view with a region of a synthetic view based on the location of the synthetic and a second mapping associating regions of camera views with regions of the synthetic view. The generated mappings can be combined to generate a combined mapping associating each region of the canvas view with regions of one or more camera views of the set of camera views which can then be applied to the camera views to generate the canvas view.

#A synthetic view can be generated, for example, based on a first and second camera view representing images of the scene sharing one or more common objects. An optical flow associating pixels between the first and second camera views can be used to relate the first and second camera views. Based on the optical flow, the first and second camera views can be “shifted” to each approximate the desired synthetic view. Both approximations of the synthetic view can then be blended or averaged together (i.e., the pixel color values) to generate the synthetic view.

#FIG. 1 is a block diagram of a system environment in which a canvas generation system operates, in accordance with an embodiment of the invention.

#FIG. 2 is a block diagram of a canvas generation system, in accordance with an embodiment of the invention.

#FIG. 3 is a line diagram showing the construction of an example image capture system, according to some embodiments.

#The figures depict various embodiments of the present invention for purposes of illustration only. One skilled in the art will readily recognize from the following discussion that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles of the invention described herein.

#System Architecture

#FIG. 1 is a block diagram of a system environment in which a canvas generation system operates, in accordance with an embodiment of the invention. The system environment 100 shown by FIG. 1 comprises an image capture system 105, a canvas generation system 110, and a client virtual reality (VR) device 115. In other embodiments, the system environment 100 can include different or additional components.