Real-time spherical light field reconstruction from a single omnidirectional image

Abstract

Using two neural networks—a reconstruction network and a refinement network—an image processing system and related method of operation converts omnidirectional image information, for example from a spherical camera, into a cubemap, from which, for each image frame, it reconstructs each of six cube faces of cubes in the cubemap into an array of the cube faces and, in a refinement process, removes border artifacts. The resulting frames can then be applied to a display for viewing by a user, for example, in a telepresence or augmented reality scenario. The invention thus reconstructs a spherical light field (LF) from a single one of the omnidirectional images, where the spherical LF represents spatial and angular information of the scene.

Dependencies are inferred from the claim text.