Layered scene decomposition CODEC with view independent rasterization

Abstract

A system and methods for a CODEC driving a real-time light field display for multi-dimensional video streaming, interactive gaming and other light field display applications is provided applying a layered scene decomposition strategy. Multi-dimensional scene data is divided into a plurality of data layers. One or more polygons representative of corresponding portions of objects in the scene are obtained for each layer and used to determine a view-independent representation. This view independent representation and data layers are sampled using a plenoptic sampling scheme and rendered using hybrid rendering to encode light fields corresponding to each data layer. The resulting compressed, (layered) core representation of the multi-dimensional scene data is produced at predictable rates, reconstructed and merged at the light field display in real-time by applying view synthesis protocols, including edge adaptive interpolation, to reconstruct pixel arrays in stages (e.g. columns then rows) from reference elemental images.

Dependencies are inferred from the claim text.