Rendering 360 depth content
AssigneeFacebook, Inc.
InventorForrest Samuel Briggs
Life of this patent
Estimating the term…
Abstract
As user device can receive and display 360 panoramic content in a 360 depth format. 360 depth content can comprise 360 panoramic image data and corresponding depth information. To display 360 depth content, the user device can generate a 3D environment based on the 360 depth content and the current user viewpoint. A content display module on the user device can render 360 depth content using a standard 3D rendering pipeline modified to render 360 depth content. The content display module can use a vertex shader or fragment shader of the 3D rendering pipeline to interpret the depth information of the 360 depth content into the 3D environment as it is rendered.
Claims20
A method comprising:
- receiving, at a user device, one or more visual content frames comprising image pixel data and corresponding depth information for the image pixel data, wherein the image pixel data is independent of a viewpoint and each visual content frame including a range of viewing angles;
- rendering, at the user device, at least one output image for a, each output image corresponding to a visual content frame of the one or more visual content frames by:
- generating at least a portion of a 3D sphere corresponding to the range of viewing angles of the visual content frame, the 3D sphere comprising a set of vertices, each vertex associated with a set of texture coordinates corresponding to image pixel data of the visual content frame;
- determining a set of fragments based on the 3D sphere, each fragment comprising a fragment position and pixel intensity information;
- applying a fragment shader to each fragment of the set of fragments, the fragment shader modifying the pixel intensity information of each fragment based on the fragment position associated with the fragment and the visual content frame;
- generating an output image based on the modified set of fragments;
- and
- sending the at least one output image for display to a user of the user device.
A non-transitory computer readable storage medium comprising instructions which, when executed by a processor, cause the processor to perform the steps of:
- receiving, at a user device, one or more visual content frames comprising image pixel data and corresponding depth information for the image pixel data, wherein the image pixel data is independent of a viewpoint and each visual content frame including a range of viewing angles;
- rendering, at the user device, at least one output image for a, each output image corresponding to a visual content frame of the one or more visual content frames by:
- generating at least a portion of a 3D sphere corresponding to the range of viewing angles of the visual content frame, the 3D sphere comprising a set of vertices, each vertex associated with a set of texture coordinates corresponding to image pixel data of the visual content frame;
- determining a set of fragments based on the 3D sphere, each fragment comprising a fragment position and pixel intensity information;
- applying a fragment shader to each fragment of the set of fragments, the fragment shader modifying the pixel intensity information of each fragment based on the fragment position associated with the fragment and the visual content frame;
- generating an output image based on the modified set of fragments;
- and
- sending the at least one output image for display to a user of the user device.
Dependencies are inferred from the claim text.