# US 12,192,651 B2 · Methods, systems, and media for generating compressed images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,192,651 B2 - Document ID: US12192651B2 - Status: Granted - Kind: B2 - Application number: 18/463,477 - Filed: 2023-09-08 - Published: 2024-02-29 - Granted: 2025-01-07 - Earliest claimed: 2020-01-07 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Ryan Overbeck - Web page: https://patentgrep.com/patent/US12192651B2 ## CPC - H04N 19/00: Methods or arrangements for coding, decoding, compressing or decompressing digital video signals - H04N 25/41: {Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors} - G06T 15/06: Ray-tracing - G06T 15/08: Volume rendering - G06T 15/205: {Image-based rendering} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Methods, systems, and media for generating compressed images are provided. In some embodiments, the method comprises: identifying a multi-plane image, MPI, that represents a three-dimensional image, wherein the MPI comprises a plurality of fronto-parallel planes; splitting the MPI into a plurality of sub-volumes, wherein each sub-volume in the plurality of sub-volumes includes a subset of the plurality of fronto-parallel planes; calculating, for each sub-volume of the MPI, a depthmap; converting each depthmap to a mesh, wherein each mesh corresponds to a layer of a plurality of layers associated with a multi-depth image, MDI, to be rendered; calculating, for each layer of the plurality of layers, an image; and storing the meshes corresponding to the plurality of layers of the MDI and the images corresponding to the plurality of layers of the MDI as the MDI. ## Claims ### Claim 1 A method, comprising: identifying a multi-plane image (MPI) that represents a three-dimensional image, the MPI including a plurality of fronto-parallel planes; using a neural network: forming, using the MPI, a plurality of sub-volumes, wherein a sub-volume in the plurality of sub-volumes includes a subset of the plurality of fronto-parallel planes; calculating, for a sub-volume of the MPI, a depthmap; converting the depthmap to a mesh corresponding to a layer of a plurality of layers associated with a multi-depth image (MDI) to be rendered; and calculating, for the plurality of layers, an image; and processing the mesh corresponding to the plurality of layers associated with the MDI and the image corresponding to the plurality of layers associated with the MDI as the MDI. ### Claim 2 (depends on 1) The method of claim 1, wherein at least one of: the depthmap indicates a location and a depth of a voxel of the MDI included in the layer of the MDI, and wherein a number of layers in the plurality of layers associated with the MDI is less than a number of fronto-parallel planes included in the plurality of fronto-parallel planes associated with the MPI; or in response to receiving a request for the three-dimensional image from a user device, transmitting the MDI to the user device configured to render the MDI by mapping the layer of the MDI to the image corresponding to the layer as a texture on the mesh corresponding to the layer. ### Claim 3 (depends on 1) The method of claim 1, further comprising generating a sequence of MDI images corresponding to a sequence of MPI images, wherein the sequence of MPI images corresponds to three-dimensional video content. ### Claim 4 (depends on 1) The method of claim 1, wherein forming the MPI into the plurality of sub-volumes includes generating a plurality of cuts for the plurality of sub-volumes by reducing a rendering error generated by rendering the MDI using the plurality of sub-volumes. ### Claim 5 (depends on 4) The method of claim 4, wherein the rendering error includes: a unary term that indicates an error in depth resulting from rendering the MDI using a cut of the plurality of cuts, or wherein the rendering error includes a smoothness term that indicates a smoothness of a cut of the plurality of cuts across voxels included in a sub-volume of the plurality of sub-volumes corresponding to the cut. ### Claim 6 (depends on 1) The method of claim 1, wherein forming the MPI into the plurality of sub-volumes includes using a trained neural network to identify a plurality of cuts that generate the plurality of sub-volumes. ### Claim 7 (depends on 1) The method of claim 1, wherein the mesh corresponding to the plurality of layers associated with the MDI is a triangular mesh. ### Claim 8 A system for generating compressed images, the system comprising: a hardware processor that is configured to: identify a multi-plane image (MPI) that represents a three-dimensional image, wherein the MPI comprises a plurality of fronto-parallel planes; using a neural network: forming, using the MPI, a plurality of sub-volumes, wherein a sub-volume in the plurality of sub-volumes includes a subset of the plurality of fronto-parallel planes; calculate, for a sub-volume of the MPI, a depthmap; convert the depthmap to a mesh corresponding to a layer of a plurality of layers associated with a multi-depth image (MDI) to be rendered; and calculate, for the plurality of layers, an image; and processing the mesh corresponding to the plurality of layers associated with the MDI and the image corresponding to the plurality of layers associated with the MDI as the MDI. ### Claim 9 (depends on 8) The system of claim 8, wherein at least one of: the depthmap indicates a location and a depth of a voxel of the MDI included in the layer of the MDI, and wherein a number of layers in the plurality of layers associated with the MDI is less than a number of fronto-parallel planes included in the plurality of fronto-parallel planes associated with the MPI; or in response to receiving a request for the three-dimensional image from a user device, transmitting the MDI to the user device configured to render the MDI by mapping the layer of the MDI to the image corresponding to the layer as a texture on the mesh corresponding to the layer. ### Claim 10 (depends on 8) The system of claim 8, wherein the hardware processor is further configured to generate a sequence of MDI images corresponding to a sequence of MPI images, wherein the sequence of MPI images corresponds to three-dimensional video content. ### Claim 11 (depends on 8) The system of claim 8, wherein forming the MPI into the plurality of sub-volumes includes generating a plurality of cuts for the plurality of sub-volumes by reducing a rendering error generated by rendering the MDI using the plurality of sub-volumes. ### Claim 12 (depends on 11) The system of claim 11, wherein the rendering error includes: a unary term that indicates an error in depth resulting from rendering the MDI using a cut of the plurality of cuts, or the rendering error includes a smoothness term that indicates a smoothness of a cut of the plurality of cuts across voxels included in a sub-volume from the plurality of sub-volumes corresponding to the cut. ### Claim 13 (depends on 8) The system of claim 8, wherein forming the MPI into the plurality of sub-volumes includes using a trained neural network to identify a plurality of cuts that generate the plurality of sub-volumes. ### Claim 14 (depends on 8) The system of claim 8, wherein the mesh corresponding to the plurality of layers associated with the MDI is a triangular mesh. ### Claim 15 A non-transitory computer-readable medium containing computer executable instructions that, when executed by a processor, cause the processor to perform a method for generating compressed images, the method comprising: identifying a multi-plane image (MPI) that represents a three-dimensional image, wherein the MPI comprises a plurality of fronto-parallel planes; using a neural network: forming, using the MPI, a plurality of sub-volumes, wherein a sub-volume in the plurality of sub-volumes includes a subset of the plurality of fronto-parallel planes; calculating, for a sub-volume of the MPI, a depthmap; converting the depthmap to a mesh corresponding to a layer of a plurality of layers associated with a multi-depth image (MDI) to be rendered; and calculating, for the plurality of layers, an image; and processing the mesh corresponding to the plurality of layers associated with the MDI and the image corresponding to the plurality of layers associated with the MDI as the MDI. ### Claim 16 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein the method further comprises generating a sequence of MDI images corresponding to a sequence of MPI images, wherein the sequence of MPI images corresponds to three-dimensional video content. ### Claim 17 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein forming the MPI into the plurality of sub-volumes includes generating a plurality of cuts for the plurality of sub-volumes by reducing a rendering error generated by rendering the MDI using the plurality of sub-volumes. ### Claim 18 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the rendering error includes a unary term that indicates an error in depth resulting from rendering the MDI using a cut of the plurality of cuts. ### Claim 19 (depends on 17) The non-transitory computer-readable medium of claim 17, wherein the rendering error includes a smoothness term that indicates a smoothness of a cut of the plurality of cuts across voxels included in a sub-volume from the plurality of sub-volumes corresponding to the cut. ### Claim 20 (depends on 15) The non-transitory computer-readable medium of claim 15, wherein forming the MPI into the plurality of sub-volumes includes using a trained neural network to identify a plurality of cuts that generate the plurality of sub-volumes.