# US 2025/0097461 A1 · Method and System for Encoding a 3D Scene > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2025/0097461 A1 - Document ID: US20250097461A1 - Status: Published application - Kind: A1 - Application number: 18/900,169 - Filed: 2024-09-27 - Published: 2025-03-20 - Earliest claimed: 2021-04-29 - Assignees: Active Theory Inc. - Applicants: Active Theory Inc - Inventors: Joshua McCready; Alexander Gourley - Web page: https://patentgrep.com/patent/US20250097461A1 ## CPC - H04N 19/29: involving scalability at the object level, e.g. video object layer [VOL] - H04N 19/597: specially adapted for multi-view video sequence encoding - H04N 19/136: Incoming video signal characteristics or properties - H04N 19/179: the unit being a scene or a shot - H04N 19/31: in the temporal domain - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A computer-implemented method for encoding a scene volume includes: (a) identifying features of a scene volume that are within a camera perspective range with respect to a default camera perspective; (b) converting the identified features into rendered features; and (c) sorting the rendered features into a plurality of scene layers, each including corresponding depth, color, and transparency maps for the respective rendered features. Further, (a), (b), and (c) may be repeated, operating on temporally ordered scene volumes, to produce and output a sequence encoding a video. Corresponding systems and non-transitory computer-readable media are disclosed for encoding a 3D scene and for decoding an encoded 3D scene. Efficient compression, transmission, and playback of video describing a 3D scene can be enabled, including for virtual reality displays with updates based on a changing perspective of a user viewer for variable-perspective playback. ## Claims ### Claim at position 1 -19. (canceled) ### Claim 20 A computer-implemented method for outputting a sequence of packed multilayer scene representations (MSRs), the method comprising: a.) identifying a given scene volume of a plurality of scene volumes by querying a 4D scene volume with a time value associated with a corresponding camera perspective of a sequence of camera perspectives; b.) identifying rendered features of the given scene volume that are visible within a camera perspective range around the corresponding camera perspective; c.) sorting by depth the rendered features into a plurality of scene layers wherein each respective scene layer of the plurality of scene layers includes, for respective rendered features that are sorted into the respective scene layer: (i) a corresponding depth map for the respective rendered features, the depth map based on depth information from the scene volume, (ii) a corresponding color map for the respective rendered features, and (iii) a corresponding transparency map for the respective rendered features; d.) packing the plurality of scene layers and the camera perspective to produce a packed MSR; and e.) repeating steps a.), b.), c.), and d.) for scene volumes of the plurality of scene volumes and corresponding camera perspectives of the sequence of camera perspectives, to produce and output a sequence of packed MSRs. ### Claim 21 (depends on 20) The method of claim 20, further comprising constructing at least one 4D scene volume using sequential color images captured by a camera. ### Claim 22 (depends on 20) The method of claim 20, further comprising calculating the color of rendered features of at least one scene volume using aggregated observations from each camera perspective inside of the camera perspective range. ### Claim 23 (depends on 20) The method of claim 20, further comprising duplicating rendered features of a scene layer into an adjacent scene layer along transition boundaries in order to reduce the chance of visible seams during playback. ### Claim 24 (depends on 20) The method of claim 20, further comprising creating occluded features in at least one scene volume using a machine learning model. ### Claim 25 (depends on 20) The method of claim 20, further comprising selecting the camera perspective range by anticipating the viewing range of a 6DoF video playback. ### Claim 26 (depends on 20) The method of claim 20, further comprising sorting the rendered features of each scene volume into a fixed number of scene layers, the fixed number of scene layers being at least two scene layers. ### Claim 27 (depends on 26) The method of claim 26, further comprising packing and sequencing the sequence of packed MSRs into a packed MSR video. ### Claim 28 (depends on 27) The method of claim 27, wherein the packed MSR combines the transparency map of each non-background scene layer into a single 2D color array using a transform function which places the transparency map of each layer into a subset of a color gamut represented by each pixel. ### Claim 29 (depends on 27) The method of claim 27, further comprising reducing the size of the depth map before producing and outputting the packed MSR. ### Claim 30 A method for decoding a packed MSR video, the packed MSR video including a sequence of packed MSRs, the method comprising: decoding the sequence of packed MSRs, the decoding including: a.) extracting a packed MSR from the packed MSR video; b.) extracting a plurality of scene layers from the packed MSR; c.) determining a camera perspective from the packed MSR; d) assigning depth to respective features of a scene volume based on respective depth maps of respective scene layers and the camera perspective; e) assigning color to the respective features of a scene volume based on respective color maps of the respective scene layers; f) assigning transparency to the respective features of a scene volume based on respective transparency maps of the respective scene layers; and g) repeating steps a), b), c), d), e), and f) for each packed MSR of the sequence of packed MSRs. ### Claim 31 (depends on 30) The method of claim 30, further comprising determining a render camera perspective based, at least in part, on a time weighted average of a user head pose. ### Claim 32 A system for outputting a sequence of packed multilayer scene representations (MSRs), the system comprising: one or more processors configured to: a.) identify a given scene volume of a plurality of scene volumes by querying a 4D scene volume with a time value associated with a corresponding camera perspective of a sequence of camera perspectives; b.) identify rendered features of the given scene volume that are visible within a camera perspective range around the corresponding camera perspective; c.) sort by depth the rendered features into a plurality of scene layers wherein each respective scene layer of the plurality of scene layers includes, for respective rendered features that are sorted into the respective scene layer: (i) a corresponding depth map for the respective rendered features, the depth map based on depth information from the scene volume, (ii) a corresponding color map for the respective rendered features, and (iii) a corresponding transparency map for the respective rendered features; d.) pack the plurality of scene layers and the camera perspective to produce a packed MSR; and e.) repeat steps a.), b.), c.), and d.) for scene volumes of the plurality of scene volumes and corresponding camera perspectives of the sequence of camera perspectives, to produce and output a sequence of packed MSRs. ### Claim 33 One or more non-transitory, computer-readable media comprising instructions that, when executed by one or more processors, cause a device to: a.) identify a given scene volume of a plurality of scene volumes by querying a 4D scene volume with a time value associated with a corresponding camera perspective of a sequence of camera perspectives; b.) identify rendered features of the given scene volume that are visible within a camera perspective range around the corresponding camera perspective; c.) sort by depth the rendered features into a plurality of scene layers wherein each respective scene layer of the plurality of scene layers includes, for respective rendered features that are sorted into the respective scene layer: (i) a corresponding depth map for the respective rendered features, the depth map based on depth information from the scene volume, (ii) a corresponding color map for the respective rendered features, and (iii) a corresponding transparency map for the respective rendered features; d.) pack the plurality of scene layers and the camera perspective to produce a packed multilayer scene representation (MSR); and e.) repeat steps a.), b.), c.), and d.) for scene volumes of the plurality of scene volumes and corresponding camera perspectives of the sequence of camera perspectives, to produce and output a sequence of packed MSRs. ### Claim 34 A system for decoding a packed MSR video, the packed MSR video including a sequence of packed MSRs, the system comprising: one or more processors configured to: a.) extract a packed MSR from a packed MSR video; b.) extract a plurality of scene layers from the packed MSR; c.) determine a camera perspective from the packed MSR; d) assign depth to respective features of a scene volume based on respective depth maps of respective scene layers and the camera perspective; e) assign color to the respective features of a scene volume based on respective color maps of the respective scene layers; f) assign transparency to the respective features of a scene volume based on respective transparency maps of the respective scene layers; and g) repeat steps a), b), c), d), e), and f) for each packed MSR of the sequence of packed MSRs. ### Claim 35 One or more non-transitory, computer-readable media comprising instructions for decoding a sequence of packed MSRs that, when executed by one or more processors, cause a device to: a.) extract a packed MSR from a packed MSR video; b.) extract a plurality of scene layers from the packed MSR; c.) determine a camera perspective from the packed MSR; d) assign depth to respective features of a scene volume based on respective depth maps of respective scene layers and the camera perspective; e) assign color to the respective features of a scene volume based on respective color maps of the respective scene layers; f) assign transparency to the respective features of a scene volume based on respective transparency maps of the respective scene layers; and g) repeat steps a), b), c), d), e), and f) for each packed MSR of the sequence of packed MSRs.