# US 10,872,463 B2 · Depth-compressed representation for 3D virtual scene > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,463 B2 - Document ID: US10872463B2 - Status: Granted - Kind: B2 - Application number: 16/372,242 - Filed: 2019-04-01 - Published: 2020-10-01 - Granted: 2020-12-22 - Earliest claimed: 2019-04-01 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Microsoft Technology Licensing, LLC - Inventors: Dag Birger Frommhold; Pavlos Mavridis; Benjamin Markus Thaut - Web page: https://patentgrep.com/patent/US10872463B2 ## CPC - G06T 15/40: Hidden part removal - G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - H04L 67/131: Protocols for games, networked simulations or virtual reality - G06T 2210/08: Bandwidth reduction - G06T 2210/12: Bounding box - G06T 2215/16: Using real world measurements to influence rendering - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) ## Abstract A method for compressing depth information for a three-dimensional virtual scene includes, at a computing device, recognizing a three-dimensional virtual scene including one or more locally-maintained virtual objects. Positioning information for one or more remotely-maintained virtual objects to be included in the three-dimensional virtual scene are received from a remote device. For a virtual view of the three-dimensional virtual scene having a virtual perspective, different portions of the virtual view are classified as belonging to one of a plurality of different occlusion classes, each of the plurality of different occlusion classes specifying an occlusion relationship between locally-maintained virtual objects and remotely-maintained virtual objects in the virtual view. A depth-compressed representation indicating the occlusion class for each of the different portions of the virtual view is output to the remote device. ## Claims ### Claim 1 A method for compressing depth information for a three-dimensional virtual scene, comprising: at a computing device, recognizing a three-dimensional virtual scene including one or more locally-maintained virtual objects; receiving, from a remote device, positioning information for one or more remotely-maintained virtual objects to be included in the three-dimensional virtual scene; for a virtual view of the three-dimensional virtual scene having a virtual perspective, classifying different portions of the virtual view as belonging to one of a plurality of different occlusion classes, each of the plurality of different occlusion classes specifying an occlusion relationship between locally-maintained virtual objects and remotely-maintained virtual objects in the virtual view; and outputting a depth-compressed representation of the virtual view to the remote device, the depth-compressed representation indicating the occlusion class for each of the different portions of the virtual view. ### Claim 2 (depends on 1) The method of claim 1, further comprising receiving the virtual perspective from the remote device. ### Claim 3 (depends on 1) The method of claim 1, where the plurality of different occlusion classes includes a no-occlusion class, the no-occlusion class specifying an occlusion relationship in which a locally-maintained virtual object is either not present or not occluded by one or more remotely-maintained virtual objects. ### Claim 4 (depends on 1) The method of claim 1, where the plurality of different occlusion classes includes a full-occlusion class, the full-occlusion class specifying an occlusion relationship in which a locally-maintained virtual object is fully occluded by one or more remotely-maintained virtual objects. ### Claim 5 (depends on 1) The method of claim 1, where the plurality of different occlusion classes includes a partial-occlusion class, the partial-occlusion class specifying an occlusion relationship in which a locally-maintained virtual object is partially occluded by one or more remotely-maintained virtual objects. ### Claim 6 (depends on 5) The method of claim 5, where the depth-compressed representation of the virtual view includes, for each different portion of the virtual view belonging to the partial-occlusion class, a pixel occlusion indicator for each of a plurality of pixels in the portion, the pixel occlusion indicator specifying whether the pixel is occluded by a remotely-maintained virtual object. ### Claim 7 (depends on 1) The method of claim 1, where the plurality of different occlusion classes includes an intersection class, the intersection class specifying an occlusion relationship in which a locally-maintained virtual object intersects with a remotely-maintained virtual object. ### Claim 8 (depends on 7) The method of claim 7, where the depth-compressed representation includes, for each different portion of the virtual view belonging to the intersection class, a pixel depth value for each of a plurality of pixels in the portion. ### Claim 9 (depends on 8) The method of claim 8, where the pixel depth value is a 16-bit value. ### Claim 10 (depends on 1) The method of claim 1, where the occlusion class for each of the different portions of the virtual view is indicated in the depth-compressed representation using a different unique indicator for each of the plurality of different occlusion classes. ### Claim 11 (depends on 10) The method of claim 10, where each different unique indicator is a 2-bit value. ### Claim 12 (depends on 1) The method of claim 1, where the positioning information for the one or more remotely-maintained virtual objects includes, for each remotely-maintained virtual object, spatial coordinates and bounding box dimensions in a coordinate system of the three-dimensional virtual scene shared between the computing device and the remote device. ### Claim 13 (depends on 12) The method of claim 12, where the coordinate system is world-locked relative to a real-world environment of the remote device. ### Claim 14 (depends on 1) The method of claim 1, where the positioning information for the one or more remotely-maintained virtual objects includes a depth image generated by the remote device that represents the one or more remotely-maintained virtual objects in the three-dimensional virtual scene from the virtual perspective. ### Claim 15 (depends on 1) The method of claim 1, further comprising dividing the virtual view of the three-dimensional virtual scene into a plurality of non-overlapping tiles including a contiguous plurality of pixels, and where each different portion of the virtual view is a different non-overlapping tile. ### Claim 16 (depends on 1) The method of claim 1, further comprising, for each of a plurality of pixels in the depth-compressed representation of the virtual view corresponding to the one or more locally-maintained virtual objects, outputting color information for the plurality of pixels. ### Claim 17 (depends on 1) The method of claim 1, further comprising, for each of a plurality of pixels in the depth-compressed representation of the virtual view corresponding to the one or more locally-maintained virtual objects, outputting lighting information for the plurality of pixels. ### Claim 18 A computing device, comprising: a logic machine; and a storage machine holding instructions executable by the logic machine to: recognize a three-dimensional virtual scene including one or more locally-maintained virtual objects; receive, from a remote device, positioning information for one or more remotely-maintained virtual objects to be included in the three-dimensional virtual scene; for a virtual view of the three-dimensional virtual scene having a virtual perspective, classify different portions of the virtual view as belonging to one of a plurality of different occlusion classes, each of the plurality of different occlusion classes specifying an occlusion relationship between locally-maintained virtual objects and remotely-maintained virtual objects in the virtual view; and output a depth-compressed representation of the virtual view to the remote device, the depth-compressed representation indicating the occlusion class for each of the different portions of the virtual view. ### Claim 19 A method for rendering a display view of a three-dimensional virtual scene, the method comprising: at a computing device, recognizing a three-dimensional virtual scene including one or more locally-maintained virtual objects; sending positioning information for the one or more locally-maintained virtual objects to a remote device, the remote device maintaining one or more remotely-maintained virtual objects to be included in the three-dimensional virtual scene; receiving a depth-compressed representation of the three-dimensional virtual scene from the remote device, the depth-compressed representation indicating, for different portions of a virtual view of the three-dimensional virtual scene having a virtual perspective, which of a plurality of different occlusion classes the different portions belong to, each of the plurality of different occlusion classes specifying an occlusion relationship between locally-maintained virtual objects and remotely-maintained virtual objects in the virtual view; and based on the depth-compressed representation, displaying at the computing device from the virtual perspective, the three-dimensional virtual scene with both the locally-maintained and remotely-maintained virtual objects. ### Claim 20 (depends on 19) The method of claim 19, where the computing device is a head-mounted display device (HMD).