# US 12,112,434 B1 · Mesh updates in an extended reality environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,112,434 B1 - Document ID: US12112434B1 - Status: Granted - Kind: B1 - Application number: 18/089,416 - Filed: 2022-12-27 - Granted: 2024-10-08 - Earliest claimed: 2020-10-16 - Assignees: Splunk Inc. - Applicants: Splunk Inc. - Inventors: Devin Bhushan; Seunghee Han; Caelin Thomas Jackson-King; Jamie Kuppel; Stanislav Yazhenskikh; Jim Jiaming Zhu - Web page: https://patentgrep.com/patent/US12112434B1 ## CPC - G06T 17/205: {Re-meshing} - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06F 3/1454: {involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay} - G06T 2219/024: Multi-user, collaborative environment - G06T 2219/2008: Assembling, disassembling - G09G 3/003: {to produce spatial visual effects} - G09G 2358/00: Arrangements for display data security - G09G 2370/022: Centralised management of display operation, e.g. in a server instead of locally - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract Various implementations or examples set forth a method for scanning a three-dimensional (3D) environment. The method includes generating a 3D representation of the 3D environment that includes one or more 3D meshes. The method also includes determining at least a portion of the 3D environment that falls within a current frame captured by the image sensor. The method further includes generating one or more additional 3D meshes representing the at least a portion of the 3D environment and combining the one or more additional 3D meshes with the one or more 3D meshes into an update to the 3D representation of the 3D environment. ## Claims ### Claim 1 A method for scanning a three-dimensional (3D) environment, comprising: generating a 3D representation of the 3D environment that includes one or more 3D meshes, wherein the one or more 3D meshes comprise a first set of vertices representing 3D points captured of the 3D environment by a depth sensor on a host device; determining at least a portion of the 3D environment that falls within a current frame captured by an image sensor; generating one or more additional 3D meshes representing the at least a portion of the 3D environment, wherein the one or more additional 3D meshes comprise a second set of vertices representing additional 3D points captured by the depth sensor concurrently with the current frame being captured by the image sensor; and combining the one or more additional 3D meshes with the one or more 3D meshes into an update to the 3D representation of the 3D environment. ### Claim 2 (depends on 1) The method of claim 1, wherein the current frame is captured by the image sensor concurrently with or within a threshold time period of generating the one or more additional 3D meshes. ### Claim 3 (depends on 1) The method of claim 1, further comprising transmitting the one or more additional 3D meshes to one or more remote devices that are executing an extended reality application that renders the 3D representation on the one or more remote devices. ### Claim 4 (depends on 1) The method of claim 1, wherein generating the one or more additional 3D meshes comprises iteratively updating the one or more additional 3D meshes with at least one of additional vertices or additional faces associated with the 3D points captured by the depth sensor until a threshold associated with capture of the 3D environment is met. ### Claim 5 (depends on 1) The method of claim 1, wherein determining the at least a portion of the 3D environment that falls within the current frame comprises projecting one or more vertices of the one or more 3D meshes onto one or more pixel locations included in the current frame. ### Claim 6 (depends on 1) The method of claim 1, wherein determining the at least a portion of the 3D environment that falls within the current frame comprises: dividing a 3D mesh included in the one or more 3D meshes into a plurality of sub-meshes; and determining that a sub-mesh included in the plurality of sub-meshes includes at least one vertex that falls within the current frame. ### Claim 7 (depends on 1) The method of claim 1, wherein determining the at least a portion of the 3D environment that falls within the current frame comprises: dividing a 3D mesh included in the one or more 3D meshes into a plurality of sub-meshes; and determining that all vertices of a sub-mesh included in the plurality of sub-meshes fall within the current frame. ### Claim 8 (depends on 1) The method of claim 1, wherein determining the at least a portion of the 3D environment that falls within the current frame comprises determining one or more portions of the one or more 3D meshes that lie within a view frustum associated with the current frame, and wherein generating the one or more additional 3D meshes comprises generating the one or more additional 3D meshes to include or exclude the one or more portions. ### Claim 9 (depends on 1) The method of claim 1, wherein determining the at least a portion of the 3D environment that falls within the current frame comprises: grouping faces in a 3D mesh included in the one or more 3D meshes into a first set of faces that are fully outside a view frustum associated with the current frame, a second set of faces that are fully inside the view frustum, and a third set of faces that intersect the view frustum; and dividing the third set of faces by one or more planes bounding the view frustum to generate a fourth set of faces that are fully outside the view frustum and a fifth set of faces that are fully inside the view frustum. ### Claim 10 (depends on 1) The method of claim 1, wherein determining the at least a portion of the 3D environment that falls within the current frame comprises: determining one or more faces in a 3D mesh included in the one or more 3D meshes that intersect a view frustum associated with the current frame; determining intersection points of the one or more faces with a plane representing a boundary of the view frustum; and forming one or more new faces using the intersection points and vertices of the one or more faces. ### Claim 11 A non-transitory computer readable medium storing instructions that, when executed by a processor, cause the processor to perform the steps of: generating a 3D representation of a 3D environment that includes one or more 3D meshes, wherein the one or more 3D meshes comprise a first set of vertices representing 3D points captured of the 3D environment by a depth sensor on a host device; determining at least a portion of the 3D environment that falls within a current frame captured by an image sensor; generating one or more additional 3D meshes representing the at least a portion of the 3D environment, wherein the one or more additional 3D meshes comprise a second set of vertices representing additional 3D points captured by the depth sensor concurrently with the current frame being captured by the image sensor; and combining the one or more additional 3D meshes with the one or more 3D meshes into an update to the 3D representation of the 3D environment. ### Claim 12 (depends on 11) The non-transitory computer readable medium of claim 11, wherein the current frame is captured by the image sensor concurrently with or within a threshold time period of generating the one or more additional 3D meshes. ### Claim 13 (depends on 11) The non-transitory computer readable medium of claim 11, wherein the steps further comprise transmitting the one or more additional 3D meshes to one or more remote devices that are executing an extended reality application that renders the 3D representation on the one or more remote devices. ### Claim 14 (depends on 11) The non-transitory computer readable medium of claim 11, wherein generating the one or more additional 3D meshes comprises iteratively updating the one or more additional 3D meshes with at least one of additional vertices or additional faces associated with the 3D points captured by the depth sensor until a threshold associated with capture of the 3D environment is met. ### Claim 15 (depends on 11) The non-transitory computer readable medium of claim 11, wherein determining the at least a portion of the 3D environment that falls within the current frame comprises projecting one or more vertices of the one or more 3D meshes onto one or more pixel locations included in the current frame. ### Claim 16 A computing device, comprising: a memory that includes an application; and a processor that is coupled to the memory, and when executing the application, performs the steps of: generating a 3D representation of a 3D environment that includes one or more 3D meshes, wherein the one or more 3D meshes comprise a first set of vertices representing 3D points captured of the 3D environment by a depth sensor on a host device; determining at least a portion of the 3D environment that falls within a current frame captured by an image sensor; generating one or more additional 3D meshes representing the at least a portion of the 3D environment, wherein the one or more additional 3D meshes comprise a second set of vertices representing additional 3D points captured by the depth sensor concurrently with the current frame being captured by the image sensor; and combining the one or more additional 3D meshes with the one or more 3D meshes into an update to the 3D representation of the 3D environment. ### Claim 17 (depends on 16) The computing device of claim 16, wherein the current frame is captured by the image sensor concurrently with or within a threshold time period of generating the one or more additional 3D meshes. ### Claim 18 (depends on 16) The computing device of claim 16, wherein the processor further performs the step of transmitting the one or more additional 3D meshes to one or more remote devices that are executing an extended reality application that renders the 3D representation on the one or more remote devices. ### Claim 19 (depends on 16) The computing device of claim 16, wherein generating the one or more additional 3D meshes comprises iteratively updating the one or more additional 3D meshes with at least one of additional vertices or additional faces associated with the 3D points captured by the depth sensor until a threshold associated with capture of the 3D environment is met. ### Claim 20 (depends on 16) The computing device of claim 16, wherein determining the at least a portion of the 3D environment that falls within the current frame comprises projecting one or more vertices of the one or more 3D meshes onto one or more pixel locations included in the current frame.