# US 12,118,677 B2 · Spatially aware environment interaction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,677 B2 - Document ID: US12118677B2 - Status: Granted - Kind: B2 - Application number: 17/560,146 - Filed: 2021-12-22 - Published: 2022-06-23 - Granted: 2024-10-15 - Earliest claimed: 2020-12-22 - Applicants: ARKH, Inc. - Inventors: Landon Nickerson; Sean Ong; Preston McCauley - Web page: https://patentgrep.com/patent/US12118677B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - H04L 67/52: specially adapted for the location of the user terminal - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 16/955: using information identifiers, e.g. uniform resource locators [URL] - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - H04L 67/131: Protocols for games, networked simulations or virtual reality - H04L 67/10: in which an application is distributed across nodes in the network (software deployment G06F8/60; multiprogramming arrangements G06F9/46) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract In some examples, an apparatus includes a memory storing computer executable instructions for implementing a spatially aware computing scheme; and a processor coupled to the memory and configured to execute the executable instructions. Executing the executable instructions causes the processor to access a spatial augmentation layer (SAL) that includes augmented reality (AR) elements placed into an AR environment for a physical environment proximate to the apparatus, and display the AR elements on a display screen communicatively coupled to the apparatus, wherein at least some of the AR elements are elements placed into the AR environment by a user other than a user of the apparatus. ## Claims ### Claim 1 An apparatus, comprising: a memory storing computer executable instructions for implementing a spatially aware computing scheme; and a processor coupled to the memory and configured to execute the executable instructions to: access a first spatial augmentation layer (SAL) that includes first augmented reality (AR) elements placed into an AR environment for a physical environment proximate to the apparatus, the first SAL including permissions permitting at least a first user of the apparatus and a second user other than the first user of the apparatus to manipulate the first SAL, wherein the second user is not a user of the apparatus; display the first AR elements on a display screen communicatively coupled to the apparatus, wherein at least some of the first AR elements are elements placed into the AR environment by the second user other than the first user of the apparatus; access a second SAL that includes second AR elements placed into the AR environment, the second SAL including permissions permitting the first user of the apparatus to manipulate the second SAL; and display the second AR elements on the display screen concurrently with the first AR elements, wherein the second AR elements are elements placed into the AR environment by the first user of the apparatus and are inaccessible to the second user for viewing or manipulating, wherein at least one of the first or second AR elements is a volumetric three-dimensional (3D) spatial data point placed in the AR environment to encompass an interactive object appearing in the physical environment, wherein the volumetric 3D spatial data point enables control of the interactive object from within the first or second SAL. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the processor is further configured to execute the executable instructions to receive a control input from the first user to cause the processor to modify at least one of the first AR elements placed into the AR environment by the second user. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the processor is further configured to execute the executable instructions to receive a control input from the first user as an interaction with at least one of the first or second AR elements, wherein the at least one of the first or second AR elements is associated with a uniform resource locator that causes the processor to display an Internet browser with Internet-accessible data defined by the uniform resource locator to the first user. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the processor is further configured to execute the executable instructions to receive a control input from the first user to place a user-placed AR element in the first or second SAL. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the second SAL is a single-user SAL stored in the memory and accessible to the first user. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the first SAL is a multi-user shared SAL stored in the memory or stored remotely to the apparatus, and wherein the multi-user shared SAL is accessible to multiple users given permission to access the multi-user shared SAL. ### Claim 7 (depends on 6) The apparatus of claim 6, wherein AR elements placed into the multi-user shared SAL by one of the multiple users are viewable and editable by other of the multiple users. ### Claim 8 (depends on 1) The apparatus of claim 1, wherein the processor is further configured to execute the executable instructions to: access a third SAL that includes third AR elements placed into the AR environment, the third SAL including permissions permitting any of multiple users, including at least the first and second users, to manipulate the third SAL; and display the third AR elements on the display screen concurrently with the first AR elements and the second AR elements, wherein the third SAL is a global SAL stored remotely to the apparatus. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein the third AR elements placed into the global SAL by one of the multiple users are viewable and editable by other of the multiple users. ### Claim 10 A system, comprising: a spatial computing hub; and a spatial computing device configured to: access a first spatial augmentation layer (SAL) that includes first augmented reality (AR) elements placed into an AR environment for a first physical environment proximate to the spatial computing device, the first SAL including permissions permitting at least a first user of the spatial computing device and a second user other than the first user to manipulate the first SAL, wherein the second user is not a user of the spatial computing device; display the first AR elements on a display screen communicatively coupled to the spatial computing device, wherein at least some of the first AR elements are elements placed into the AR environment by the second user other than the first user of the spatial computing device; access a second SAL that includes second AR elements placed into the AR environment, the second SAL including permissions permitting the first user of an apparatus to manipulate the second SAL; and display the second AR elements on the display screen concurrently with the first AR elements, wherein the second AR elements are elements placed into the AR environment by the first user and are inaccessible to the second user for viewing or manipulating, wherein at least one of the first or second AR elements is a volumetric three-dimensional (3D) spatial data point placed in the AR environment to encompass an interactive object appearing in the first physical environment, wherein the volumetric 3D spatial data point enables control of the interactive object from within the first or second SAL. ### Claim 11 (depends on 10) The system of claim 10, wherein the second SAL is stored locally on the spatial computing device and includes AR elements, including at least the second AR elements, placed into the AR environment by the first user. ### Claim 12 (depends on 11) The system of claim 11, wherein the second SAL is a single-user SAL available for viewing only to the first user. ### Claim 13 (depends on 10) The system of claim 10, wherein the first SAL is stored on the spatial computing hub and includes AR elements, including at least the first AR elements, placed into the AR environment by multiple users, including the second user, other than the first user. ### Claim 14 (depends on 10) The system of claim 10, wherein the first SAL is a multi-user SAL available for viewing by multiple users in a second physical environment proximate to the spatial computing hub. ### Claim 15 (depends on 10) The system of claim 10, wherein the first SAL is stored remotely from both the spatial computing device and the spatial computing hub and includes AR elements that are interactive and replace elements of the first physical environment. ### Claim 16 An apparatus, comprising: a memory storing computer executable instructions for implementing a spatially aware computing scheme; and a processor coupled to the memory and configured to execute the executable instructions to: receive an indication of first augmented reality (AR) elements placed in a first spatial augmentation layer (SAL) of an AR environment by a first user of a first spatial computing device for a physical environment proximate to the first spatial computing device, the first SAL including permissions permitting the first user to manipulate the first SAL; store the first SAL; receive an indication of second AR elements placed in a second SAL of the AR environment by a second user other than the first user for the physical environment proximate to the first spatial computing device, the second SAL including permissions permitting at least the first user and the second user to manipulate the second SAL, wherein the second user is not a user of the first spatial computing device; and provide the first SAL and the second SAL to the first spatial computing device responsive to the first spatial computing device initiating an AR session in the physical environment, wherein at least one of the first or second AR elements is a volumetric three-dimensional (3D) spatial data point placed in the AR environment to encompass an interactive object appearing in the physical environment, wherein the volumetric 3D spatial data point enables control of the interactive object from within the first or second SAL. ### Claim 17 (depends on 16) The apparatus of claim 16, wherein the first SAL is a single-user SAL available only to the first user of the first spatial computing device. ### Claim 18 (depends on 16) The apparatus of claim 16, wherein the second SAL is a multi-user SAL that includes AR elements, including at least the second AR elements, placed into the multi-user SAL by multiple users of multiple spatial computing devices, including at least the first user and the second user, in the physical environment proximate to the first spatial computing device. ### Claim 19 (depends on 16) The apparatus of claim 16, wherein the second SAL is a global SAL available to all spatial computing device users who access the global SAL, and wherein the second AR elements include at least some elements not placed in the AR environment by the first user and that replace the at least some of the first AR elements in the physical environment. ### Claim 20 (depends on 19) The apparatus of claim 19, wherein at least some of the second AR elements are interactive AR elements that grant rewards to the first user of the first spatial computing device responsive to interaction by the first user with the interactive AR elements.