# US 12,008,806 B2 · Methods and systems to allow three-dimensional map sharing between heterogeneous computing systems, cross-localization, and sharing content in three-dimensional space > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,806 B2 - Document ID: US12008806B2 - Status: Granted - Kind: B2 - Application number: 17/406,399 - Filed: 2021-08-19 - Published: 2023-02-23 - Granted: 2024-06-11 - Earliest claimed: 2021-08-19 - Assignees: Meta Platforms Technologies, LLC - Applicants: Meta Platforms Technologies, LLC - Inventors: Evyatar Bluzer; Elad Joseph; Ezri Joseph Sonn; Shanghsuan Tsai - Web page: https://patentgrep.com/patent/US12008806B2 ## CPC - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06V 20/20: in augmented reality scenes - G06V 10/40: Extraction of image or video features - G06V 10/46: Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features (colour feature extraction G06V10/56) - G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method includes a computing system associated with a first artificial-reality device accessing sensor data of a real environment, generating a first plurality of feature descriptors for the real environment, and generating a second plurality of feature descriptors for the real environment based on the first plurality of feature descriptors, wherein the second plurality of feature descriptors have lower detail than the first plurality of feature descriptors. The computing system may further cause the first plurality of feature descriptors to be transmitted using a wireless connection, wherein the transmitted first plurality of feature descriptors are configured to be converted into a third plurality of feature descriptors different from the second plurality of feature descriptors. The third plurality of feature descriptors are configured to be used by a second artificial-reality device for tracking the real environment. The computing system tracks the real environment using the second plurality of feature descriptors. ## Claims ### Claim 1 A method comprising, by a computing system associated with a first device, wherein the first device is an artificial-reality device: accessing sensor data of a real environment captured by the first device; generating, based on the sensor data, a first plurality of feature descriptors for the real environment using a machine-learning model; generating a second plurality of feature descriptors for the real environment based on the first plurality of feature descriptors, wherein the second plurality of feature descriptors have lower detail than the first plurality of feature descriptors; transmitting the first plurality of feature descriptors using a wireless connection, wherein the transmitted first plurality of feature descriptors are configured to be converted into a third plurality of feature descriptors different from the second plurality of feature descriptors, the third plurality of feature descriptors being configured to be used by a second artificial-reality device for tracking the real environment; and tracking the real environment using the second plurality of feature descriptors. ### Claim 2 (depends on 1) The method of claim 1, wherein the sensor data is an image of the real environment captured by a camera worn by a user of the first artificial-reality device. ### Claim 3 (depends on 1) The method of claim 1, wherein tracking the real environment using the second plurality of feature descriptors further comprises localizing the first artificial-reality device in the real environment. ### Claim 4 (depends on 1) The method of claim 1, wherein the first artificial-reality device is a high-performant device relative to the second artificial-reality device, wherein the second plurality of feature descriptors have higher detail relative to the third plurality of feature descriptors. ### Claim 5 (depends on 1) The method of claim 1, wherein the first artificial-reality device is a less-performant device relative to the second artificial-reality device, wherein the second plurality of feature descriptors have less detail relative to the third plurality of feature descriptors. ### Claim 6 (depends on 1) The method of claim 1, wherein the first plurality of feature descriptors are transmitted to a server configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors and transmit the third plurality of feature descriptors to the second artificial reality device. ### Claim 7 (depends on 1) The method of claim 1, wherein the first plurality of feature descriptors are transmitted to the second artificial reality device, the second artificial reality device being configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors. ### Claim 8 (depends on 1) The method of claim 1, wherein the second plurality of feature descriptors are generated by a second machine learning model stored on the first artificial reality device. ### Claim 9 (depends on 1) The method of claim 1, wherein the first plurality of feature descriptors are lossless feature descriptors. ### Claim 10 One or more computer-readable non-transitory storage media embodying software that is operable when executed to: access sensor data of a real environment captured by the first device; generate, based on the sensor data, a first plurality of feature descriptors for the real environment using a machine-learning model; generate a second plurality of feature descriptors for the real environment based on the first plurality of feature descriptors, wherein the second plurality of feature descriptors have lower detail than the first plurality of feature descriptors; transmit the first plurality of feature descriptors using a wireless connection, wherein the transmitted first plurality of feature descriptors are configured to be converted into a third plurality of feature descriptors different from the second plurality of feature descriptors, the third plurality of feature descriptors being configured to be used by a second artificial-reality device for tracking the real environment; and track the real environment using the second plurality of feature descriptors. ### Claim 11 (depends on 10) The media of claim 10, wherein the sensor data is an image of the real environment captured by a camera worn by a user of the first artificial-reality device. ### Claim 12 (depends on 10) The media of claim 10, wherein the first plurality of feature descriptors are transmitted to a server configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors and transmit the third plurality of feature descriptors to the second artificial reality device. ### Claim 13 (depends on 10) The media of claim 10, wherein the first plurality of feature descriptors are transmitted to the second artificial reality device, the second artificial reality device being configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors. ### Claim 14 (depends on 10) The media of claim 10, wherein the first artificial-reality device is a high-performant device relative to the second artificial-reality device, wherein the second plurality of feature descriptors have higher detail relative to the third plurality of feature descriptors. ### Claim 15 (depends on 10) The media of claim 10, wherein the second plurality of feature descriptors are generated by a second machine learning model stored on the first artificial reality device. ### Claim 16 A system comprising: one or more processors; and one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to: access sensor data of a real environment captured by the first device; generate, based on the sensor data, a first plurality of feature descriptors for the real environment using a machine-learning model; generate a second plurality of feature descriptors for the real environment based on the first plurality of feature descriptors, wherein the second plurality of feature descriptors have lower detail than the first plurality of feature descriptors; transmit the first plurality of feature descriptors using a wireless connection, wherein the transmitted first plurality of feature descriptors are configured to be converted into a third plurality of feature descriptors different from the second plurality of feature descriptors, the third plurality of feature descriptors being configured to be used by a second artificial-reality device for tracking the real environment; and track the real environment using the second plurality of feature descriptors. ### Claim 17 (depends on 16) The system of claim 16, wherein the sensor data is an image of the real environment captured by a camera worn by a user of the first artificial-reality device. ### Claim 18 (depends on 16) The system of claim 16, wherein the first plurality of feature descriptors are transmitted to a server configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors and transmit the third plurality of feature descriptors to the second artificial reality device. ### Claim 19 (depends on 16) The system of claim 16, wherein the first plurality of feature descriptors are transmitted to the second artificial reality device, the second artificial reality device being configured to convert the first plurality of feature descriptors into the third plurality of feature descriptors. ### Claim 20 (depends on 16) The system of claim 16, wherein the first artificial-reality device is a high-performant device relative to the second artificial-reality device, wherein the second plurality of feature descriptors have higher detail relative to the third plurality of feature descriptors.