# US 12,008,686 B2 · Augmented reality map curation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,686 B2 - Document ID: US12008686B2 - Status: Granted - Kind: B2 - Application number: 18/057,615 - Filed: 2022-11-21 - Published: 2023-05-25 - Granted: 2024-06-11 - Earliest claimed: 2020-01-27 - Assignees: Magic Leap, Inc. - Applicants: Magic Leap, Inc. - Inventors: Griffith Buckley Hazen; Amy DeDonato; Ali Shahrokni; Ben Weisbih; Vinayram Balakumar - Web page: https://patentgrep.com/patent/US12008686B2 ## CPC - G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) - G01C 21/383: {Indoor data} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0346: with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors - G06T 3/60: Rotation of whole images or parts thereof - G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis - G06V 20/20: in augmented reality scenes - G06V 40/193: {Preprocessing; Feature extraction} - G09B 29/106: {using electronic means} - G06T 2200/24: involving graphical user interfaces [GUIs] - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) ## Abstract An augmented reality device may communicate with a map server via an API interface to provide mapping data that may be implemented into a canonical map, and may also receive map data from the map server. A visualization of map quality, including quality indicators for multiple cells of the environment, may be provided to the user as an overlay to the current real-world environment seen through the AR device. These visualizations may include, for example, a map quality minimap and/or a map quality overlay. The visualizations provide guidance to the user that allows more efficient updates to the map, thereby improving map quality and localization of users into the map. ## Claims ### Claim 1 A computerized method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising: determining a cell in which a wearable headset is positioned; determining a cell quality score associated with the determined cell usable to localize a user into the determined cell; determining a cell saturation indicator indicative of whether the user has been positioned within the determined cell for at least a threshold time period; determining a cell score indicative of the cell quality score and the cell saturation-score indicator; and updating a user interface viewable in the wearable headset to indicate the cell score for the determined cell. ### Claim 2 (depends on 1) The computerized method of claim 1, wherein the cell quality score and the cell saturation indicator are determined by a server remote to the wearable headset and are transmitted to the wearable headset via an application programming interface (API) communication channel. ### Claim 3 (depends on 2) The computerized method of claim 2, wherein the wearable headset initiates the API communication channel by transmitting credentials for directly accessing map data from the remote server. ### Claim 4 (depends on 1) The computerized method of claim 1, wherein the cell quality score is between 0 and 1, wherein 0 indicates a lowest cell quality and 1 indicates a highest cell quality. ### Claim 5 (depends on 1) The computerized method of claim 1, further comprising: in response to determining that the cell saturation indicator is positive, summing-the cell quality score and 0.5 to calculate the cell score, with a maximum cell score of 1. ### Claim 6 (depends on 1) The computerized method of claim 1, further comprising: in response to determining that the cell saturation indicator is negative; setting the cell score equal to the cell quality score. ### Claim 7 (depends on 1) The computerized method of claim 1, wherein the user interface includes a minimap of a portion of a plurality of cells and their corresponding cell scores. ### Claim 8 (depends on 7) The computerized method of claim 7, wherein the minimap is associated with a position of a user input device such that the minimap moves in conjunction with the user input device. ### Claim 9 (depends on 8) The computerized method of claim 8, wherein the minimap is displayed in front of the user input device. ### Claim 10 (depends on 7) The computerized method of claim 7, further comprising: in response to movement of the user, rotating the minimap to maintain orientation of the user. ### Claim 11 (depends on 1) The computerized method of claim 1, wherein the user interface includes a map quality overlay with indicators of cell scores overlaying corresponding portions of a real world environment. ### Claim 12 (depends on 1) The computerized method of claim 1, wherein the user interface includes at least one cell quality subscore determined based on images obtained from a particular viewing directions. ### Claim 13 (depends on 12) The computerized method of claim 12, wherein the at least one cell quality subscore includes a north viewing direction subscore, a south viewing direction subscore, a west viewing direction subscore, and an east viewing direction subscore. ### Claim 14 (depends on 13) The computerized method of claim 13, wherein the cell quality subscores are indicated in the determined cell. ### Claim 15 (depends on 14) The computerized method of claim 14, wherein the cell quality subscores are indicated as colored areas around an indicator of the cell score. ### Claim 16 (depends on 7) The computerized method of claim 7, wherein the cell scores are indicated in the user interface with colors, wherein lower cell scores are a first color and higher cell scores are a second color. ### Claim 17 (depends on 16) The computerized method of claim 16, wherein the first color is red and the second color is green. ### Claim 18 (depends on 7) The computerized method of claim 7, wherein the user interface indicates the determined cell with a user icon. ### Claim 19 (depends on 1) The computerized method of claim 1, wherein the cell is one of a plurality of cells in a grid pattern. ### Claim 20 A computerized method, performed by a computing system having one or more hardware computer processors and one or more non-transitory computer readable storage device storing software instructions executable by the computing system to perform the computerized method comprising: displaying via a wearable headset a minimap indicating quality of canonical map data at each of a plurality of cells of a canonical map, the quality of the canonical map data at each of the plurality of cells characterized by a cell quality score; determining a cell in which the wearable headset is positioned; determining a cell saturation indicator indicative of whether the wearable headset has been positioned within the determined cell for at least a threshold time period; obtaining, via one or more sensors of the wearable headset, images of an environment as a user moves about the environment; determining updates to the quality of the canonical map data based on the obtained images of the environment; updating a cell score indicative of the cell quality score and the cell saturation indicator; and updating the minimap to indicate the update to the cell score.