# US 12,602,948 B2 · Generating computer augmented maps from physical maps

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,948 B2
- Document ID: US12602948B2
- Status: Granted
- Kind: B2
- Application number: 17/793,951
- Filed: 2020-01-30
- Published: 2023-02-16
- Granted: 2026-04-14
- Earliest claimed: 2020-01-30
- Assignees: Telefonaktiebolaget LM Ericsson (publ)
- Applicants: Telefonaktiebolaget LM Ericsson (publ)
- Inventors: Xiaoping Li; Tommy Arngren; Andreas Kristensson
- Web page: https://patentgrep.com/patent/US12602948B2

## CPC

- G06V 30/422: Technical drawings; Geographical maps
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 10/17: {using hand-held instruments}
- G06V 10/235: {based on user input or interaction}
- G06V 20/20: in augmented reality scenes
- G06V 40/1312: {direct reading, e.g. contactless acquisition}
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G10L 15/00: Speech recognition (G10L17/00 takes precedence)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A method by a computing device obtains a digital image of a physical map, identifies features in the digital image, and obtains map augmentation information based on the identified features. The method then generates an augmented map based on the map augmentation information, and provides the augmented map for display. Related mobile devices and computer program products are disclosed.

## Claims

### Claim 1

A method by a computing device, the method comprising:
obtaining a digital image of a physical map;
identifying features in the digital image;
obtaining map augmentation information based on the identified features;
generating an augmented map based on the map augmentation information; and
providing the augmented map for display;
wherein the method further comprises:
determining a pose for an augmented reality (AR) image based on a pose of an AR headset and a pose of the physical map,
wherein the generating of the augmented map comprises generating the AR image based on the augmentation information and the determined pose; and
wherein the providing of the augmented map comprises providing the AR image to a see-through display screen of the AR headset for display with the determined pose on a see-through display screen for viewing by a user as an overlaid AR image on the physical map.

### Claim 2 (depends on 1)

The method of claim 1, wherein obtaining a digital image of the physical map, comprises receiving a digital picture of the physical map from a camera of a mobile device, a camera communicatively connected to the mobile device, a scanner device, and/or a digital picture repository communicatively connected to the mobile device.

### Claim 3 (depends on 1)

The method of claim 1, further comprising:
determining a pose for an augmented reality (AR) image based on a pose of a projector device and a pose of the physical map,
wherein the generating of the augmented map comprises generating the AR image based on the map augmentation information and the determined pose, and
wherein the providing of the augmented map comprises providing the AR image to the projector device for projection with the determined pose onto the physical map.

### Claim 4 (depends on 1)

The method of claim 1, further comprising:
receiving, from an on-line map application, points-of-interest (POIs) within a digital map that are related to a defined location; and
receiving user selections of POIs of the digital map to be added to the augmented map,
wherein the obtaining of the map augmentation information comprises generating the map augmentation information based on a combination of the selected POIs and the features that have been identified in the digital image, and
wherein the generating of the augmented map comprises combining the map augmentation information and the digital image of the physical map.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
receiving, from an on-line map application, points-of-interest (POIs) within a digital map that are related to a defined location;
receiving user selections of POIs of the digital map to be added to the augmented map; and
saving the digital map and selected POIs as a combined POI-map image in local memory of the computing device,
wherein the obtaining of the map augmentation information comprises comparing features in the combined POI-map image with the features that have been identified in the digital image, and
wherein the generating of the augmented map comprises combining the map augmentation information and the digital image of the physical map.

### Claim 6 (depends on 1)

The method of claim 1, wherein:
the identifying of features in the digital image comprises identifying a scale and orientation of features in the digital image of the physical map; and
the generating of the augmented map based on the map augmentation information comprises scaling and orienting how the map augmentation information is to be displayed in the augmented map based on the identified scale and orientation of features in the digital image of the physical map.

### Claim 7 (depends on 1)

The method of claim 1, wherein:
the identifying of features in the digital image comprises identifying a language of features in the digital image of the physical map; and
the obtaining of the map augmentation information comprises determining a default user language defined for a user interface, and responsive to the identified language being different than the default user language performing translation of the features from the identified language to the default user language.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
adapting alignment, pan, and/or zoom of what is displayed in the augmented map based on command received from a user interface.

### Claim 9 (depends on 8)

The method of claim 8, wherein the adapting alignment, pan, and/or zoom of what is displayed in the augmented map, comprises:
responsive to determining that the command corresponds to a zoom-in command,
obtaining another digital image for a smaller region of the physical map defined based on a zoom-in level indicated by the zoom-in command, and
adapting which of the map augmentation information is used to generate the augmented map,
wherein the providing of the augmented map comprises providing the augmented map to a projector device for projection onto the physical map and/or to an augmented reality (AR) headset for display on a see-through display screen viewable by a user as an AR image on the physical map.

### Claim 10 (depends on 1)

The method of claim 1, further comprising:
responsive to sensing a user gesture performed with respect to a particular one of the identified features in the digital map, adapting which of the map augmentation information associated with the particular one of the identified features in the digital map is used to generate the augmented map.

### Claim 11 (depends on 1)

The method of claim 1, further comprising:
adapting alignment, pan, and/or zoom of what is displayed in the augmented map based on sensing via a camera that a user gesture is made relative to a portion of the physical map.

### Claim 12 (depends on 11)

The method of claim 11, wherein the adapting alignment, pan, and/or zoom of what is displayed in the augmented map, comprises:
identifying a command that is associated with the user gesture; and
performing an action adapting which of the map augmentation information is used to generate the augmented map,
wherein the providing of the augmented map comprises providing the augmented map to a projector device for projection onto the physical map and/or to an augmented reality (AR) headset for display on a see-through display screen viewable by a user as an AR image overlaid on the physical map.

### Claim 13 (depends on 1)

The method of claim 1, wherein the obtaining of the map augmentation information based on the identified features comprises:
performing natural language processing on digitized sound from a microphone to identify keywords associated with the digital image of the physical map;
identifying points-of-interest (POIs) based on the identified keywords;
determining locations of the POIs relative to the digital image of the physical map; and
generating the map augmentation information based on the POIs that are identified and the determined locations of the POIs relative to the digital image of the physical map.

### Claim 14 (depends on 1)

The method of claim 1, wherein the computing device comprises a mobile device that provides the augmented map to a connected display screen.

### Claim 15 (depends on 1)

The method of claim 1, wherein the computing device comprises a map server that provides the augmented map to a mobile device for display.

### Claim 16

A computing device comprising:
at least one processor;
at least one memory connected to the at least one processor and storing program code that is executed by the at least one processor to perform operations to:
obtain a digital image of a physical map;
identify features in the digital image;
obtain map augmentation information based on the identified features;
determine a pose for an augmented reality (AR) image based on a pose of an AR headset and a pose of the physical map;
generate an augmented map based on the map augmentation information and the determined pose; and
provide the augmented map for display, wherein providing the augmented map for display comprises providing the AR image to a see-through display screen of the AR headset for display with the determined pose on a see-through display screen for viewing by a user as an overlaid AR image on the physical map.

### Claim 17 (depends on 16)

The computing device of claim 16, wherein the computing device comprises a mobile device, and the at least one processor is configured to provide the augmented map to at least one of a display screen component of the mobile device, a projector component of the mobile device, and a see-through display screen of an augmented reality headset communicatively connected to the mobile device.

### Claim 18 (depends on 16)

The computing device of claim 16, wherein the computing device comprises a map server, and the at least one processor is configured to provide the augmented map to at least one of a display screen component of a mobile device communicatively connected to the map server, a projector component of the mobile device, and a see-through display screen of an augmented reality headset communicatively connected to at least one of the map server and the mobile device.

### Claim 19

A non-transitory computer readable medium storing program code executable by at least one processor of a computing device, the program code being configured to cause the at least one processor to perform operations comprising:
obtaining a digital image of a physical map;
identifying features in the digital image;
obtaining map augmentation information based on the identified features;
determining a pose for an augmented reality (AR) image based on a pose of an AR headset and a pose of the physical map;
generating an augmented map based on the map augmentation information and the determined pose; and
providing the augmented map for display, wherein providing the augmented map for display comprises providing the AR image to a see-through display screen of the AR headset for display with the determined pose on a see-through display screen for viewing by a user as an overlaid AR image on the physical map.