# US 2022/0026233 A1 · Navigation using augmented reality

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0026233 A1
- Document ID: US20220026233A1
- Status: Published application
- Kind: A1
- Application number: 17/395,997
- Filed: 2021-08-06
- Published: 2022-01-27
- Earliest claimed: 2018-01-19
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Jae Woo Chang
- Web page: https://patentgrep.com/patent/US20220026233A1

## CPC

- G06V 20/20: in augmented reality scenes
- G01C 21/3632: {Guidance using simplified or iconic instructions, e.g. using arrows (G01C21/365 takes precedence)}
- G01C 21/3638: {including 3D objects and buildings (three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05)}
- G01C 21/3647: {Guidance involving output of stored or live camera images or video streams}
- G01C 21/3658: {Lane guidance}
- G06T 11/00: Two-dimensional [2D] image generation
- G06T 11/65: on geographic maps
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

Embodiments disclose a computer-implemented method for guiding a user along a route, the method including receiving route information from a route server, where the route information includes geographical positions of roads and designated lanes of travel for each of a plurality of segments along the route, and receiving a series of images captured by the camera along the route. The method includes analyzing the series of images to determine a horizon and one or more available lanes of travel along the route, identifying one or more designated lanes from the one or more available lanes based on the analyzing of the series of images, and displaying the series of images along with a navigational layer superimposed over the series of images to the user, the navigational layer comprising a blocked region covering areas of the series of images outside of the one or more designated lanes and below the horizon.

## Claims

### Claim 1

(canceled)

### Claim 2

A computer-implemented method for guiding a user along a route between at least two locations, the method comprising, at a mobile device that has a display and a camera positioned to take images in front of a vehicle:
storing route information, the route information comprising geographical positions of roads and designated lanes of travel for each of a plurality of segments along the route;
receiving a series of images captured by the camera along the route;
analyzing the series of images to determine one or more available lanes of travel along the route;
identifying one or more of the designated lanes from the one or more available lanes based on the analyzing of the series of images and the route information; and
displaying the series of images along with a navigational layer superimposed over the series of images to the user, the navigational layer indicating the one or more designated lanes.

### Claim 3 (depends on 2)

The method of claim 2, wherein the navigational layer comprises an unblocked region revealing the one or more designated lanes.

### Claim 4 (depends on 2)

The method of claim 2, wherein the navigational layer further comprises one or more guidance arrows positioned near a center of at least one image of the series of images.

### Claim 5 (depends on 4)

The method of claim 4, wherein the one or more guidance arrows are positioned point toward the one or more designated lanes when the user is positioned outside of the one or more designated lanes along the route.

### Claim 6 (depends on 4)

The method of claim 4, wherein the one or more guidance arrows are positioned over an unblocked region.

### Claim 7 (depends on 6)

The method of claim 6, wherein the one or more guidance arrows indicate an upcoming turn or that the user should switch lanes.

### Claim 8 (depends on 7)

The method of claim 7, wherein the one or more guidance arrows are animated such that the one or more guidance arrows blink at a frequency inversely proportional to a distance to the upcoming turn.

### Claim 9 (depends on 8)

The method of claim 8, wherein the one or more guidance arrows comprise multiple arrows that blink in a sequential order.

### Claim 10 (depends on 2)

The method of claim 2, wherein the navigational layer comprises a blocked region covering other lanes captured in the series of images that are outside of the one or more designated lanes.

### Claim 11 (depends on 2)

The method of claim 2, wherein the navigational layer further comprises a pop-up window that temporarily appears as the user approaches a turn.

### Claim 12 (depends on 11)

The method of claim 11 wherein the pop-up window appears within a threshold distance to the turn and disappears after the turn has been traversed by the user.

### Claim 13 (depends on 11)

The method of claim 11 wherein the pop-up window displays a top-down view of an area surrounding the user that includes a complete profile of the turn.

### Claim 14

A computer product comprising a non-transitory computer readable medium storing a plurality of instructions that when executed control a mobile device including a display, a camera positioned to take images in front of a vehicle, and one or more processors for guiding a user along a route between at least two locations, the instructions comprising:
storing route information, the route information comprising geographical positions of roads and designated lanes of travel for each of a plurality of segments along the route;
receiving a series of images captured by the camera along the route;
analyzing the series of images to determine one or more available lanes of travel along the route;
identifying one or more of the designated lanes from the one or more available lanes based on the analyzing of the series of images and the route information; and
displaying the series of images along with a navigational layer superimposed over the series of images to the user, the navigational layer indicating the one or more designated lanes.

### Claim 15 (depends on 14)

The computer product of claim 14, wherein the navigational layer comprises an unblocked region revealing the one or more designated lanes.

### Claim 16 (depends on 14)

The computer product of claim 14, wherein the navigational layer further comprises one or more guidance arrows positioned near a center at least one image of the series of images.

### Claim 17 (depends on 16)

The computer product of claim 16, wherein the one or more guidance arrows point toward the one or more designated lanes in an event the user is positioned outside of the one or more designated lanes along the route.

### Claim 18

A mobile device for guiding a user along a route between at least two locations, the mobile device comprising:
a camera positioned to take images in front of a vehicle;
a display; and
one or more processors coupled to the camera and the display, the one or more processors configured to:
receive route information from a route server, the route information comprising geographical positions of roads and designated lanes of travel for each of a plurality of segments along the route;
receive a series of images captured by the camera along the route;
analyze the series of images to determine one or more available lanes of travel along the route;
identify one or more of the designated lanes from the one or more available lanes based on the analyzing of the series of images and the route information; and
display the series of images along with a navigational layer superimposed over the series of images to the user, the navigational layer indicating the one or more designated lanes.

### Claim 19 (depends on 18)

The mobile device of claim 18, wherein the navigational layer comprises an unblocked region revealing the one or more designated lanes.

### Claim 20 (depends on 18)

The mobile device of claim 18, wherein the navigational layer further comprises one or more guidance arrows positioned near a center of at least one image of the series of images.

### Claim 21 (depends on 20)

The mobile device of claim 20, wherein the one or more guidance arrows are positioned point toward the one or more designated lanes in an event the user is positioned outside of the one or more designated lanes along the route.
