# US 2022/0279147 A1 · Method of Tracking a Mobile Device and Method of Generating a Geometrical Model of a Real Environment Using a Camera of a Mobile Device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0279147 A1
- Document ID: US20220279147A1
- Status: Published application
- Kind: A1
- Application number: 17/663,533
- Filed: 2022-05-16
- Published: 2022-09-01
- Earliest claimed: 2013-12-19
- Assignees: Apple Inc.
- Applicants: Apple Inc.
- Inventors: Peter Meier; Sebastian Lieberknecht
- Web page: https://patentgrep.com/patent/US20220279147A1

## CPC

- H04N 7/185: {from a mobile camera, e.g. for remote control}
- G06T 7/251: {involving models}
- G06T 7/579: from motion
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
- G06T 2207/10016: Video; Image sequence
- G06T 2207/30244: Camera pose
- G06T 2207/30264: Parking
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method of tracking a mobile device comprising at least one camera in a real environment comprises the steps of receiving image information associated with at least one image captured by the at least one camera, generating a first geometrical model of at least part of the real environment based on environmental data or mobile system state data acquired in an acquisition process by at least one sensor of a mobile system, which is different from the mobile device, and performing a tracking process based on the image information associated with the at least one image and at least partially according to the first geometrical model, wherein the tracking process determines at least one parameter of a pose of the mobile device relative to the real environment. The invention is also related to a method of generating a geometrical model of at least part of a real environment using image information from at least one camera of a mobile device

## Claims

### Claim 1

A method comprising:
determining that a vehicle has reached a first threshold distance from a predefined destination within an environment while the vehicle is moving toward the predefined destination, wherein a mobile device is located within the vehicle while the vehicle is moving;
in accordance with determining that the vehicle has reached the first threshold distance from the predefined destination:
acquiring environmental data from one or more sensors of the vehicle for the environment, and
generating a reference geometric model of the environment based on the acquired environmental data; and
determining that a trigger condition is satisfied based on a state of the vehicle; and
in accordance with detecting that the trigger condition is satisfied, transmitting the reference geometric model of the environment to the mobile device.

### Claim 2 (depends on 1)

The method of claim 1, wherein the reference geometrical model is further generated based on vehicle status information associated with the acquired environmental data.

### Claim 3 (depends on 2)

The method of claim 2, wherein the vehicle status information comprises at least one of a speed of the vehicle or a steering direction of the vehicle.

### Claim 4 (depends on 1)

The method of claim 1, wherein the one or more sensors of the vehicle comprises at least two vehicle cameras, with a known spatial relationship between the at least two vehicle cameras.

### Claim 5 (depends on 1)

The method of claim 1, wherein the trigger condition comprises a predetermined vehicle state.

### Claim 6 (depends on 1)

The method of claim 1, wherein the trigger condition comprises a determination that the vehicle has reached a second threshold distance of the predetermined destination.

### Claim 7 (depends on 1)

The method of claim 1, wherein the reference geometrical model is generated using a motion of the vehicle that is more constrained that motion of the mobile device.

### Claim 8

A non-transitory computer readable medium comprising computer readable code executable by one or more processors to:
determine that a vehicle has reached a first threshold distance from a predefined destination within an environment while the vehicle is moving toward the predefined destination, wherein a mobile device is located within the vehicle while the vehicle is moving;
in accordance with determining that the vehicle has reached the first threshold distance from the predefined destination:
acquire environmental data from one or more sensors of the vehicle for the environment, and
generate a reference geometric model of the environment based on the acquired environmental data; and
determine that a trigger condition is satisfied based on a state of the vehicle; and
in accordance with detecting that the trigger condition is satisfied, transmit the reference geometric model of the environment to the mobile device.

### Claim 9 (depends on 8)

The non-transitory computer readable medium of claim 8, wherein the reference geometrical model is further generated based on vehicle status information associated with the acquired environmental data.

### Claim 10 (depends on 9)

The non-transitory computer readable medium of claim 9, wherein the vehicle status information comprises at least one of a speed of the vehicle or a steering direction of the vehicle.

### Claim 11 (depends on 8)

The non-transitory computer readable medium of claim 8, wherein the one or more sensors of the vehicle comprises at least two vehicle cameras, with a known spatial relationship between the at least two vehicle cameras.

### Claim 12 (depends on 8)

The non-transitory computer readable medium of claim 8, wherein the trigger condition comprises a predetermined vehicle state.

### Claim 13 (depends on 8)

The non-transitory computer readable medium of claim 8, wherein the trigger condition comprises a determination that the vehicle has reached a second threshold distance of the predetermined destination.

### Claim 14 (depends on 8)

The non-transitory computer readable medium of claim 8, wherein the reference geometrical model is generated using a motion of the vehicle that is more constrained that motion of the mobile device.

### Claim 15

A system comprising:
one or more processors; and
one or more computer readable media comprising computer readable code executable by the one or more processors to:
determine that a vehicle has reached a first threshold distance from a predefined destination within an environment while the vehicle is moving toward the predefined destination, wherein a mobile device is located within the vehicle while the vehicle is moving;
in accordance with determining that the vehicle has reached the first threshold distance from the predefined destination:
acquire environmental data from one or more sensors of the vehicle for the environment, and
generate a reference geometric model of the environment based on the acquired environmental data; and
determine that a trigger condition is satisfied based on a state of the vehicle; and
in accordance with detecting that the trigger condition is satisfied, transmit the reference geometric model of the environment to the mobile device.

### Claim 16 (depends on 15)

The system of claim 15, wherein the reference geometrical model is further generated based on vehicle status information associated with the acquired environmental data.

### Claim 17 (depends on 16)

The system of claim 16, wherein the vehicle status information comprises at least one of a speed of the vehicle or a steering direction of the vehicle.

### Claim 18 (depends on 15)

The system of claim 15, wherein the one or more sensors of the vehicle comprises at least two vehicle cameras, with a known spatial relationship between the at least two vehicle cameras.

### Claim 19 (depends on 15)

The system of claim 15, wherein the trigger condition comprises a predetermined vehicle state.

### Claim 20 (depends on 15)

The system of claim 15, wherein the trigger condition comprises a determination that the vehicle has reached a second threshold distance of the predetermined destination.
