# US 2023/0410363 A1 · Camera pose estimation techniques

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0410363 A1
- Document ID: US20230410363A1
- Status: Published application
- Kind: A1
- Application number: 18/461,625
- Filed: 2023-09-06
- Published: 2023-12-21
- Earliest claimed: 2020-04-09
- Assignees: Tusimple, Inc.
- Applicants: TuSimple, Inc.
- Inventors: Chenzhe QIAN; Yi LUO; Ke XU; Mingdong WANG; Xiaodi HOU
- Web page: https://patentgrep.com/patent/US20230410363A1

## CPC

- G06T 7/74: {involving reference images or patches}
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G06T 2207/30244: Camera pose
- G06T 2207/30256: Lane; Road marking
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Techniques are described for estimating pose of a camera located on a vehicle. An exemplary method of estimating camera pose includes obtaining, from a camera located on a vehicle, an image including a lane marker on a road on which the vehicle is driven, and estimating a pose of the camera such that the pose of the camera provides a best match according to a criterion between a first position of the lane marker determined from the image and a second position of the lane marker determined from a stored map of the road.

## Claims

### Claim 1

A method of estimating camera pose, comprising:
receiving, by a computer located in a vehicle, an image from a camera located on the vehicle, wherein the image comprises a lane marker on a road;
determining pixel locations of a plurality of corners of the lane marker;
obtaining, from a database, three-dimensional (3D) coordinates of the plurality of corners of the lane marker; and
determining a pose of the camera by minimizing a distance from 3D world coordinates of at least one corner from the plurality of corners of the lane marker and at least one pixel location of the at least one corner of the lane marker.

### Claim 2 (depends on 1)

The method of claim 1, wherein the 3D coordinates of the plurality of corners of the lane marker are obtained by:
obtaining, from the database and based on a location of the vehicle, 3D coordinates of corners of each lane marker in a first set of lane markers;
obtaining, from the first set of lane markers and based on a direction in which the vehicle is driven, a second set of lane markers that are located within a first pre-determined distance from the vehicle;
obtaining, from the second set of lane markers, a third set of lane markers that are located within a pre-determined field of view of the camera;
obtaining a fourth set of lane markers by removing one or more lane markers from the third set of lane markers located past a second pre-determined distance from the location of the vehicle; and
obtaining a fifth set of lane markers by removing one or more lane markers obstructed by one or more objects on the road from the fourth set of lane markers, wherein the fifth set of lane markers includes the lane marker.

### Claim 3 (depends on 2)

The method of claim 2, wherein the third set of lane markers are obtained by removing at least one lane marker that is occluded by landscapes.

### Claim 4 (depends on 2)

The method of claim 2, wherein the direction in which the vehicle is driven is obtained from an inertial measurement unit (IMU) sensor located on the vehicle.

### Claim 5 (depends on 1)

The method of claim 1, further comprising:
generating, from the image, a second image that includes a plurality of pixels, wherein each of the plurality of pixels has a value that is directly related to another distance between a pixel and a corner of the lane marker.

### Claim 6 (depends on 1)

The method of claim 1, wherein the lane marker has a rectangular shape.

### Claim 7 (depends on 1)

The method of claim 1, wherein the pose includes values for an orientation and a position of the camera.

### Claim 8 (depends on 1)

The method of claim 1, wherein the pose of the camera is determined as the vehicle is driven on the road.

### Claim 9

A computer, comprising:
a processor configured to:
receive an image from a camera located on a vehicle, wherein the image comprises a lane marker on a road, wherein the computer is located in the vehicle;
determine pixel locations of a plurality of corners of the lane marker;
obtain, from a database, three-dimensional (3D) coordinates of the plurality of corners of the lane marker; and
determine a pose of the camera by minimizing a distance from 3D world coordinates of at least one corner from the plurality of corners of the lane marker and at least one pixel location of the at least one corner of the lane marker.

### Claim 10 (depends on 9)

The computer of claim 9, wherein the 3D coordinates of the plurality of corners of the lane marker are obtained by the processor configured to:
obtain, from the database and based on a location of the vehicle, 3D coordinates of corners of each lane marker in a first set of lane markers;
obtain, from the first set of lane markers and based on a direction in which the vehicle is driven, a second set of lane markers that are located within a first pre-determined distance from the vehicle;
obtain, from the second set of lane markers, a third set of lane markers that are located within a pre-determined field of view of the camera;
obtain a fourth set of lane markers by removal of one or more lane markers from the third set of lane markers located past a second pre-determined distance from the location of the vehicle; and
obtain a fifth set of lane markers by removal of one or more lane markers obstructed by one or more objects on the road from the fourth set of lane markers, wherein the fifth set of lane markers includes the lane marker.

### Claim 11 (depends on 10)

The computer of claim 10, wherein the third set of lane markers are obtained by the processor configured to remove at least one lane marker that is occluded by landscapes.

### Claim 12 (depends on 10)

The computer of claim 10, wherein the direction in which the vehicle is driven is obtained from an inertial measurement unit (IMU) sensor located on the vehicle.

### Claim 13 (depends on 9)

The computer of claim 9, wherein the processor is further configured to:
generate, from the image, a second image that includes a plurality of pixels, wherein each of the plurality of pixels has a value that is directly related to another distance between a pixel and a corner of the lane marker.

### Claim 14 (depends on 9)

The computer of claim 9, wherein the lane marker has a rectangular shape.

### Claim 15 (depends on 9)

The computer of claim 9, wherein the pose includes values for an orientation and a position of the camera.

### Claim 16 (depends on 9)

The computer of claim 9, wherein the pose of the camera is determined as the vehicle is driven on the road.

### Claim 17

A non-transitory computer readable program storage medium having code stored thereon, the code, when executed by a processor, causing the processor to implement a method, comprising:
receiving, by a computer located in a vehicle, an image from a camera located on the vehicle, wherein the image comprises a lane marker on a road;
determining pixel locations of a plurality of corners of the lane marker;
obtaining, from a database, three-dimensional (3D) coordinates of the plurality of corners of the lane marker; and
determining a pose of the camera by minimizing a distance from 3D world coordinates of at least one corner from the plurality of corners of the lane marker and at least one pixel location of the at least one corner of the lane marker.

### Claim 18 (depends on 17)

The non-transitory computer readable program storage medium of claim 17, wherein the 3D coordinates of the plurality of corners of the lane marker are obtained by:
obtaining, from the database and based on a location of the vehicle, 3D coordinates of corners of each lane marker in a first set of lane markers;
obtaining, from the first set of lane markers and based on a direction in which the vehicle is driven, a second set of lane markers that are located within a first pre-determined distance from the vehicle;
obtaining, from the second set of lane markers, a third set of lane markers that are located within a pre-determined field of view of the camera;
obtaining a fourth set of lane markers by removing one or more lane markers from the third set of lane markers located past a second pre-determined distance from the location of the vehicle; and
obtaining a fifth set of lane markers by removing one or more lane markers obstructed by one or more objects on the road from the fourth set of lane markers, wherein the fifth set of lane markers includes the lane marker.

### Claim 19 (depends on 18)

The non-transitory computer readable program storage medium of claim 18, wherein the third set of lane markers are obtained by removing at least one lane marker that is occluded by landscapes.

### Claim 20 (depends on 18)

The non-transitory computer readable program storage medium of claim 18, wherein the direction in which the vehicle is driven is obtained from an inertial measurement unit (IMU) sensor located on the vehicle.
