# US 2024/0202964 A1 · Method and apparatus for determining a pose of a vehicle, and vehicle containing same

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0202964 A1
- Document ID: US20240202964A1
- Status: Published application
- Kind: A1
- Application number: 18/082,738
- Filed: 2022-12-16
- Published: 2024-06-20
- Earliest claimed: 2022-12-16
- Assignees: Huawei Technologies Co., Ltd.
- Applicants: Huawei Technologies Co., Ltd.
- Inventors: James Gregson; Shao Hua Chen
- Web page: https://patentgrep.com/patent/US20240202964A1

## CPC

- G06T 7/73: using feature-based methods
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30256: Lane; Road marking
- G06T 2207/30261: Obstacle
- G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})

## Abstract

A set of unordered points associated with road markings is received. The unordered points are inputted to a trained artificial neural network. Using the artificial neural network, an output is generated by applying non-linear regression to the unordered points. Based on the output, a pose of a vehicle is determined.

## Claims

### Claim 1

A method of determining a pose of a vehicle, comprising:
receiving a set of unordered points associated with road markings;
inputting the unordered points to a trained artificial neural network;
using the artificial neural network to generate an output by applying non-linear regression to the unordered points; and
determining, based on the output, the pose of the vehicle.

### Claim 2 (depends on 1)

The method of claim 1, wherein receiving the set of unordered points comprises:
receiving the set of unordered points from a lane detection computer module configured to identify, within one or more images of the road markings, the unordered points.

### Claim 3 (depends on 2)

The method of claim 2, wherein the lane detection computer module is configured to identify the unordered points using one or more convolutional neural networks.

### Claim 4 (depends on 1)

The method of claim 1, further comprising, before inputting the unordered points to the artificial neural network:
determining that the set of unordered points is valid.

### Claim 5 (depends on 4)

The method of claim 4, wherein determining that the set of unordered points is valid comprises determining that one or more of:
the unordered points are associated with at least two sets of lane markings, each set of lane markings comprising one or more aligned lane markings; and
at least a preset number of the unordered points are associated with a given set of lane markings, each set of lane markings comprising one or more aligned lane markings.

### Claim 6 (depends on 1)

The method of claim 1, further comprising, before inputting the unordered points to the artificial neural network:
normalizing the set of unordered points based on a camera intrinsic matrix associated with a camera onboard the vehicle.

### Claim 7 (depends on 1)

The method of claim 1, further comprising, before inputting the unordered points to the artificial neural network:
augmenting the set of unordered points by increasing a dimensionality of each unordered point by at least one.

### Claim 8 (depends on 7)

The method of claim 7, wherein augmenting the set of unordered points comprises associating one or more of the following with each unordered point:
a direction associated with the unordered point and at least one other unordered point;
a group comprising the unordered point and at least one other unordered point; and
a line fitted to the unordered point and at least one other unordered point.

### Claim 9 (depends on 1)

The method of claim 1, further comprising, before inputting the unordered points to the artificial neural network, one or more of:
increasing the number of unordered points by randomly sampling with replacement at least some of the unordered points;
in response to determining that the number of unordered points is above a threshold, reducing the number of unordered points by randomly sampling at least some of the unordered points;
in response to determining that the number of unordered points is above a threshold, reducing the number of unordered points by filtering out one or more unordered points that are neither within a minimum preset distance nor a maximum preset distance from the vehicle; and
for each unordered point, filtering out of the unordered point if it is determined to not relate to a road marking.

### Claim 10 (depends on 1)

The method of claim 1, wherein generating the output by applying non-linear regression to the set of unordered points comprises:
passing each unordered point through a first multi-layer perceptron neural network to generate an updated unordered point comprising an R-dimensional feature;
passing each updated unordered point through a second multi-layer perceptron neural network to generate an S-dimensional feature, wherein S>R;
generating an S-dimensional global feature by applying an order-independent reduction operation to each S-dimensional feature; and
generating the output by passing the S-dimensional global feature through a third multi-layer perceptron neural network.

### Claim 11 (depends on 1)

The method of claim 1, wherein determining the pose of the vehicle comprises:
adjusting the output based on an orientation of a camera onboard the vehicle; and
determining the pose based on the adjusted output.

### Claim 12 (depends on 1)

The method of claim 1, further comprising:
providing the determined pose to a collision warning system configured to identify, in one or more images, one or more objects and their respective one or more positions relative to the vehicle; and
using the collision warning system to adjust, based on the determined pose, the position of at least one identified object.

### Claim 13 (depends on 1)

The method of claim 1, wherein the pose of the vehicle comprises a pitch of the vehicle.

### Claim 14

A vehicle comprising:
a vehicle pose module comprising one or more processors communicative with one or more computer-readable media and configured to:
receive a set of unordered points associated with road markings;
input the unordered points to a trained artificial neural network;
use the artificial neural network to generate an output by applying non-linear regression to the unordered points; and
determine, based on the output, a pose of the vehicle.

### Claim 15 (depends on 14)

The vehicle of claim 14, wherein the one or more processors are further configured to:
receive the set of unordered points from a lane detection computer module configured to identify, within one or more images of the road markings, the unordered points.

### Claim 16 (depends on 14)

The vehicle of claim 14, wherein the one or more processors are further configured to, before inputting the unordered points to the artificial neural network:
normalize the set of unordered points based on a camera intrinsic matrix associated with a camera onboard the vehicle.

### Claim 17 (depends on 14)

The vehicle of claim 14, wherein the one or more processors are further configured to, before inputting the unordered points to the artificial neural network:
augment the set of unordered points by increasing a dimensionality of each unordered point by at least one.

### Claim 18 (depends on 14)

The vehicle of claim 14, wherein the one or more processors are further configured to, before inputting the unordered points to the artificial neural network, one or more of:
increase the number of unordered points by randomly sampling with replacement at least some of the unordered points;
in response to determining that the number of unordered points is above a threshold, reduce the number of unordered points by randomly sampling at least some of the unordered points;
in response to determining that the number of unordered points is above a threshold, reduce the number of unordered points by filtering out one or more unordered points that are neither within a minimum preset distance nor a maximum preset distance from the vehicle; and
for each unordered point, filter out of the unordered point if it is determined to not relate to a road marking.

### Claim 19 (depends on 14)

The vehicle of claim 14, wherein the one or more processors are further configured to:
provide the determined pose to a collision warning system configured to identify, in one or more images, one or more objects and their respective one or more positions relative to the vehicle; and
use the collision warning system to adjust, based on the determined pose, the position of at least one identified object.

### Claim 20

A non-transitory computer-readable medium comprising computer program code configured, when executed by one or more processors, to cause the one or more processors to perform a method comprising:
receiving a set of unordered points associated with road markings;
inputting the unordered points to a trained artificial neural network;
using the artificial neural network to generate an output by applying non-linear regression to the unordered points; and
determining, based on the output, a pose of the vehicle.
