# US 12,217,448 B2 · Camera pose information detection method and apparatus, and corresponding intelligent driving device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,217,448 B2
- Document ID: US12217448B2
- Status: Granted
- Kind: B2
- Application number: 17/610,514
- Filed: 2020-05-12
- Published: 2022-07-07
- Granted: 2025-02-04
- Earliest claimed: 2019-05-13
- Assignees: CiDi Inc.
- Applicants: CiDi Inc.
- Inventors: Rongdong Hu; Mingxi Tang; Linjiang Xie; Qing Peng
- Web page: https://patentgrep.com/patent/US12217448B2

## CPC

- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 7/73: using feature-based methods
- G06T 3/00: Geometric image transformations in the plane of the image
- G06T 7/50: Depth or shape recovery
- G06V 20/588: {Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road}
- H04N 17/00: Diagnosis, testing or measuring for television systems or their details
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30244: Camera pose
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A camera pose information detection method, includes collecting a plurality of images by means of a multi-lens camera, and determining a road surface region image based on the plurality of images; and projecting points in the road surface region image onto a camera coordinate system, so as to obtain camera pose information.

## Claims

### Claim 1

A camera pose information detection method, the method being implemented by a processor to execute a computer program stored on a memory, and comprising:
collecting a plurality of images by means of a multi-lens camera, and determining a road surface region image based on the plurality of images; and
projecting points in the road surface region image onto a coordinate plane of a camera coordinate system, so as to obtain at least a pitch angle and/or a roll angle in camera pose information;
the determining a road surface region image based on the plurality of images comprises:
selecting one of the plurality of images as an original image, and generating a depth map of the original image in combination with other images in the plurality of images;
traversing the original image by using a search box, and obtaining a plurality of subimages corresponding to various search boxes; and
calculating road surface confidences of the subimages, and selecting the subimage with the maximum road surface confidence as the road surface region image.

### Claim 2 (depends on 1)

The camera pose information detection method according to claim 1, further comprising:
acquiring points on a lane line, and projecting the points on the lane line onto the coordinate plane of the camera coordinate system, so as to obtain at least a yaw angle in the camera pose information.

### Claim 3 (depends on 1)

The camera pose information detection method according to claim 1, wherein the projecting points in the road surface region image onto a coordinate plane of a camera coordinate system to obtain camera pose information, or the projecting the points on the lane line onto the coordinate plane of the camera coordinate system to obtain the camera pose information comprises:
fitting the projected points in the coordinate plane where the points are located, and obtaining the camera pose information based on an angle formed by a fitted line and a coordinate axis of the coordinate plane.

### Claim 4 (depends on 1)

An intelligent driving device, comprising:
a processor, and a memory and a network interface that are coupled to the processor;
a vehicle sensor comprising a multi-lens camera configured to acquire a plurality of images;
wherein the processor is configured to perform the method according to claim 1.

### Claim 5 (depends on 1)

The camera pose information detection method according to claim 1, wherein the determining a road surface region image based on the plurality of images further comprises:
filtering the plurality of images based on a preset depth threshold, and retaining only the subimages with center point depth values less than the depth threshold; and
calculating road surface confidences for the retained subimages, and taking the subimage with the highest confidence as the road surface region image.

### Claim 6 (depends on 5)

The camera pose information detection method according to claim 5, wherein the depth threshold ranges at least from 15 m to 25 m.

### Claim 7 (depends on 1)

The camera pose information detection method according to claim 1, wherein the calculating road surface confidences of the subimages comprises:
calculating information entropies of the subimages by using the following formula, and taking reciprocals of the information entropies as the road surface confidences,
E
=
-
∑
g
=
0
2
m
-
1
⁢
⁢
p
⁡
(
g
)
⁢
log
2
⁡
(
p
⁡
(
g
)
)
where E is an image information entropy, g represents a grayscale value, P(g) is a probability of appearance of a current grayscale value g in the subimage, and m is a natural number greater than or equal to 1.

### Claim 8 (depends on 1)

The camera pose information detection method according to claim 1, wherein the traversing the original image by using a search box comprises: determining dimensions of the search box according to the following formulas:
a width of the search box being Ws=|W/(n*δ)|,
a height of the search box being Hs=|H/n|;
where W is a width of the original image, H is a height of the original image, δ is an aspect ratio W/H of the original image, ┌ ┐ is a rounding-up symbol, n is a search ratio parameter, and n is a natural number.

### Claim 9

A camera pose detection apparatus, comprising:
a processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the computer program causes the processor to:
obtain a plurality of images;
determine a road surface region image based on the plurality of images; and
determine at least a pitch angle and/or a roll angle in camera pose information based on projections of points in the road surface region image onto a coordinate plane of a camera coordinate system;
wherein when the computer program is executed by the processor, the computer program further causes the processor to select one of the plurality of images as an original image, and generate a depth map of the original image in combination with other images in the plurality of images;
wherein the processor is further caused to:
traverse the original image by using a search box, and obtain a plurality of subimages corresponding to various search boxes; and
calculate road surface confidences of the subimages, and select the subimage with the maximum road surface confidence as the road surface region image.

### Claim 10 (depends on 9)

The camera pose detection apparatus according to claim 9, wherein when the computer program is executed by the processor, the computer program further causes the processor to acquire pixel points on a lane line and determine at least a yaw angle in the camera pose information based on projections of the pixel points on the lane line onto the coordinate plane of the camera coordinate system.

### Claim 11 (depends on 9)

The camera pose detection apparatus according to claim 9, wherein when the computer program is executed by the processor, the computer program further causes the processor to fit the points in the road surface region image or the points on the lane line, which are projected onto the coordinate plane of the camera coordinate system, in the corresponding coordinate plane, and obtain the camera pose information based on an angle formed by a fitted line and a coordinate axis of the coordinate plane.

### Claim 12 (depends on 9)

The camera pose detection apparatus according to claim 9, wherein when the computer program is executed by the processor, the computer program further causes the processor to:
filter the plurality of images based on a preset depth threshold, and retain only the subimages with center point depth values less than the depth threshold; and
calculate road surface confidences of the retained subimages, and select the subimage with the maximum road surface confidence as the road surface region image.

### Claim 13 (depends on 12)

The camera pose detection apparatus according to claim 12, wherein the depth threshold ranges at least from 15 m to 25 m.

### Claim 14 (depends on 9)

The camera pose detection apparatus according to claim 9, wherein when the computer program is executed by the processor, the computer program further causes the processor to calculate information entropies of the subimages by using the following formula, and take reciprocals of the information entropies as the road surface confidences,
E
=
-
∑
g
=
0
2
m
-
1
⁢
⁢
p
⁡
(
g
)
⁢
log
2
⁡
(
p
⁡
(
g
)
)
where E is an image information entropy, g represents a grayscale value, P(g) is a probability of appearance of a current grayscale value g in the subimage, and m is a natural number greater than 1.

### Claim 15 (depends on 9)

The camera pose detection apparatus according to claim 9, wherein when the computer program is executed by the processor, the computer program further causes the processor to determine dimensions of the search box according to the following formulas:
a width of the search box being Ws=|W/(n*δ)|,
a height of the search box being Hs=(H/n|;
where W is a width of the original image, H is a height of the original image, δ is an aspect ratio W/H of the original image, ┌ ┐ is a rounding-up symbol, n is a search ratio parameter, and n is a natural number.