# US 2022/0284626 A1 · Lidar assisted wheel encoder to camera calibration

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0284626 A1
- Document ID: US20220284626A1
- Status: Published application
- Kind: A1
- Application number: 17/193,852
- Filed: 2021-03-05
- Published: 2022-09-08
- Earliest claimed: 2021-03-05
- Assignees: Black Sesame International Holding Limited
- Applicants: Black Sesame International Holding Limited
- Inventors: Yu Huang; Ruihui Di; Zheng Qi; Jizhang Shan
- Web page: https://patentgrep.com/patent/US20220284626A1

## CPC

- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G01S 7/4808: {Evaluating distance, position or velocity data}
- G01S 7/4972: {Alignment of sensor}
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/89: for mapping or imaging
- G01S 17/931: of land vehicles
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method of wheel encoder to camera calibration, including receiving a LiDAR (Light Detection and Ranging) signal, receiving a camera signal, receiving a wheel encoder signal, calibrating the camera signal to the LiDAR signal, calibrating the wheel encoder signal to the LiDAR signal and calibrating the camera signal to the wheel encoder signal based on the calibration of the camera signal to the LiDAR signal and the wheel encoder signal to the LiDAR signal.

## Claims

### Claim 1

A method of wheel encoder to camera calibration, comprising:
receiving a LiDAR signal;
receiving a camera signal;
receiving a wheel encoder signal;
calibrating the camera signal to the LiDAR signal;
calibrating the wheel encoder signal to the LiDAR signal; and
calibrating the camera signal to the wheel encoder signal based on the calibration of the camera signal to the LiDAR signal and the wheel encoder signal to the LiDAR signal, wherein the LiDAR signal is utilized as a bridge between the camera signal and the wheel encoder signal.

### Claim 2 (depends on 1)

The method of wheel encoder to camera calibration of claim 1, further comprising matching a LiDAR signal point cloud of the LiDAR signal to an image pixel of the camera signal for a scene point.

### Claim 3 (depends on 1)

The method of wheel encoder to camera calibration of claim 1, wherein the camera signal to wheel encoder signal calibration is a hand-eye calibration method.

### Claim 4 (depends on 1)

The method of wheel encoder to camera calibration of claim 1, further comprising:
determining a scaling factor for the camera signal to wheel encoder signal calibration; and
estimating a rotation of the wheel encoder signal to the camera signal for a plane motion translation in only one plane.

### Claim 5 (depends on 1)

The method of wheel encoder to camera calibration of claim 1, wherein the camera signal to LiDAR signal calibration is performed statically.

### Claim 6 (depends on 1)

The method of wheel encoder to camera calibration of claim 1, wherein the wheel encoder signal to LiDAR signal calibration is performed dynamically.

### Claim 7 (depends on 1)

The method of wheel encoder to camera calibration of claim 1, wherein the wheel encoder signal to LiDAR signal calibration is a hand-eye calibration method.

### Claim 8 (depends on 1)

The method of wheel encoder to camera calibration of claim 1, further comprising rotating a transport that receives the LiDAR signal, the camera signal and the wheel encoder signal via a rolling board.

### Claim 9 (depends on 1)

The method of wheel encoder to camera calibration of claim 1, further comprising:
determining a camera signal to wheel encoder signal scaling factor;
estimating the wheel encoder signal indicating a rotation in an X-direction without a rotation about a Z-axis; and
determining a wheel encoder translation in the X-direction without the rotation about the Z-axis.

### Claim 10 (depends on 1)

The method of wheel encoder to camera calibration of claim 1, further comprising:
determining a LiDAR signal to wheel encoder signal scaling factor;
estimating the wheel encoder signal indicating a rotation in an X-direction without a rotation about a Z-axis; and
determining a translation in the X-direction without the rotation about the Z-axis.

### Claim 11

A method of wheel encoder to camera calibration, comprising:
receiving a LiDAR signal;
receiving a camera signal;
receiving a wheel encoder signal;
determining a camera signal to wheel encoder signal scaling factor;
determining a LiDAR signal to wheel encoder signal scaling factor;
estimating the wheel encoder signal indicating a rotation in an X-direction without a rotation about a Z-axis based on the camera signal to wheel encoder signal scaling factor and the LiDAR signal to wheel encoder signal scaling factor;
determining a wheel encoder translation in the X-direction without the rotation about the Z-axis; and
calibrating the camera signal to the wheel encoder signal based on the calibration of the camera signal to the LiDAR signal and the wheel encoder signal to the LiDAR signal, wherein the LiDAR signal is utilized as a bridge between the camera signal and the wheel encoder signal.

### Claim 12 (depends on 11)

The method of wheel encoder to camera calibration of claim 11, further comprising matching a LiDAR signal point cloud of the LiDAR signal to an image pixel of the camera signal for a scene point.

### Claim 13 (depends on 11)

The method of wheel encoder to camera calibration of claim 11, wherein the camera signal to LiDAR signal calibration is performed statically.

### Claim 14 (depends on 11)

The method of wheel encoder to camera calibration of claim 11, wherein the wheel encoder signal to LiDAR signal calibration is performed dynamically.

### Claim 15 (depends on 11)

The method of wheel encoder to camera calibration of claim 11, further comprising rotating a transport that receives the LiDAR signal, the camera signal and the wheel encoder signal via a rolling board.
