# US 2026/0087673 A1 · Non-Repetitive Scanning Solid State LiDAR and Camera Extrinsic Calibration

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0087673 A1
- Document ID: US20260087673A1
- Status: Published application
- Kind: A1
- Application number: 19/337,789
- Filed: 2025-09-23
- Published: 2026-03-26
- Earliest claimed: 2024-09-23
- Assignees: Niantic Spatial, Inc.
- Applicants: Niantic Spatial, Inc.
- Inventors: David Zuñiga-Noël; Christopher Joseph Shaw; Victor Adrian Prisacariu
- Web page: https://patentgrep.com/patent/US20260087673A1

## CPC

- G06T 7/85: {Stereo camera calibration}
- G01S 7/4817: {relating to scanning}
- G01S 7/497: Means for monitoring or calibrating
- G01S 17/89: for mapping or imaging
- G06T 7/73: using feature-based methods
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 2207/10012: Stereo images
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30244: Camera pose
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES

## Abstract

A system performs pose calibration between a LiDAR sensor and a camera. The system may receive an image frame captured by the camera mounted on a device. The system may receive a point cloud captured by the LiDAR sensor mounted on the device, the LiDAR sensor having an overlapping field of view with the camera. The system may identify markers of a calibration target captured in the image frame by applying a feature identification model to the image frame. The system may identify the markers of the calibration target captured in the point cloud by: clustering points in the point cloud into one or more planes, selecting one of the planes based on sizes of the planes; and identifying holes in the selected plane as the markers of the calibration target. The system may determine a pose transformation between the camera and the LiDAR sensor based on information identifying the markers from the image frame and information identifying the markers from the point cloud.

## Claims

### Claim 1

A computer-implemented method for performing extrinsic calibration between a camera and a light detection and ranging (LiDAR) sensor, the method comprising:
receiving an image frame captured by the camera mounted on a device;
receiving a point cloud captured by the LiDAR sensor mounted on the device, the LiDAR sensor having an overlapping field of view with the camera;
identifying markers of a calibration target captured in the image frame by applying a feature identification model to the image frame;
identifying the markers of the calibration target captured in the point cloud by:
clustering points in the point cloud into one or more planes;
selecting one of the planes based on sizes of the planes; and
identifying holes in the selected plane as the markers of the calibration target; and
determining a pose transformation between the camera and the LiDAR sensor based on information identifying the markers from the image frame and information identifying the markers from the point cloud.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein determining the pose transformation between the camera and the LiDAR sensor is further based on intrinsic parameters of the camera.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the camera is part of a stereoscopic camera pair mounted on the device.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the LiDAR sensor is a non-repetitive scanning solid state LiDAR sensor.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein identifying the markers of the calibration target captured in the point cloud further comprises:
projecting points clustered in the selected plane into a two-dimensional grid; and
identifying the holes in the projected points.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein identifying the markers of the calibration target captured in the point cloud further comprises:
projecting the markers into three-dimensional coordinate system of the point cloud;
determining three-dimensional coordinates for each marker in the three-dimensional coordinate system.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein clustering the points in the point cloud into one or more planes comprises:
performing a voxel growing approach to incrementally capture points into one cluster of points; and
identifying the one or more planes from the clusters of points.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein selecting one of the planes sized to match the calibration target comprises selecting the plane of largest size.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein identifying the holes in the selected plane as the markers of the calibration target comprises identifying the holes informed by a spatial configuration of the markers in the calibration target.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein determining the pose transformation comprises performing a Perspective-n-Point algorithm with the markers in the image frame and the markers in the point cloud to determine the pose transformation.

### Claim 11

A system for performing extrinsic calibration between a camera and a light detection and ranging (LiDAR) sensor, the system comprising:
a processor; and
a non-transitory computer-readable storage medium storing instructions that, when executed by the processor, cause the processor to perform operations comprising:
receiving an image frame captured by the camera mounted on a device;
receiving a point cloud captured by the LiDAR sensor mounted on the device, the LiDAR sensor having an overlapping field of view with the camera;
identifying markers of a calibration target captured in the image frame by applying a feature identification model to the image frame;
identifying the markers of the calibration target captured in the point cloud by:
clustering points in the point cloud into one or more planes;
selecting one of the planes based on sizes of the planes; and
identifying holes in the selected plane as the markers of the calibration target; and
determining a pose transformation between the camera and the LiDAR sensor based on information identifying the markers from the image frame and information identifying the markers from the point cloud.

### Claim 12 (depends on 11)

The system of claim 11, wherein determining the pose transformation between the camera and the LiDAR sensor is further based on intrinsic parameters of the camera.

### Claim 13 (depends on 11)

The system of claim 11, wherein the camera is part of a stereoscopic camera pair mounted on the device.

### Claim 14 (depends on 11)

The system of claim 11, wherein the LiDAR sensor is a non-repetitive scanning solid state LiDAR sensor.

### Claim 15 (depends on 11)

The system of claim 11, wherein identifying the markers of the calibration target captured in the point cloud further comprises:
projecting points clustered in the selected plane into a two-dimensional grid; and
identifying the holes in the projected points.

### Claim 16 (depends on 11)

The system of claim 11, wherein identifying the markers of the calibration target captured in the point cloud further comprises:
projecting the markers into three-dimensional coordinate system of the point cloud;
determining three-dimensional coordinates for each marker in the three-dimensional coordinate system.

### Claim 17 (depends on 11)

The system of claim 11, wherein clustering the points in the point cloud into one or more planes comprises:
performing a voxel growing approach to incrementally capture points into one cluster of points; and
identifying the one or more planes from the clusters of points.

### Claim 18 (depends on 11)

The system of claim 11, wherein selecting one of the planes sized to match the calibration target comprises selecting the plane of largest size.

### Claim 19 (depends on 11)

The system of claim 11, wherein identifying the holes in the selected plane as the markers of the calibration target comprises identifying the holes informed by a spatial configuration of the markers in the calibration target.

### Claim 20 (depends on 11)

The system of claim 11, wherein determining the pose transformation comprises performing a Perspective-n-Point algorithm with the markers in the image frame and the markers in the point cloud to determine the pose transformation.
