# US 2026/0017826 A1 · Method and computer system for calibrating relative sensor orientations

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0017826 A1
- Document ID: US20260017826A1
- Status: Published application
- Kind: A1
- Application number: 19/233,048
- Filed: 2025-06-10
- Published: 2026-01-15
- Earliest claimed: 2024-07-11
- Assignees: dSPACE GmbH
- Applicants: dSPACE GmbH
- Inventors: Simon ROMANSKI; Guillaume ROY; Harald HANSELMANN
- Web page: https://patentgrep.com/patent/US20260017826A1

## CPC

- G01S 7/4972: {Alignment of sensor}
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G01S 17/86: Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
- G01S 17/931: of land vehicles
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06N 5/00: Computing arrangements using knowledge-based models
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A method for calibrating the relative orientation of a spatial sensor and an area sensor includes: receiving a plurality of sensor frames of the spatial sensor and the area sensor; grouping sensor frames of the spatial sensor with sensor frames of the area sensor based on temporal correlation; selecting at least one group of frames comprising a spatial sensor frame and an area sensor frame; receiving annotations for the sensor frames in the at least one selected group, wherein the annotations comprise bounding boxes for detected objects; projecting the corners of a three-dimensional bounding box in the spatial sensor frame to an image plane of the area sensor, to produce a projected rectangle; calculating an energy term based on the coordinates of the corners of the projected rectangle and the corresponding bounding box in the area sensor frame; and optimizing the relative sensor orientation by minimizing the energy term.

## Claims

### Claim 1

A method for calibrating the relative orientation of a spatial sensor and an area sensor, wherein the measuring zones of the spatial sensor and the area sensor are overlapping, the method comprising
receiving, by a computing system, a plurality of sensor frames of the spatial sensor and the area sensor;
grouping, by the computing system, sensor frames of the spatial sensor with sensor frames of the area sensor based on temporal correlation;
selecting, by the computing system, at least one group of frames comprising a spatial sensor frame and an area sensor frame;
receiving, by the computing system, annotations for the sensor frames in the at least one selected group, wherein the annotations comprise bounding boxes for detected objects;
projecting, by the computing system, the corners of a three-dimensional bounding box in the spatial sensor frame to an image plane of the area sensor, to produce a projected rectangle;
calculating, by the computing system, an energy term based on the coordinates of the corners of the projected rectangle and the corresponding bounding box in the area sensor frame; and
optimizing, by the computing system, the relative sensor orientation by minimizing the energy term.

### Claim 2 (depends on 1)

The method according to claim 1, wherein the plurality of sensor frames are consecutive, and wherein a limited number of groups of frames is selected for optimizing the relative sensor orientation.

### Claim 3 (depends on 2)

The method according to claim 2, wherein the selected groups of frames comprise one group at the beginning of the sequence of consecutive frames and one group at the end of the sequence of consecutive frames.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the annotations for at least one group of frames are produced by a neural network trained on spatial sensor frames and/or a neural network trained on area sensor frames and/or a human annotator.

### Claim 5 (depends on 1)

The method according to claim 1, wherein for a group of frames comprising multiple spatial sensor frames and area sensor frames and/or for a group of spatial sensor frame and area sensor frame comprising multiple annotated objects, optimizing the relative sensor orientation is based on the sum of the energy terms for all annotated objects in the selected group.

### Claim 6 (depends on 1)

The method according to claim 1, wherein the intrinsic parameters of the area sensor are kept constant when optimizing the relative orientation.

### Claim 7 (depends on 1)

The method according to claim 1, wherein for corner coordinates x0, y0, x1, y1 of the bounding box and for corner coordinates x0′, y0′, x1′, y1′ of the corresponding projected rectangle, the energy term is
E
=
1
(
x
1
-
x
0
)
⁢
(
y
1
-
y
0
)
⁢
(
(
x
0
-
x
0
′
)
2
+
(
y
0
-
y
0
′
)
2
+
(
x
1
-
x
1
′
)
2
+
(
y
1
-
y
1
′
)
2
)
.

### Claim 8 (depends on 1)

The method according to claim 1, wherein optimizing the relative sensor orientation is performed iteratively until a convergence criterion is met.

### Claim 9 (depends on 8)

The method according to claim 8, wherein the relative sensor orientation is described using a matrix describing rotation and translation of a sensor in three-dimensional space, and wherein during or subsequent to each optimization iteration, a factorization algorithm is applied to the rotation matrix for ensuring orthonormality.

### Claim 10 (depends on 1)

The method according to claim 1, wherein for data sets comprising additional sensor data, additional groups of spatial sensor frames and area sensor frames are selected based on the acceleration in at least one space axis.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the plurality of sensor frames are consecutive sensor frames;
wherein the selected groups of frames represent a fraction of the received sequence of consecutive frames; and
wherein all of the consecutive spatial sensor frames are labeled;
wherein the method further comprises:
projecting the corners of a three-dimensional bounding box in the space sensor frame to the image plane of the area sensor, to produce a projected rectangle,
using the projected rectangle as two-dimensional bounding box in the area sensor frame, and
determining at least one attribute of the object.

### Claim 12

A non-transitory computer-readable medium having processor-executable instructions stored thereon for calibrating the relative orientation of a spatial sensor and an area sensor, wherein the measuring zones of the spatial sensor and the area sensor are overlapping, and wherein the processor-executable instructions, when executed, facilitate performance of the following:
receiving, by a computing system, a plurality of sensor frames of the spatial sensor and the area sensor;
grouping, by the computing system, sensor frames of the spatial sensor with sensor frames of the area sensor based on temporal correlation;
selecting, by the computing system, at least one group of frames comprising a spatial sensor frame and an area sensor frame;
receiving, by the computing system, annotations for the sensor frames in the at least one selected group, wherein the annotations comprise bounding boxes for detected objects;
projecting, by the computing system, the corners of a three-dimensional bounding box in the spatial sensor frame to an image plane of the area sensor, to produce a projected rectangle;
calculating, by the computing system, an energy term based on the coordinates of the corners of the projected rectangle and the corresponding bounding box in the area sensor frame; and
optimizing, by the computing system, the relative sensor orientation by minimizing the energy term.

### Claim 13

A computer system, comprising:
a host computer comprising a processor, a working memory, a display, an input device and a non-volatile memory;
wherein the non-volatile memory comprises instructions;
wherein the processor is configured to execute the instructions to facilitate the following being performed by the computing system:
receiving a plurality of sensor frames of the spatial sensor and the area sensor;
grouping sensor frames of the spatial sensor with sensor frames of the area sensor based on temporal correlation;
selecting at least one group of frames comprising a spatial sensor frame and an area sensor frame;
receiving annotations for the sensor frames in the at least one selected group, wherein the annotations comprise bounding boxes for detected objects;
projecting the corners of a three-dimensional bounding box in the spatial sensor frame to an image plane of the area sensor, to produce a projected rectangle;
calculating an energy term based on the coordinates of the corners of the projected rectangle and the corresponding bounding box in the area sensor frame; and
optimizing the relative sensor orientation by minimizing the energy term.
