# US 2024/0193815 A1 · Verification of a multi-camera calibration

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0193815 A1
- Document ID: US20240193815A1
- Status: Published application
- Kind: A1
- Application number: 18/534,152
- Filed: 2023-12-08
- Published: 2024-06-13
- Earliest claimed: 2022-12-09
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Ali Osman Ulusoy
- Web page: https://patentgrep.com/patent/US20240193815A1

## CPC

- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 7/85: {Stereo camera calibration}
- H04N 17/002: {for television cameras}
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G06T 2207/30208: Marker matrix
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method may receive first image data from a first camera and second image data from a second camera. A method may determine a calibration target position based on an extrinsic calibration, the first image data, the second image data, and calibration target information. A method may measure a first reprojection error of the first camera based on the extrinsic calibration, the calibration target position, and the first image data. A method may measure a second reprojection error of the second camera based on the extrinsic calibration, the calibration target position, and the second image data. Upon determining that any combination of the first reprojection error is greater than a threshold reprojection error or the second reprojection error is greater than the threshold reprojection error, the method may provide an indication that the first camera and the second camera are out of calibration.

## Claims

### Claim 1

A method comprising:
receiving first image data from a first camera and second image data from a second camera, the first camera and the second camera having an overlapping field of view including a calibration target in a calibration target orientation;
determining a calibration target position based on an extrinsic calibration, the first image data, the second image data, and calibration target information;
measuring a first reprojection error of the first camera based on the extrinsic calibration, the calibration target position, and the first image data;
measuring a second reprojection error of the second camera based on the extrinsic calibration, the calibration target position, and the second image data; and
upon determining that any combination of the first reprojection error is greater than a threshold reprojection error or the second reprojection error is greater than the threshold reprojection error, providing an indication that the first camera and the second camera are out of calibration.

### Claim 2 (depends on 1)

The method of claim 1, wherein determining the calibration target position further comprises identifying a first fiducial and a second fiducial in the calibration target.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
determining a first fiducial 3D position and a second fiducial 3D position based on the extrinsic calibration, the first image data and the second image data.

### Claim 4 (depends on 1)

The method of claim 1, wherein determining the calibration target position further comprises determining a first calibration target region pose and a second calibration target region pose, the first calibration target region pose being different from the second calibration target region pose.

### Claim 5 (depends on 1)

The method of claim 1, wherein the calibration target position is further determined based on a camera intrinsic comprising at least one of: a focal length, a principal point, a pixel skew, or distortion parameters.

### Claim 6 (depends on 1)

The method of claim 1, wherein the calibration target orientation is a first calibration target orientation, the calibration target position is a first calibration target position, and the method further comprises:
receiving third image data from the first camera and fourth image data from the second camera, the first camera and the second camera having the calibration target in the overlapping field of view in a second calibration target orientation; and
determining a second calibration target position based on the extrinsic calibration, the third image data, and the fourth image data, wherein measuring the first reprojection error of the first camera is further based on the extrinsic calibration, the second calibration target position, and the third image data, and measuring the second reprojection error of the second camera is further based on the extrinsic calibration, the second calibration target position, and the fourth image data.

### Claim 7 (depends on 1)

The method of claim 1, wherein the threshold reprojection error is one pixel.

### Claim 8

A device comprising:
a first camera;
a second camera;
a processor in communication with the first camera and the second camera; and
a memory coupled to the processor, the memory configured with instructions to:
receive first image data from the first camera and second image data from the second camera, the first camera and the second camera having an overlapping field of view including a calibration target in a calibration target orientation,
determine a calibration target position based on an extrinsic calibration, the first image data, the second image data, and calibration target information,
measure a first reprojection error of the first camera based on the extrinsic calibration, the calibration target position, and the first image data,
measure a second reprojection error of the second camera based on the extrinsic calibration, the calibration target position, and the second image data, and
upon determining that any combination of: the first reprojection error is greater than a threshold reprojection error or the second reprojection error is greater than the threshold reprojection error, provide an indication that the first camera and the second camera are out of calibration.

### Claim 9 (depends on 8)

The device of claim 8, wherein determining the calibration target position further comprises identifying a first fiducial and a second fiducial in the calibration target.

### Claim 10 (depends on 9)

The device of claim 9, wherein the memory is further configured with instructions to:
determine a first fiducial 3D position and a second fiducial 3D position based on the extrinsic calibration, the first image data and the second image data.

### Claim 11 (depends on 8)

The device of claim 8, wherein determining the calibration target position further comprises determining a first calibration target region pose and a second calibration target region pose, the first calibration target region pose being different from the second calibration target region pose.

### Claim 12 (depends on 8)

The device of claim 8, wherein the calibration target position is further determined based on a camera intrinsic comprising at least one of: a focal length, a principal point, a pixel skew, or distortion parameters.

### Claim 13 (depends on 8)

The device of claim 8, wherein the calibration target orientation is a first calibration target orientation, the calibration target position is a first calibration target position, and the memory is further configured with instructions to:
receive third image data from the first camera and fourth image data from the second camera, the first camera and the second camera having the calibration target in the overlapping field of view in a second calibration target orientation; and
determine a second calibration target position based on the extrinsic calibration, the third image data, and the fourth image data, wherein measuring the first reprojection error of the first camera is further based on the extrinsic calibration, the second calibration target position, and the third image data, and measuring the second reprojection error of the second camera is further based on the extrinsic calibration, the second calibration target position, and the fourth image data.

### Claim 14 (depends on 8)

The device of claim 8, wherein the threshold reprojection error is one pixel.

### Claim 15

A non-transitory computer readable memory that, when executed by a processor, causes a device connected to a first camera and a second camera to:
receive first image data from the first camera and second image data from the second camera, the first camera and the second camera having an overlapping field of view including a calibration target in a calibration target orientation;
determine a calibration target position based on an extrinsic calibration, the first image data, the second image data, and calibration target information;
measure a first reprojection error of the first camera based on the extrinsic calibration, the calibration target position, and the first image data;
measure a second reprojection error of the second camera based on the extrinsic calibration, the calibration target position, and the second image data; and
upon determining that any combination of: the first reprojection error is greater than a threshold reprojection error or the second reprojection error is greater than the threshold reprojection error, provide an indication that the first camera and the second camera are out of calibration.

### Claim 16 (depends on 15)

The non-transitory computer readable memory of claim 15 wherein determining the calibration target position further comprises identifying a first fiducial and a second fiducial in the calibration target.

### Claim 17 (depends on 16)

The non-transitory computer readable memory of claim 16, wherein the non-transitory computer readable memory further causes the device to:
determine a first fiducial 3D position and a second fiducial 3D position based on the extrinsic calibration, the first image data and the second image data.

### Claim 18 (depends on 15)

The non-transitory computer readable memory of claim 15, wherein determining the calibration target position further comprises determining a first calibration target region pose and a second calibration target region pose, the first calibration target region pose being different from the second calibration target region pose.

### Claim 19 (depends on 15)

The non-transitory computer readable memory of claim 15, wherein the calibration target position is further determined based on a camera intrinsic comprising at least one of: a focal length, a principal point, a pixel skew, or distortion parameters.

### Claim 20 (depends on 15)

The non-transitory computer readable memory of claim 15, wherein the calibration target orientation is a first calibration target orientation, the calibration target position is a first calibration target position, and the non-transitory computer readable memory further causes the device to:
receive third image data from the first camera and fourth image data from the second camera, the first camera and the second camera having the calibration target in the overlapping field of view in a second calibration target orientation; and
determine a second calibration target position based on the extrinsic calibration, the third image data, and the fourth image data, wherein measuring the first reprojection error of the first camera is further based on the extrinsic calibration, the second calibration target position, and the third image data, and measuring the second reprojection error of the second camera is further based on the extrinsic calibration, the second calibration target position, and the fourth image data.
