# US 2026/0094449 A1 · Self extrinsic self-calibration via geometrically consistent self-supervised depth and ego-motion learning

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0094449 A1
- Document ID: US20260094449A1
- Status: Published application
- Kind: A1
- Application number: 19/411,318
- Filed: 2025-12-07
- Published: 2026-04-02
- Earliest claimed: 2023-09-18
- Assignees: Toyota Research Institute, Inc.; Toyota Jidosha Kabushiki Kaisha
- Applicants: Toyota Research Institute, Inc.
- Inventors: TAKAYUKI KANAI; Vitor Campagnolo Guizilini; Rares A. Ambrus; Adrien Gaidon; Igor Vasiljevic
- Web page: https://patentgrep.com/patent/US20260094449A1

## CPC

- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- B60W 40/105: Speed
- B60W 2420/403: Image sensing, e.g. optical camera
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT

## Abstract

Systems and methods described herein relate to self-supervised scale-aware learning of camera extrinsic parameters. One embodiment processes instantaneous velocity between a target image and a context image captured by a first camera; jointly training a depth network and pose network based on scaling by the instantaneous velocity; produce depth map using the depth network; produce ego-motion of the first camera using the pose network; generate synthesized image from the target image using a reprojection operation based on the depth map, the ego-motion, the context image and camera intrinsics; determine photometric loss by comparing the synthesized image to the target image; generate photometric consistency constraint using a gradient from the photometric loss; determine pose consistency constraint between the first camera and a second camera; and optimize the photometric consistency constraint, the pose consistency constraint, the depth network and the pose network to generate estimated extrinsic parameters.

## Claims

### Claim 1

A system for self-supervised learning of camera extrinsic parameters, comprising:
one or more processors; and
a memory communicably coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to:
obtain scale and motion magnitude information associated with image data, the image data comprising at least two images from a first camera;
estimate, from the image data, scene geometry and inter-frame motion associated with the first camera;
generate a reconstructed image using the estimated scene geometry, the estimated inter-frame motion, and at least one other image;
compute one or more image-based consistency measures between the reconstructed and observed images;
enforce one or more consistency constraints across time and across the first camera and a second camera; and
optimize one or more self-supervised objectives based on the image-based consistency measures and the consistency constraints to infer extrinsic parameters of at least the first camera, while updating parameters of a geometry estimator and a motion estimator.

### Claim 2 (depends on 1)

The system of claim 1, wherein the instructions further cause the one or more processors to adapt a learned model using self-supervised objectives to incorporate a scale constraint based on the scale and motion magnitude information.

### Claim 3 (depends on 2)

The system of claim 2, wherein the scale and motion magnitude information is derived from the image data and/or other senor or kinematic data, and wherein the scale constraint comprises enforcing consistency between a predicted inter-frame translation magnitude and measured motion magnitude over an inter-frame time interval.

### Claim 4 (depends on 1)

The system of claim 1, wherein estimating scene geometry and inter-frame motion is performed by the geometry estimator comprising a depth network and a motion estimator comprising a pose network that outputs ego-motion, and wherein optimizing the self-supervised objectives updates parameters of the depth network and the pose network.

### Claim 5 (depends on 1)

The system of claim 1, wherein generating the reconstructed image comprises a differentiable view-synthesis or reprojection based on camera intrinsics associated with a parametric camera model selected from a pinhole model, a unified camera model, an extended unified camera model, and a double sphere model, and wherein the image-based consistency measure comprises a photometric loss.

### Claim 6 (depends on 1)

The system of claim 1, wherein enforcing the one or more consistency constraints across cameras comprises a pose consistency constraint determined by converting predicted inter-frame motions from the first and second cameras to common coordinate frame and constraining translation vectors and rotation parameters to compute respective consistency losses.

### Claim 7 (depends on 1)

The system of claim 1, wherein the instructions further cause the one or more processors to:
receive image sequences from the first camera and the second camera mounted to a common platform;
warp images across spatial and temporal aces to form spatio-temporal contexts used in the self-supervised objectives; and
update camera intrinsics on a per-image sequence basis using gradients of the image-based consistency measures.

### Claim 8

A non-transitory computer-readable medium for self-supervised learning of camera extrinsic parameters and storing instructions that when executed by one or more processors cause the one or more processors to:
obtain scale and motion magnitude information associated with image data comprising at least two images from a first camera;
estimate, from the image data, scene geometry and inter-frame motion associated with the first camera;
generate a reconstructed image using the scene estimated geometry, the estimated inter-frame motion, and at least one other image;
compute one or more image-based consistency measures between the reconstructed and observed images;
enforce one or more consistency constraints across time and across the first camera and a second camera; and
optimize one or more self-supervised objectives based on the image-based consistency measures and the consistency constraints to infer extrinsic parameters of at least the first camera, while updating parameters of a geometry estimator and a motion estimator.

### Claim 9 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the instructions further cause the one or more processors to adapt a learned model using self-supervised objectives to incorporate a scale constraint based on the scale and/or motion magnitude information.

### Claim 10 (depends on 9)

The non-transitory computer-readable medium of claim 9, wherein the scale and motion magnitude information is derived from the image data and/or other sensor or kinematic data, and wherein the scale constraint comprises enforcing consistency between a predicted inter-frame translation magnitude and a measured motion magnitude over an inter-frame time interval.

### Claim 11 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein estimating the scene geometry and the inter-frame motion is performed by the geometry estimator comprising a depth network and the motion estimator comprising a pose network that outputs ego-motion, and wherein optimizing the self-supervised objectives updates parameters of the depth network and the pose network.

### Claim 12 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein generating the reconstructed image comprises a differentiable view-synthesis or reprojection using camera intrinsics associated with a parametric camera model selected from a pinhole model, a unified camera model, an extended unified camera model, or a double sphere model, and wherein the image-based consistency measures include a photometric loss.

### Claim 13 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein enforcing the one or more consistency constraints across cameras comprises a pose consistency constraint determined by converting predicted inter-frame motions from multiple cameras to a common coordinate frame and constraining translation vectors and rotation parameters to compute respective consistency losses.

### Claim 14 (depends on 8)

The non-transitory computer-readable medium of claim 8, wherein the instructions further cause the one or more processors to:
receive image sequences from the first camera and the second camera mounted to a common platform;
warp images across spatial and temporal axes to form spatio-temporal contexts used in the self-supervised objectives; and
update camera intrinsics on a per-image-sequence basis using gradients of the image-based consistency measures.

### Claim 15

A method for self-supervised learning of camera extrinsic parameters, the method comprising:
obtaining scale and motion magnitude information associated with image data comprising at least two images from a first camera;
estimating, from the image data, scene geometry and inter-frame motion associated with the first camera;
generating a reconstructed image using the estimated scene geometry, the estimated inter-frame motion, and at least one other image;
computing one or more image-based consistency measures between the reconstructed and observed images;
enforcing one or more consistency constraints across time and across the first camera and a second camera; and
optimizing one or more self-supervised objectives based on the image-based consistency measures and the consistency constraints to infer extrinsic parameters of at least the first camera, while updating parameters of a geometry estimator and a motion estimator.

### Claim 16 (depends on 15)

The method of claim 15, further comprising adapting a learned model using self-supervised objectives by incorporating a scale constraint based on the scale and motion magnitude information.

### Claim 17 (depends on 16)

The method of claim 16, wherein the scale and motion magnitude information is derived from the image data and/or other sensor or kinematic data, and wherein the scale constraint comprises enforcing consistency between a predicted inter-frame translation magnitude and a measured motion magnitude over an inter-frame time interval.

### Claim 18 (depends on 15)

The method of claim 15, wherein estimating the scene geometry and the inter-frame motion comprises applying the geometry estimator including a depth network to output a depth map and the motion estimator including a pose network to output ego-motion, and wherein optimizing the self-supervised objectives updates parameters of the depth network and the pose network.

### Claim 19 (depends on 15)

The method of claim 15, wherein generating the reconstructed image comprises a differentiable view-synthesis or reprojection using camera intrinsics associated with a parametric camera model selected from a pinhole model, a unified camera model, an extended unified camera model, or a double sphere model, and wherein the image-based consistency measures include a photometric loss.

### Claim 20 (depends on 15)

The method of claim 15, wherein enforcing the one or more consistency constraints across cameras comprises a pose consistency constraint determined by converting predicted inter-frame motions from multiple cameras to a common coordinate frame and constraining translation vectors and rotation parameters to compute respective consistency losses, and further comprising receiving image sequences from the first camera and the second camera, warping images across spatial and temporal axes to form spatio-temporal contexts used in the self-supervised objectives, and updating camera intrinsics on a per-image-sequence basis using gradients of the image-based consistency measures.
