US 2026/0094449 A1 · SELF EXTRINSIC SELF-CALIBRATION VIA GEOMETRICALLY CONSISTENT SELF-SUPERVISED DEPTH AND EGO-MOTION LEARNING

Abstract

Systems and methods described herein relate to self-supervised scale-aware learning of camera extrinsic parameters. One embodiment processes instantaneous velocity between a target image and a context image captured by a first camera; jointly training a depth network and pose network based on scaling by the instantaneous velocity; produce depth map using the depth network; produce ego-motion of the first camera using the pose network; generate synthesized image from the target image using a reprojection operation based on the depth map, the ego-motion, the context image and camera intrinsics; determine photometric loss by comparing the synthesized image to the target image; generate photometric consistency constraint using a gradient from the photometric loss; determine pose consistency constraint between the first camera and a second camera; and optimize the photometric consistency constraint, the pose consistency constraint, the depth network and the pose network to generate estimated extrinsic parameters.

Claims20

Dependent claims

A system for self-supervised learning of camera extrinsic parameters, comprising:

one or more processors;
and
a memory communicably coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to:
obtain scale and motion magnitude information associated with image data, the image data comprising at least two images from a first camera;
estimate, from the image data, scene geometry and inter-frame motion associated with the first camera;
generate a reconstructed image using the estimated scene geometry, the estimated inter-frame motion, and at least one other image;
compute one or more image-based consistency measures between the reconstructed and observed images;
enforce one or more consistency constraints across time and across the first camera and a second camera;
and
optimize one or more self-supervised objectives based on the image-based consistency measures and the consistency constraints to infer extrinsic parameters of at least the first camera, while updating parameters of a geometry estimator and a motion estimator.

7 more elements

A non-transitory computer-readable medium for self-supervised learning of camera extrinsic parameters and storing instructions that when executed by one or more processors cause the one or more processors to:

obtain scale and motion magnitude information associated with image data comprising at least two images from a first camera;
estimate, from the image data, scene geometry and inter-frame motion associated with the first camera;
generate a reconstructed image using the scene estimated geometry, the estimated inter-frame motion, and at least one other image;
compute one or more image-based consistency measures between the reconstructed and observed images;
enforce one or more consistency constraints across time and across the first camera and a second camera;
and
optimize one or more self-supervised objectives based on the image-based consistency measures and the consistency constraints to infer extrinsic parameters of at least the first camera, while updating parameters of a geometry estimator and a motion estimator.

4 more elements

Dependencies are inferred from the claim text.

Self extrinsic self-calibration via geometrically consistent self-supervised depth and ego-motion learning

Life of this patent

Self extrinsic self-calibration via geometrically consistent self-supervised depth and ego-motion learning

Life of this patent

Abstract

Claims20