US 2026/0228904 A1Published application

Systems and methods for performing monocular depth estimation under viewpoint shifts

AssigneeVerizon Patent and Licensing Inc.

InventorsAurel PJETRI, Stefano CAPRASECCA, Leonardo TACCARI, Henrique Pineiro MONTEAGUDO, Francesco SAMBO, Matteo SIMONCINI, Wallace WALTER, Douglas COIMBRA DE ANDRADE, Andrew David BAGDANOV

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

A device may receive base video data and shifted video data, and may select a base video frame from the base video data and a shifted video frame from the shifted video data. The device may estimate a base homography for the base video frame and a shifted homography for the shifted video frame. The device may process the base/shifted video frames, with an object detection model, to detect objects and to generate bounding boxes for the objects, and may utilize the base homography, the shifted homography, and the bounding boxes to calculate ground truth distances to the objects. The device may utilize a depth estimation model to calculate estimated distances to the objects, and may calculate differences between the ground truth and estimated distances. The device may determine, based on the differences, that the depth estimation model needs to be trained and may generate a trained depth estimation model.

Claims20

Dependent claims

A method, comprising:

receiving, by a device, base video data that includes base video frames and shifted video data that includes shifted video frames;
selecting, by the device, a base video frame from the base video data and a shifted video frame from the shifted video data;
estimating, by the device, a base homography that maps ground points in the base video frame to a target plane with a metric reference system;
estimating, by the device, a shifted homography that maps ground points in the shifted video frame to the target plane with the metric reference system;
processing, by the device, the base video frame and the shifted video frame, with an object detection model, to detect objects in the base video frame and the shifted video frame and to generate bounding boxes for the objects;
utilizing, by the device, the base homography, the shifted homography, and the bounding boxes to calculate ground truth distances to the objects of the base video frame and the shifted video frame;
utilizing, by the device, a depth estimation model to calculate estimated distances to the objects;
calculating, by the device, differences between the ground truth distances and the estimated distances;
determining, by the device and based on the differences, that the depth estimation model needs to be trained and generating a trained depth estimation model;
and
performing, by the device, one or more actions based on the trained depth estimation model.

8 more elements

A device, comprising:

one or more processors configured to:
receive base video data that includes base video frames and shifted video data that includes shifted video frames;
select a base video frame from the base video data and a shifted video frame from the shifted video data;
estimate a base homography that maps ground points in the base video frame to a target plane with a metric reference system;
estimate a shifted homography that maps ground points in the shifted video frame to the target plane with the metric reference system;
process the base video frame and the shifted video frame, with an object detection model, to detect objects in the base video frame and the shifted video frame and to generate bounding boxes for the objects;
utilize the base homography, the shifted homography, and the bounding boxes to calculate ground truth distances to the objects of the base video frame and the shifted video frame;
utilize a depth estimation model to calculate estimated distances to the objects;
calculate differences between the ground truth distances and the estimated distances;
determine, based on the differences, that the depth estimation model needs to be trained and generate a trained depth estimation model;
receive additional video data;
process the additional video data, with the trained depth estimation model, to generate new estimated distances;
and
perform one or more actions based on the new estimated distances.

11 more elements

A non-transitory computer-readable medium storing a set of instructions, the set of instructions comprising:

one or more instructions that, when executed by one or more processors of a device, cause the device to:
receive video data that includes base video frames and shifted video frames;
select a base video frame and a shifted video frame from the video data;
estimate a base homography that maps ground points in the base video frame to a target plane with a metric reference system;
estimate a shifted homography that maps ground points in the shifted video frame to the target plane with the metric reference system;
process the base video frame and the shifted video frame, with an object detection model, to detect objects in the base video frame and the shifted video frame and to generate bounding boxes for the objects;
utilize the base homography, the shifted homography, and the bounding boxes to calculate ground truth distances to the objects of the base video frame and the shifted video frame;
utilize a depth estimation model to calculate estimated distances to the objects;
calculate differences between the ground truth distances and the estimated distances;
determine, based on the differences, the depth estimation model needs to be trained and generate a trained depth estimation model;
and
perform one or more actions based on the trained depth estimation model.

9 more elements

Dependencies are inferred from the claim text.