US 2026/0148405 A1 · FOUNDATION MODEL FOR ZERO-SHOT STEREO MATCHING

Abstract

Systems and methods are disclosed that use a Foundational Stereo Model to generate an output disparity map. The Foundational Stereo Model includes side-tuning adapters (STA) that utilize a vision transformer (ViT) and a convolutional neural network (CNN) to generate feature maps. Specifically, the CNN may be used to adapt the ViT-based monocular depth estimation network for the stereo setup, which synergizes the strengths of both CNN and ViT architectures. In addition, the Foundational Stereo Model includes an attentive hybrid cost filtering (AHCF) that uses two branches that also utilizes the advantages of both a transformer architecture and the CNN architecture. Furthermore, the Foundational Stereo Model may perform iterative refinement of an initial disparity map to obtain the output disparity map based on performing a convolutional gated recurrent unit (GRU) operation.

Claims29

Dependent claims

A computer-implemented method for using a Foundational Stereo Model to generate an output disparity map, comprising:

processing a first stereo image using a first side-tuning adapter (STA) to generate a first feature map associated with the first stereo image;
processing a second stereo image using a second STA to generate a second feature map associated with the second stereo image;
generating an initial hybrid cost volume based on the first feature map and the second feature map;
processing the initial hybrid cost volume using attentive hybrid cost filtering (AHCF) to generate a filtered hybrid cost volume;
and
generating the output disparity map based on the filtered hybrid cost volume.

3 more elements

A computer-implemented method for using a Foundational Stereo Model to generate an output disparity map, comprising:

processing a first stereo image using a first side-tuning adapter (STA) to generate a first feature map associated with the first stereo image, wherein the first STA comprises a first monocular depth estimation model and a first side-tuning convolutional neural network (CNN);
processing a second stereo image using a second STA to generate a second feature map associated with the second stereo image, wherein the second STA comprises a second monocular depth estimation model and a second side-tuning CNN;
generating an initial hybrid cost volume based on the first feature map and the second feature map;
and
generating the output disparity map based on the initial hybrid cost volume.

2 more elements

Dependencies are inferred from the claim text.

Foundation model for zero-shot stereo matching

Life of this patent

Foundation model for zero-shot stereo matching

Life of this patent

Abstract

Claims29