US 2026/0301208 A1Published application

Systems and Methods for Estimating Depth and Visibility from a Reference Viewpoint for Pixels in a Set of Images Captured from Different Viewpoints

AssigneeAdeia Imaging LLC

InventorsFlorian Ciurea, Kartik Venkataraman, Gabriel Molina, Dan Lelescu

Life of this patent

Estimating the term…

Loading patent…

Abstract

Systems in accordance with embodiments of the invention can perform parallax detection and correction in images captured using array cameras. Due to the different viewpoints of the cameras, parallax results in variations in the position of objects within the captured images of the scene. Methods in accordance with embodiments of the invention provide an accurate account of the pixel disparity due to parallax between the different cameras in the array, so that appropriate scene-dependent geometric shifts can be applied to the pixels of the captured images when performing super-resolution processing. In a number of embodiments, generating depth estimates considers the similarity of pixels in multiple spectral channels. In certain embodiments, generating depth estimates involves generating a confidence map indicating the reliability of depth estimates.

Claims21

Dependent claims

A computer-implemented method comprising:

obtaining a set of images captured from different viewpoints using a processor configured by an image processing application;
selecting a reference image from the set of images, wherein the reference image comprises a reference viewpoint;
determining depth estimates for pixel locations in the reference image using at least a subset of the set of images, where a depth estimate for a given pixel location in the reference image is determined by:
dividing the reference image into spatial blocks, wherein each spatial block comprises at least one indicator pixel and at least one non-indicator pixel;
for each spatial block within the reference image:
generating at least one depth measurement for the at least one indicator pixel within the spatial block;
and
assigning at least one depth measurement to the at least one non-indicator pixel in the spatial block based upon the at least one depth measurement of the at least one indicator pixel;
and
determining a confidence metric for at least one pixel within each of the spatial blocks based on a corresponding depth measurement;
generating a depth map based on the depth measurements and the confidence metrics;
and
generating a confidence map using at least the depth measurements of the pixels within each of the spatial blocks.

10 more elements

A computer-implemented method comprising:

obtaining a set of images captured from different viewpoints using a processor configured by an image processing application;
selecting a reference image from the set of images, wherein the reference image comprises a reference viewpoint;
determining depth estimates for pixel locations in the reference image using at least a subset of the set of images, where a depth estimate for a given pixel location in the reference image is determined by:
dividing the reference image into spatial blocks, wherein each spatial block comprises at least one indicator pixel and at least one non-indicator pixel;
for each spatial block within the reference image:
generating at least one depth measurement for the at least one indicator pixel within the spatial block by:
identifying pixels as corresponding pixels in the at least a subset of the set of images that correspond to the indicator pixel in the reference image based upon expected disparity at a plurality of depths;
comparing the corresponding pixels identified in the subset of the set of images at each of the plurality of depths using a block-based similarity measure;
and
selecting the depth from the plurality of depths at which the identified corresponding pixels have the highest degree of similarity as a depth estimate for the indicator pixel in the reference image;
assigning at least one depth measurement to the at least one non-indicator pixel in the spatial block based upon the at least one depth measurement of the at least one indicator pixel;
and
determining a confidence metric for at least one pixel within each of the spatial blocks based on the depth measurements and the block-based similarity measure;
generating a depth map based on the depth measurements and the confidence metrics;
and
generating a confidence map using at least the depth measurements of the pixels within each of the spatial blocks.

13 more elements

A computer-implemented method comprising:

obtaining a set of images captured from different viewpoints using a processor configured by an image processing application;
normalizing the set of images, wherein the normalizing comprises:
utilizing calibration information to correct for photometric variations and scene-independent geometric distortions in the images in the set of images;
and
rectification of the images in the set of images;
selecting a reference image from the set of images, wherein the reference image comprises a reference viewpoint;
determining depth estimates for pixel locations in the reference image using at least a subset of the set of images, where a depth estimate for a given pixel location in the reference image is determined by:
dividing the reference image into spatial blocks, wherein each spatial block comprises at least one indicator pixel at a center of the spatial block and at least one non-indicator pixel;
for each spatial block within the reference image:
generating at least one depth measurement for the at least one indicator pixel within the spatial block by:
identifying pixels as corresponding pixels in the at least a subset of the set of images that correspond to the indicator pixel in the reference image based upon expected disparity at a plurality of depths;
comparing the corresponding pixels identified in the subset of the set of images at each of the plurality of depths using a census transform;
and
selecting the depth from the plurality of depths at which the identified corresponding pixels have the highest degree of similarity as a depth estimate for the indicator pixel in the reference image;
assigning at least one depth measurement to the at least one non-indicator pixel in the spatial block based upon the at least one depth measurement of the at least one indicator pixel;
and
determining a confidence metric for at least one pixel within each of the spatial blocks based on the depth measurements and the census transform;
generating a depth map based on the depth measurements and the confidence metrics;
and
generating a confidence map using at least the depth measurements of the pixels within each of the spatial blocks.

17 more elements

Dependencies are inferred from the claim text.

Facts

Application: 19/704,296Filed 10 Jun 2026
Priority: 21 Aug 2012
Published: 1 Oct 2026
Term ends: Not estimatedThe source records don’t support an estimate.

Classification18 codes

G06T 7/593
From multiple images: from stereo images
G02B 27/0075
Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00: with means for altering, e.g. increasing, the…
G06T 7/557
From multiple images: from light fields, e.g. from plenoptic cameras
G06T 7/85
Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration: stereo camera calibration

Family

Application61/691,666Us provisional application · filed 21 Aug 2012
Application61/780,906Us provisional application · filed 13 Mar 2013
Application13/972,881Continuation · filed 21 Aug 2013
Application14/144,458Continuation · filed 30 Dec 2013
Application14/329,754Continuation · filed 11 Jul 2014
Application14/526,392Continuation · filed 28 Oct 2014
Application14/988,680Continuation · filed 5 Jan 2016
Application15/858,974Continuation · filed 29 Dec 2017
Application16/537,369Continuation · filed 9 Aug 2019
Application17/162,542Continuation · filed 29 Jan 2021
Application18/434,684Continuation · filed 6 Feb 2024
Application19/187,897Continuation · filed 23 Apr 2025
Application19/704,296Continuation
US 10,380,752 B2Related patent
US 10,909,707 B2Related patent
US 11,941,833 B2Related patent
US 12,002,233 B2Related patent
US 12,437,432 B2Related patent
US 2014/0321712 A1Published application
US 2015/0049916 A1Published application
US 2016/0267672 A1Published application
US 2018/0211402 A1Published application
US 2019/0362515 A1Published application
US 2021/0150748 A1Published application
US 2024/0331181 A1Published application
US 2025/0252589 A1Published application
US 8,619,082 B1Related patent
US 8,780,113 B1Related patent
US 9,235,900 B2Related patent
US 9,240,049 B2Related patent
US 9,858,673 B2Related patent
US 2026/0301208 A1This patent

From the US filing history. Foreign family members aren’t listed.