US 10,897,609 B2Granted

Systems and methods for multiscopic noise reduction and high-dynamic range

AssigneeGoogle Inc.

InventorsJonathan Tilton Barron, Stephen Joseph DiVerdi, Ryan Geiss

Life of this patent

Estimating the term…

Loading patent…

Abstract

The present disclosure relates to methods and systems that may improve and/or modify images captured using multiscopic image capture systems. In an example embodiment, burst image data is captured via a multiscopic image capture system. The burst image data may include at least one image pair. The at least one image pair is aligned based on at least one rectifying homography function. The at least one aligned image pair is warped based on a stereo disparity between the respective images of the image pair. The warped and aligned images are then stacked and a denoising algorithm is applied. Optionally, a high dynamic range algorithm may be applied to at least one output image of the aligned, warped, and denoised images.

Claims18

Dependent claims

A system comprising:

a plurality of image capture systems, wherein the plurality of image capture systems comprises:
a first image capture system operable to capture first image data, wherein the first image capture system comprises a first image sensor optically coupled to first optics, and wherein the first image sensor and the first optics are associated with a first focal length;
and
a second image capture system operable to capture second image data, wherein the second image capture system comprises a second image sensor optically coupled to second optics, and wherein the second image sensor the second optics are associated with a second focal length;
wherein the first focal length is different than the second focal length, and wherein the first image data and the second image data comprise burst image data having at least one image pair;
and
a controller comprising a memory and a processor, wherein the memory is configured to contain instructions, and wherein the processor is configured to execute the instructions to perform operations, the operations comprising:
receiving a desired image characteristic, wherein the desired image characteristic comprises a desired field of view;
receiving burst image data from the plurality of image capture systems;
adjusting at least one of the first image data or the second image data based on the desired image characteristic, wherein adjusting at least one of the first image data or the second image data based on the desired image characteristic comprises generating a depth map based on the desired field of view;
generating at least one output image based on the adjusted first image data or the adjusted second image data.

8 more elements

A method comprising:

receiving, by a computing system, burst image data from a plurality of image capture systems, wherein the burst image data comprises first image data captured by a first image capture system associated with a first focal length and second image data captured by a second image capture system associated with a second focal length, wherein the first focal length is different than the second focal length, and wherein the first image data and the second image data comprise burst image data having at least one image pair;
receiving a desired image characteristic, wherein the desired image characteristic comprises a desired field of view;
adjusting at least one of the first image data or the second image data based on the desired image characteristic, wherein adjusting at least one of the first image data or the second image data based on the desired image characteristic comprises generating a depth map based on the desired field of view;
and
generating at least one output image based on the adjusted first image data or the adjusted second image data.

2 more elements

A non-transitory computer readable medium having stored therein instructions executable by a computing system to cause the computing system to perform functions comprising:

receiving, by a computing system, burst image data from a plurality of image capture systems wherein the burst image data comprises first image data captured by a first image capture system associated with a first focal length and second image data captured by a second image capture system associated with a second focal length, wherein the first focal length is different than the second focal length, and wherein the first image data and the second image data comprise burst image data having at least one image pair;
receiving a desired image characteristic, wherein the desired image characteristic comprises a desired field of view;
adjusting at least one of the first image data or the second image data based on the desired image characteristic, wherein adjusting at least one of the first image data or the second image data based on the desired image characteristic comprises generating a depth map based on the desired field of view;
and
generating at least one output image based on the adjusted first image data or the adjusted second image data.

2 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/680,474Filed 11 Nov 2019 · continuation
Priority: 7 Dec 2015
Published: US 2020/0084429 A112 Mar 2020
Granted: 19 Jan 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification23 codes

H04N 13/239
Using stereoscopic image cameras: using two two-dimensional [2D] image sensors having a relative position equal to or related to the interocular…
G06T 5/10
Image enhancement or restoration: using non-spatial domain filtering
G06T 5/70
Image enhancement or restoration: denoising; Smoothing
G06T 5/90
Image enhancement or restoration: dynamic range modification of images or parts thereof

Family

Application14/961,102Parent of this continuation · filed 7 Dec 2015
Application15/676,145Parent of this continuation · filed 14 Aug 2017
US 9,762,893 B2Parent of this continuation · filed 12 Sep 2017
Application16/226,643Parent of this continuation · filed 20 Dec 2018
US 10,187,628 B2Parent of this continuation · filed 22 Jan 2019
US 2020/0084429 A1Pre-grant publication · published 12 Mar 2020
US 10,477,185 B2Parent of this continuation
US 2017/0163966 A1Published application
US 2017/0347088 A1Published application
US 2019/0124319 A1Published application
US 10,897,609 B2This patent

From the US filing history. Foreign family members aren’t listed.