US 10,873,738 B2Granted

Multi-frame range gating for lighting-invariant depth maps for in-motion applications and attenuating environments

Assignee4D Intellectual Properties, LLC

InventorJames E. Retterath

Life of this patent

Estimating the term…

Loading patent…

Abstract

Range gated imaging systems utilize short light pulses and synchronized detector cycles to image objects that are contained within a distance range. Single-image range gating allows depth estimation for static scenes, but this technique is not well-suited for in-motion applications or attenuating environments. In various embodiments, a multi-frame range gating system uses a series of range gate frames that are produced within a brief timing window, whereby each range-gate frame illuminates a different lighting-invariant depth range within the field of view. Multi-frame range gate cycles of as little as three frames are sufficient to produce precise lighting-invariant depth information for each pixel in a sensor for in-motion applications. Lighting-invariant depth information is produced with multi-frame range gate cycles of as little as four frames for in-motion applications for attenuating environments like fog, rain, dust, smoke, smog, and underwater.

Claims32

Dependent claims

An active image sensor system configured to generate a lighting-invariant depth map of a scene comprising:

at least one emitter configured to emit a set of light pulses toward the scene;
an array of detectors configured to receive light for a field of view that includes at least a portion of the scene;
control circuitry operably coupled to the at least one emitter and the array of detectors and configured to cause the at least one emitter to begin to emit the set of light pulses at a first time and to cause the array of detectors to begin to receive light for the field of view and store in one or more frame buffers at least three successive frames of data as a multi-frame representation of the field of view during an emitter/detector cycle that includes at least a second time, a third time and a fourth time after the first time, wherein the control circuitry is configured to vary an elapsed time during the emitter/detector cycle between the first time and the second time, between the first time and the third time and between the first time and the fourth time;
and
a processing system operably coupled to the control circuitry and the one or more frame buffers, the processing system configured to generate the lighting-invariant depth map by analyzing the frame buffers for each of a plurality of the emitter/detector cycles to process the at least three successive frames of data for each emitter/detector cycle to determine intensity values and to generate an additional frame of data as a two-dimensional representation of distance values to objects in the field of view by performing the following for each detector in the detector array:
analyze the at least three successive frames of data for the detector and determine a minimum non-zero intensity value due to ambient light, a maximum intensity value, and a frame of the at least three successive frames at which the maximum intensity value occurs;
and
determine a distance to an object based on the frame at which the maximum intensity value occurs.

5 more elements

An active image sensor system configured to generate a lighting-invariant depth map of a scene within an attenuating environment comprising:

at least one emitter configured to emit a set of light pulses toward the scene;
an array of detectors configured to receive light for a field of view that includes at least a portion of the scene, wherein the light includes a varying amount of light due to ambient light and to light pulse reflections from the attenuating environment;
control circuitry operably coupled to the at least one emitter and the array of detectors and configured to cause the at least one emitter to begin to emit the set of light pulses at a first time and to cause the array of detectors to begin to receive light for the field of view and store in one or more frame buffers at least five successive frames of data as a multi-frame representation of the field of view during an emitter/detector cycle that includes at a second time, a third time, a fourth time, a fifth time and a sixth time after the first time, wherein the control circuitry is configured to vary an elapsed time during the emitter/detector cycle between the first time and the second time, between the first time and the third time, between the first time and the fourth time, between the first time and the fifth time and between the first time and the sixth time to generate the at least five successive frames of data;
and
a processing system operably coupled to the control circuitry and the one or more frame buffers to generate the lighting-invariant depth map by analyzing the frame buffers for each of a plurality of the emitter/detector cycles to process the at least five successive frames of data for each emitter/detector cycle to determine intensity values and to generate an additional frame of data as a two-dimensional representation of distance values to objects in the field of view by performing the following for each detector in the detector array:
analyze the at least five successive frames of data for the detector and determine a minimum intensity value due to ambient light, a maximum intensity value due to light reflected from the attenuating environment, and a frame of the at least five successive frames at which the minimum intensity value occurs;
and
determine a distance to an object based on the frame at which the minimum intensity value occurs.

5 more elements

An active image sensor system configured to generate a depth map of a scene within an attenuating environment comprising:

at least one emitter configured to emit a set of light pulses toward the scene;
an array of detectors configured to receive light for a field of view that includes at least a portion of the scene, wherein the light includes a varying amount of light due to ambient light and to light pulse reflections from the attenuating environment;
control circuitry operably coupled to the at least one emitter and the array of detectors and configured to cause the at least one emitter to begin to emit the set of light pulses at a first time and to cause the array of detectors to begin to receive light for the field of view and store in one or more frame buffers at least four successive frames of data as a multi-frame representation of the field of view during an emitter/detector cycle that includes at a second time, a third time, a fourth time and a fifth time after the first time, wherein the control circuitry is configured to vary an elapsed time during the emitter/detector cycle between the first time second time, between the first time and the third time, between the first time and the fourth time, and between the first time and the fifth time to generate the at least four successive frames of data;
and
a processing system operably coupled to the control circuitry and the one or more frame buffers to generate the depth map by analyzing the frame buffers for each of a plurality of the emitter/detector cycles to process the at least five successive frames of data for each emitter/detector cycle to determine intensity values and to generate an additional frame of data as a two-dimensional representation of distance values to objects in the field of view by performing the following for each detector in the detector array:
analyze the at least four successive of data for the detector and determine a minimum intensity value due to ambient light, a maximum intensity value due to light reflected from the attenuating environment, and a frame of the at least four successive frames, exclusive of a first frame of the at least four successive frames, at which the minimum intensity value occurs;
and
determine a distance to an object based on the frame at which the minimum intensity value occurs.

5 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/537,305Filed 9 Aug 2019 · division
Priority: 3 Mar 2016
Published: US 2020/0036958 A130 Jan 2020
Granted: 22 Dec 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification12 codes

H04N 23/671
Focus control based on electronic image sensor signals: in combination with active ranging signals, e.g. using light or sound signals emitted…
G01S 7/4863
Circuits for detection, sampling, integration or read-out: detector arrays, e.g. charge-transfer gates
G01S 17/18
Using transmission of interrupted, pulse-modulated waves: wherein range gates are used
G01S 17/894
For mapping or imaging: three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver…

Family

Application15/059,811Parent of this continuation · filed 3 Mar 2016
Application15/853,222Parent of this continuation · filed 22 Dec 2017
US 9,866,816 B2Parent of this continuation · filed 9 Jan 2018
Application16/167,196Parent of this division · filed 22 Oct 2018
US 10,298,908 B2Parent of this continuation · filed 21 May 2019
US 2020/0036958 A1Pre-grant publication · published 30 Jan 2020
US 10,382,742 B2Parent of this division
Application16/167,196Parent of this continuation
US 2017/0257617 A1Published application
US 2018/0295344 A1Published application
US 2019/0058867 A1Published application
US 10,873,738 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key