Semi-dense depth estimation from a dynamic vision sensor (DVS) stereo pair and a pulsed speckle pattern projector

An apparatus, comprising:

• a speckle pattern projector (SPP);
• a dynamic vision sensor (DVS) stereo pair;
• a processor;
• and
• a memory comprising instructions, which, when executed by the processor, cause the apparatus to:
• receive, a control signal of the SPP, the control signal indicating an on/off state, as a function of time, of a predetermined light pattern projected by the SPP on a field of view,
• receive, from each sensor of the dynamic vision sensor (DVS) stereo pair, an event stream of pixel intensity change data, wherein the event stream is time-synchronized with the control signal of the SPP and comprises a first portion associated with light from a scene in the field of view and a second portion associated with the predetermined light pattern projected by the SPP,
• perform projected light filtering on the event stream of pixel intensity change data for each sensor of the DVS stereo pair, to generate synthesized event image data, the synthesized event image data having one or more channels, each channel based on an isolated portion of the event stream of pixel intensity change data, and
• perform stereo matching on at least one channel of the synthesized event image data for each sensor of the DVS stereo pair to generate a depth map for at least a portion of the field of view.

A non-transitory computer-readable medium comprising program code, which, when executed by a processor, causes an electronic device to:

• receive, at the electronic device, a control signal of a speckle pattern projector (SPP), the control signal indicating an on/off state, as a function of time, of a predetermined light pattern projected by the SPP on a field of view,
• receive, from each sensor of a dynamic vision sensor (DVS) stereo pair, an event stream of pixel intensity change data, wherein the event stream is time-synchronized with the control signal of the SPP and comprises a first portion associated with light from a scene in the field of view and a second portion associated with the predetermined light pattern projected by the SPP,
• perform projected light filtering on the event stream of pixel intensity change data for each sensor of the DVS stereo pair, to generate synthesized event image data, the synthesized event image data having one or more channels, each channel based on an isolated portion of the event stream of pixel intensity change data, and
• perform stereo matching on at least one channel of the synthesized event image data for each sensor of the DVS stereo pair to generate a depth map for at least a portion of the field of view.