Precision prediction of time-of-flight sensor measurements

A method comprising:

• illuminating one or more targets in a scene proximal to a time-of-flight (TOF) camera with a modulated signal from a light transmitter, the modulated light signal being scattered off the one or more targets in the scene such that a reflected modulated signal is received at a light receiver and processed by a sensor chip to generate a video sequence of the scene comprising a plurality of frames, wherein each of the plurality of frames includes a pixel array representing the scene;
• receiving, for each of the plurality of frames of the video sequence, depth data associated with a respective frame, the depth data comprising a plurality of correlation samples based on the reflected signal;
• receiving, from the sensor chip, for each respective frame, grayscale image data corresponding to the respective frame;
• determining, for each respective frame, a modulation amplitude of the reflected modulated signal based on the plurality of correlation samples;
• determining, for each respective frame, one or more parameters associated with at least one of the depth data and the grayscale image data;
• determining, for each respective frame, a reliability of the depth data comprising an estimated precision of TOF measurement(s) for each pixel of the pixel array, based on the modulation amplitude of the reflected modulated signal, the grayscale image data, and the one or more parameters associated with at least one of the depth data and the grayscale image data;
• generating, for each respective frame, a confidence map comprising mask values for each pixel of the pixel array by comparing the estimated precision for each pixel to a precision threshold;
• and
• masking, for each respective frame, one or more pixels of the pixel array based on the mask values.

A non-transitory computer-readable medium having stored thereon instructions which, when executed by one or more processors, cause the one or more processors to:

• command a time-of-flight (TOF) camera having a light transmitter, a light receiver, and a sensor chip, to illuminate one or more targets in a scene proximal to the TOF camera with a modulated signal from the light transmitter, the modulated light signal being scattered off the one or more targets such that a reflected modulated signal is received at the light receiver and processed by the sensor chip to generate a video sequence of the scene comprising a plurality of frames, wherein each of the plurality of frames includes a pixel array representing the scene;
• receive for each of the plurality of frames of the video sequence, depth data associated with a respective frame, the depth data comprising a plurality of correlation samples based on the reflected modulated signal associated with the frame;
• receive, from the sensor chip, for each respective frame, grayscale image data corresponding to the respective frame;
• determine, for each respective frame, a modulation amplitude of the reflected modulated signal based on the plurality of correlation samples;
• determine, for each respective frame, one or more parameters associated with at least one of the depth data and the grayscale image data;
• determine, for each respective frame, a reliability of the depth data comprising an estimated precision of TOF measurement(s) for each pixel of the pixel array, based on the modulation amplitude of the reflected modulated signal, the grayscale image data, and the one or more parameters associated with at least one of the depth data and the grayscale image data;
• generate, for each respective frame, a confidence map comprising mask values for each pixel of the pixel array by comparing the estimated precision for each pixel to a precision threshold;
• and
• mask, for each respective frame, one or more pixels of the pixel array based on the mask values.