US 11,636,656 B1 · Depth rate up-conversion

Abstract

Various implementations disclosed herein include devices, systems, and methods that create additional depth frames where a depth camera runs at a lower frame rate than a light intensity camera. Rather than upconverting the depth frames by simply repeating a previous depth camera frame, additional depth frames are created by adjusting some of the depth values of a prior frame based on the RGB camera data (e.g., by “dragging” depths from their positions in the prior depth frame to new positions for a new frame). Specifically, a contour image is generated, and changes in the contour image are used to determine how to adjust (e.g., drag) the depth values for the additional depth frames. The contour image may be based on a mask (e.g., occlusions masks identifying where the hand occludes the virtual cube).

Claims20

Dependent claims

A method comprising:

at an electronic device having a processor:
receiving a first sequence of images corresponding to light intensities of a physical environment and a second sequence of images corresponding to depths of the physical environment, wherein the first sequence has a first rate and the second sequence has a second rate that differs from the first rate;
generating a contour image for a first frame of the first sequence of images;
adjusting the contour image based on a second frame of the first sequence of images;
and
determining additional depth images corresponding to depths of the physical environment based on the adjusting of the contour image.

3 more elements

Dependencies are inferred from the claim text.

Depth rate up-conversion

Life of this patent

Depth rate up-conversion

Life of this patent

Abstract

Claims20