US 2019/0102044 A1 · Depth-Based Touch Detection

Abstract

Systems, methods, and computer readable media to improve the operation of detecting contact between a finger or other object and a surface are described. In general, techniques disclosed herein utilize a depth map to identify an object and a surface, and a classifier to determine when the object is touching the surface. Unlike the prior art, a measure of the object's “distance” is made relative to the surface and not the camera(s) thereby providing some measure of invariance with respect to camera pose. The object-surface distance measure can be used to construct an identifier or “feature vector” that, when applied to a classifier, generates an output indicative of whether the object is touching the surface. The classifier may be based on machine learning and can be trained off-line before run-time operations are commenced. In some embodiments, temporal filtering may be used to improve surface detection operations.

Claims20

Dependent claims

A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:

obtain a depth map of a scene having a surface, the depth map comprising a plurality of pixel values;
identify a first region of the depth map, the first region comprising a first plurality of pixel values indicative of an object other than the surface in the scene;
identify a surface region of the depth map based on the first region, the surface region comprising a second plurality of pixel values indicative of the surface;
normalize the first region based on the surface region, wherein each normalized pixel value in the normalized first region is indicative of a distance relative to the surface;
generate an identifier based on the normalized first region;
apply the identifier to a classifier;
obtain an output from the classifier based on the applied identifier;
and
perform a first affirmative operation when the classifier output is indicative of a touch between the object and the surface.

6 more elements

A depth-based touch detection method, comprising:

obtaining a depth map of a scene having a surface, the depth map comprising a plurality of pixel values;
identifying a first region of the depth map, the first region comprising a first plurality of pixel values indicative of an object other than the surface in the scene;
identifying a surface region of the depth map based on the first region, the surface region comprising a second plurality of pixel values indicative of the surface;
normalizing the first region based on the surface region, wherein each normalized pixel value in the normalized first region is indicative of a distance relative to the surface;
generating an identifier based on the normalized first region;
applying the identifier to a classifier;
obtaining an output from the classifier based on the applied identifier;
and
performing a first affirmative operation when the classifier output is indicative of a touch between the object and the surface.

6 more elements

An electronic system comprising:

memory;
a display unit operatively coupled to the memory;
a depth map capture system operatively coupled to the memory;
and
one or more processors operatively coupled to the memory, the display unit and the depth map capture system, the one or more processors configured to execute computer instructions stored in the memory to cause the electronic system to
display a scene on the display unit, the scene including a surface;
obtain, via the depth map capture system, a depth map of the scene, the depth map comprising a plurality of pixel values stored in the memory,
identify a first region of the depth map, the first region comprising a first plurality of pixel values indicative of an object other than the surface in the scene,
identify a surface region of the depth map based on the first region, the surface region comprising a second plurality of pixel values indicative of the surface,
normalize the first region based on the surface region, wherein each normalized pixel value in the normalized first region is indicative of a distance relative to the surface,
generate an identifier based on the normalized first region,
apply the identifier to a classifier,
obtain an output from the classifier based on the applied identifier, and
perform a first affirmative operation when the classifier output is indicative of a touch between the object and the surface.

11 more elements

Dependencies are inferred from the claim text.

Depth-Based Touch Detection

Life of this patent

Depth-Based Touch Detection

Life of this patent

Abstract

Claims20