US 12,717,421 B2 · Multi-camera, multimodal gesture detection

Abstract

A multi-camera, multimodal gesture detection system for augmented reality devices combines outputs from multiple image sensors, using different modalities, to generate fused gesture detection trigger signals and gesture detection release signals. The system processes fused trigger signals using a trigger suspension component, which suppresses false triggers based on hand velocity, position of a detected hand within a field of view of an image sensor, and visibility of key hand landmarks. The system processes fused release signals using a release suspension component, which suppresses false release signals based on hand velocity, position of a detected hand within a field of view of an image sensor, and visibility of key hand landmarks. This approach enhances gesture detection accuracy and reliability in challenging environments.

Claims20

Dependent claims

An augmented reality (AR) device configured to generate a final gesture detection signal for hand gesture detection, the AR device comprising:

a display;
a processor;
two or more image sensors;
a memory storing instructions thereon, which, when executed by the processor, cause the AR device to perform operations comprising:
generating a fused gesture detection signal by combining outputs from hand gesture detectors associated with different modalities, wherein each modality corresponds to a different type of image sensor, and wherein the hand gesture detectors analyze movements of a hand captured in images by the respective image sensors to detect potential hand gestures;
processing the fused gesture detection signal to generate the final gesture detection signal, by:
suppressing the fused gesture detection signal based on a determination that a velocity of the hand exceeds a predetermined velocity threshold.

4 more elements

A method for generating a final gesture detection signal for hand gesture detection in an augmented reality (AR) system, the method comprising:

generating a fused gesture detection signal by combining outputs from hand gesture detectors associated with different modalities, wherein each modality corresponds to a different type of image sensor, and wherein the hand gesture detectors analyze movements of a hand captured in images by respective image sensors to detect potential hand gestures;
processing the fused gesture detection signal to generate the final gesture detection signal, by:
suppressing the fused gesture detection signal based on a determination that a velocity of the hand exceeds a predetermined velocity threshold.

An augmented reality (AR) device configured to generate a final gesture detection signal for hand gesture detection, the AR device comprising:

means for generating a fused gesture detection signal by combining outputs from hand gesture detectors associated with different modalities, wherein each modality corresponds to a different type of image sensor, and wherein the hand gesture detectors analyze movements of a hand captured in images by the respective image sensors to detect potential hand gestures;
means for processing the fused gesture detection signal to generate the final gesture detection signal by suppressing the fused gesture detection signal based on a determination that a velocity of the hand exceeds a predetermined velocity threshold.

Dependencies are inferred from the claim text.

Multi-camera, multimodal gesture detection

Life of this patent

Multi-camera, multimodal gesture detection

Life of this patent

Abstract

Claims20