Back to results
US 2020/0050353 A1Published application
Robust gesture recognizer for projector-camera interactive displays using deep neural networks with a depth camera
AssigneeFuji Xerox Co., Ltd.
InventorsPatrick CHIU, Chelhwon KIM
Life of this patent
Estimating the term…
Abstract
Systems and methods described herein utilize a deep learning algorithm to recognize gestures and other actions on a projected user interface provided by a projector. A camera that incorporates depth information and color information records gestures and actions detected on the projected user interface. The deep learning algorithm can be configured to be engaged when an action is detected to save on processing cycles for the hardware system.
Claims18
A system, comprising:
- a projector system, configured to project a user interface (UI) directly outwards onto a real world location;
- a camera system, configured to record interactions on the projected user interface;
- and
- a processor, configured to:
- upon detection of an interaction recorded by the camera system, determine execution of a command for action based on an application of a deep learning algorithm trained to recognize gesture actions from the interaction recorded by the camera system.
2 more elements
A system, comprising:
- a projector system, configured to project a user interface (UI) directly outwards onto a real world location;
- a camera system, configured to record interactions on the projected user interface;
- and
- a processor, configured to:
- upon detection of an interaction recorded by the camera system:
- compute an optical flow for a region within the projected UI for color channels and depth channels of the camera system;
- apply a deep learning algorithm on the optical flow to recognize a gesture action with a UI widget, the deep learning algorithm trained to recognize gesture actions from the optical flow;
- and
- for the gesture action being recognized, execute a command corresponding to the recognized gesture action and the UI widget.
6 more elements
A device, comprising:
- a projector system, configured to project a user interface (UI) directly outwards onto a real world location;
- a camera system, configured to record interactions on the projected user interface;
- and
- a special purpose hardware processor, configured to apply a deep learning algorithm trained to recognize gesture actions from an interaction recorded by the camera system upon detection of the interaction recorded by the camera system, the special purpose hardware processor configured to:
- for a non-detection of the interaction, not applying the deep learning algorithm;
- and for a detection of the interaction, determine execution of a command for action based on an application of the deep learning algorithm.
3 more elements
Dependencies are inferred from the claim text.