Customizable gesture commands
AssigneeHewlett-Packard Development Company, L.P.
InventorsAndrew Pornthep Pechin, Kishore Venkat Rao Goka, Nhu Quynh Pham Nguyen, Srinivas Bommareddy, Raghavendra Balavalikar Krishnamurthy
Life of this patent
Estimating the term…
Abstract
A method implements customizable gesture commands. The method includes constructing a set of gesture containers and training a machine learning model, for a gesture container of the set of gesture containers, to detect a performance of a video gesture. The method further includes detecting the performance of the video gesture, from a gesture container of the set of gesture containers, in a video stream using the machine learning model. The method further includes executing, in response to detecting the video gesture, a command assigned to the video gesture by the gesture container.
Claims20
A method, comprising operations of:
- storing, by a computing system to a gesture container, training data comprised of:
- a video gesture, and
- a command linked to the video gesture;
- training, by the computing system when the computing system processes the training data, a machine learning model to detect the video gesture;
- detecting, by the computing system when the computing system trains the machine learning model, the video gesture in a live media stream;
- and
- executing, by the computing system in response to the computing system detecting the video gesture in the live media stream, the command,
- wherein training the machine learning model comprises:
- generating outputs from the training data using the machine learning model, and
- updating the machine learning model by measuring error of the outputs generated from the training data, and
- wherein detecting the video gesture further comprises:
- identifying, in one or more input images using a keypoint model of the machine learning model, locations of anatomical landmarks on a human body,
- processing a gesture model of the machine learning model to generate a gesture output from the locations, and
- identifying, using the gesture output, the video gesture in the live media stream.
A non-transitory computer readable medium comprising computer readable program code, the computer readable program code when executed by a processor causing a computing system to perform a method comprising operations of:
- storing, by the computing system to a gesture container, training data comprised of:
- a video gesture, and
- a command linked to the video gesture;
- training, by the computing system when the computing system processes the training data, a machine learning model to detect the video gesture;
- detecting, by the computing system when the computing system trains the machine learning model, the video gesture in a live media stream;
- and
- executing, by the computing system in response to the computing system detecting the video gesture in the live media stream, the command,
- wherein training the machine learning model comprises:
- generating outputs from the training data using the machine learning model, and
- updating the machine learning model by measuring error of the outputs generated from the training data, and
- wherein detecting the video gesture further comprises:
- identifying, in one or more input images using a keypoint model of the machine learning model, locations of anatomical landmarks on a human body,
- processing a gesture model of the machine learning model to generate a gesture output from the locations, and
- identifying, using the gesture output, the video gesture in the live media stream.
A computing system comprising:
- a processing circuitry electrically connected to memory,
- wherein the memory is to store training data comprised of:
- a video gesture, and
- a command linked to the video gesture, and
- wherein the processing circuitry is to:
- extract, from a live media stream, an image,
- process, to train a machine learning model, the training data,
- identify, in the image when the processing circuitry trains a keypoint model of the machine learning model, locations of anatomical landmarks on a human body,
- generate, from the locations when the processing circuitry trains a gesture model of the machine learning model, a gesture output,
- update, when the processing circuitry generates the gesture output, machine learning model by measuring error of the gesture output,
- identify, when the processing circuitry generates the gesture output, the video gesture in the live media stream, and
- execute, when the processing circuitry identifies the video gesture, the command.
Dependencies are inferred from the claim text.