# US 10,503,270 B2 · Method of training neural networks for hand pose detection

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,270 B2
- Document ID: US10503270B2
- Status: Granted
- Kind: B2
- Application number: 16/436,588
- Filed: 2019-06-10
- Published: 2019-10-10
- Granted: 2019-12-10
- Earliest claimed: 2015-12-15
- Assignees: Purdue Research Foundation
- Applicants: Purdue Research Foundation
- Inventors: Ayan Sinha; Chiho Choi; Joon Hee Choi; Karthik Ramani
- Web page: https://patentgrep.com/patent/US10503270B2

## CPC

- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06N 20/00: Machine learning
- G06V 10/454: {Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]}
- G06V 20/653: {by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces}
- G06V 30/194: References adjustable by an adaptive method, e.g. learning
- G06V 40/11: {Hand-related biometrics; Hand pose recognition}
- G06V 40/113: {Recognition of static hand signs}
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method for training a hierarchy of trained neural networks for hand pose detection includes training a first neural network to generate a first plurality of activation features that classify an input depth map data corresponding to a hand based on a wrist angle of the hand, the training using a plurality of depth maps of a hand with predetermined wrist angles as inputs to the first neural network during the training, and storing the first neural network in a memory after the training for use in classifying an additional depth map corresponding to a hand based on an angle of a wrist of the hand in the additional depth map.

## Claims

### Claim 1

A method for training a hierarchy of trained neural networks for hand pose detection comprising:
training, with a processor, a first neural network to generate a first plurality of activation features that classify an input depth map data corresponding to a hand based on a wrist angle of the hand, the training using a plurality of depth maps of a hand with predetermined wrist angles as inputs to the first neural network during the training; and
storing, with the processor, the first neural network in a memory after the training for use in classifying an additional depth map corresponding to a hand based on an angle of a wrist of the hand in the additional depth map.

### Claim 2 (depends on 1)

The method of claim 1 further comprising:
generating, with the processor, the plurality of training depth maps prior to training of the first neural network, the generating comprising:
generating, with the processor, a synthetic hand model in a predetermined pose based on hand model data stored in the memory and a predetermined plurality of constraints for an angle of at least one joint in the synthetic hand model;
generating, with the processor, a depth map of the hand model in the predetermined pose; and
storing, with the processor, the depth map with the plurality of depth maps in the memory.

### Claim 3 (depends on 2)

The method of claim 2 further comprising:
applying, with the processor, Gaussian noise to the depth map of the hand model in the predetermined pose prior to storing the depth map in the memory.

### Claim 4 (depends on 1)

The method of claim 1 further comprising:
providing, with the processor, one depth map in the plurality of depth maps in the training data to the first neural network after the training to generate a first plurality of activation features corresponding to the one depth map; and
storing, with the processor, the first plurality of activation features and a hand pose parameter of the hand in the one depth map in a database in the memory to map the first plurality of activation features to the hand pose parameter.

### Claim 5 (depends on 4)

The method of claim 4, wherein the hand pose parameter is a wrist angle parameter for the hand in the one depth map.

### Claim 6 (depends on 1)

The method of claim 1 further comprising:
identifying, with the processor, a first subset of the plurality of depth maps that correspond to a first predetermined range of wrist angles;
training, with the processor, a second neural network to generate a second plurality of activation features that classify the input depth map data corresponding to the hand based on an angle of at least one joint in a finger of the hand, the training using the first subset of the plurality of depth maps of the hand as inputs to the second neural network during the training; and
storing, with the processor, the second neural network in the memory after the training for use in classifying the additional depth map corresponding to the hand based on an angle of the at least one joint in the finger of the hand in the additional depth map.

### Claim 7 (depends on 6)

The method of claim 6 further comprising:
identifying, with the processor, a second subset of the plurality of depth maps that correspond to a second predetermined range of wrist angles, the second predetermined range of wrist angles being different than the first predetermined range of wrist angles;
training, with the processor, a third neural network to generate a third plurality of activation features that classify the input depth map data corresponding to the hand based on an angle of the at least one joint in the finger of the hand, the training using the second subset of the plurality of depth maps of the hand as inputs to the third neural network during the training; and
storing, with the processor, the third neural network in the memory after the training for use in classifying the additional depth map corresponding to the hand based on an angle of the at least one joint in the finger of the hand in the additional depth map.