# US 2013/0278501 A1 · Systems and methods of identifying a gesture using gesture data compressed by principal joint variable analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0278501 A1
- Document ID: US20130278501A1
- Status: Published application
- Kind: A1
- Application number: 13/838,294
- Filed: 2013-03-15
- Published: 2013-10-24
- Earliest claimed: 2012-04-18
- Assignees: Arb Labs Inc.
- Applicants: Arb Labs Inc.
- Inventors: Adrian BULZACKI
- Web page: https://patentgrep.com/patent/US20130278501A1

## CPC

- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/847: Cooperative playing, e.g. requiring coordinated actions from several players to achieve a common goal
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06F 18/2135: based on approximation criteria, e.g. principal component analysis
- G06F 18/2413: based on distances to training or reference patterns
- G06V 20/40: in video content (extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44)
- G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16)
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- A63F 2300/1087: comprising photodetecting means, e.g. a camera
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Systems and method described herein present techniques for identifying a gesture using gesture data compressed by principal joint variable analysis. A classifier of a gesture recognition system may receive a frame comprising a set of gesture data points identifying locations of body parts of a subject. The classifier may determining that a subset of the set of gesture data points is sufficient to recognize a first gesture. The subset may be stored into a database in reference to the first gesture. A recognizer may receive a new frame of new gesture data points identifying locations of body parts of a new subject. The recognizer may recognize that the gesture of the new subject corresponds to the first gesture responsive to comparing at least one new gesture data point from the new frame to at least one gesture data point of the subset.

## Claims

### Claim 1

1) A method of identifying a gesture of a subject based on self-referenced gesture data compressed by principal joint variable analysis, the method comprising:
receiving, by a classifier of a system, a frame comprising a set of gesture data points, each gesture data point identifying a location of a part of a body of a subject with respect to a reference point on the body of the subject;
determining, by the classifier, that a subset of the set of gesture data points is sufficient to recognize a first gesture;
storing, into a database of the system, the subset in reference to the first gesture;
receiving, by a recognizer of the system, a new frame of new gesture data points, each new gesture data point identifying a location of a part of a body of a new subject with respect to a reference point on the body of the new subject; and
recognizing, by the recognizer, that the gesture of the new subject corresponds to the first gesture responsive to comparing at least one new gesture data point from the new frame to at least one gesture data point of the subset.

### Claim 2 (depends on 1)

2) The method of claim 1, further comprising determining, by the system, that the new subject performed the first gesture in response to the recognition.

### Claim 3 (depends on 1)

3) The method of claim 1, further comprising selecting, by the recognizer, new gesture data points within a subset of the new gesture data points for identifying the first gesture, the new gesture data points in the new subset corresponding to the gesture data points of the subset.

### Claim 4 (depends on 1)

4) The method of claim 1, further identifying, by the recognizer, within a first threshold of accuracy that at least a first new gesture data point of the frame matches at least one gesture data point of the subset.

### Claim 5 (depends on 4)

5) The method of claim 4, further identifying, by the recognizer, within a second threshold of greater accuracy than the first threshold of accuracy that at least a second new gesture data point matches at least a second gesture data point of the subset.

### Claim 6 (depends on 1)

6) The method of claim 1, further comprising:
receiving, by the classifier, together with the frame, a second frame comprising a second set of gesture data points, the second set of gesture data points having gesture data values different from the data values of the set of gesture data points, and
extracting a second subset of the gesture data points, the second subset comprising the same gesture data points as the subset.

### Claim 7 (depends on 1)

7) The method of claim 1, wherein the reference point includes a waist of the body of the subject.

### Claim 8 (depends on 1)

8) The method of claim 1, wherein the reference point includes the mid-point of the chest of the subject.

### Claim 9 (depends on 1)

9) The method of claim 1, further comprising comparing, by the recognizer, a new gesture data point corresponding to a location of a wrist of the new subject with respect to the new subject's waist to a gesture data point of the wrist of the subject with respect to the subject's waist.

### Claim 10 (depends on 1)

10) The method of claim 1, further comprising comparing, by the recognizer, a new gesture data point corresponding to a location of an elbow of the new subject with respect to the new subject's mid-point chest to a gesture data point corresponding to a location of an elbow of the subject with respect to the subject's waist.

### Claim 11

11) A system for using self-referenced gesture data compressed by principal joint variable analysis to identify a gesture of a subject, the system comprising:
a classifier of a system receiving a frame comprising a set of gesture data points, each gesture data point identifying a location of a part of a body of a subject with respect to a reference point on the body of the subject and determining that a subset of the set of gesture data points is sufficient to recognize a first gesture;
the system storing into a database of the system the subset in reference to the first gesture; and
a recognizer of the system receiving a new frame of new gesture data points, each new gesture data point identifying a location of a part of a body of a new subject with respect to a reference point on the body of the new subject and recognizing that the gesture of the new subject corresponds to the first gesture responsive to comparing at least one new gesture data point from the new frame to at least one gesture data point of the subset.

### Claim 12 (depends on 11)

12) The system of claim 11, wherein the system further determines that the new subject performed the first gesture in response to the recognition.

### Claim 13 (depends on 11)

13) The system of claim 11, wherein the recognizer further selects the new gesture data points within a subset of the new gesture data points for identifying the first gesture, the new gesture data points in the new subset corresponding to the gesture data points of the subset.

### Claim 14 (depends on 11)

14) The system of claim 11, wherein the recognizer further identifies, within a first threshold of accuracy, that at least a first new gesture data point of the frame matches at least one gesture data point of the subset.

### Claim 15 (depends on 14)

15) The system of claim 14, wherein the recognizer further identifies, within a second threshold of greater accuracy than the first threshold of accuracy, that at least a second new gesture data point matches at least a second gesture data point of the subset.

### Claim 16 (depends on 11)

16) The system of claim 11, wherein the classifier further receives, together with the frame, a second frame comprising a second set of gesture data points, the second set of gesture data points having gesture data values different from the data values of the set of gesture data points, and extracts a second subset of the gesture data points, the second subset comprising the same gesture data points as the subset.

### Claim 17 (depends on 11)

17) The system of claim 11, wherein the reference point includes a waist of the body of the subject.

### Claim 18 (depends on 11)

18) The system of claim 11, wherein the reference point includes the mid-point of the chest of the subject.

### Claim 19 (depends on 11)

19) The system of claim 11, wherein the recognizer compares a new gesture data point corresponding to a location of a wrist of the new subject with respect to the new subject's waist to a gesture data point of the wrist of the subject with respect to the subject's waist.

### Claim 20 (depends on 11)

20) The system of claim 11, wherein the recognizer compares a new gesture data point corresponding to a location of an elbow of the new subject with respect to the new subject's mid-point chest to a gesture data point corresponding to a location of an elbow of the subject with respect to the subject's waist.
