# US 8,938,124 B2 · Computer vision based tracking of a hand

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,938,124 B2
- Document ID: US8938124B2
- Status: Granted
- Kind: B2
- Application number: 13/926,445
- Filed: 2013-06-25
- Published: 2013-11-14
- Granted: 2015-01-20
- Earliest claimed: 2012-05-10
- Assignees: Pointgrab Ltd.
- Applicants: Pointgrab Ltd.
- Inventors: Eran Eilat; Amir Kaplan; Ovadya Menadeva; Haim Perski
- Web page: https://patentgrep.com/patent/US8938124B2

## CPC

- G06V 40/28: {Recognition of hand or arm movements, e.g. recognition of deaf sign language (static hand signs G06V40/113)}
- G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)}
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)}
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method for computer vision based tracking of a hand may include receiving a sequence of images, the images including at least one object having a shape of a hand. A first selected feature is tracked from within the hand shaped object. Shape recognition algorithms are applied at a suspected location of the hand shaped object in an image from the sequence of images to detect a shape of a hand in the image and a second feature from within the detected shape of the hand is then selected and tracked, thereby providing verification and updating of the location of the hand shaped object.

## Claims

### Claim 1

A method for computer vision based tracking of a hand, the method comprising:
i) receiving a sequence of images of a field of view, said images comprising at least one object having a shape of a hand;
ii) tracking at least one first selected feature from within the shape of the object;
iii) applying a shape recognition algorithm at a suspected location of the object in an image from the sequence of images to detect a newly detected shape of a hand in the image;
iv) selecting at least one newly selected feature to be tracked from within the newly detected shape of the hand; and
v) tracking the newly selected feature.

### Claim 2 (depends on 1)

The method of claim 1 comprising repeating steps (iii)-(v) to track a hand throughout a plurality of images.

### Claim 3 (depends on 2)

The method of claim 2 comprising:
assigning a probability grade to a detected shape for each of the plurality of images;
using the probability grades to obtain a combined probability grade; and
determining that no shape of a hand is detected if the combined probability grade is below a predetermined threshold.

### Claim 4 (depends on 1)

The method of claim 1 comprising terminating tracking if no shape of a hand is detected at the suspected location.

### Claim 5 (depends on 1)

The method of claim 1 wherein the first and newly selected features comprise pixels, patterns or vectors.

### Claim 6 (depends on 1)

The method of claim 1 comprising detecting a size of the object having a shape of the hand and determining the suspected location based on the detected size.

### Claim 7 (depends on 1)

The method of claim 1 wherein tracking the first feature and the newly selected feature comprises tracking a hand, the method further comprising controlling a device according to the tracking of the hand.

### Claim 8 (depends on 7)

The method of claim 7 wherein controlling the device comprises controlling movement of an icon on a display according to movement of the hand.

### Claim 9 (depends on 8)

The method of claim 8 wherein the icon is a cursor.

### Claim 10 (depends on 1)

The method of claim 1 comprising controlling a display of a device according to the detection of the newly detected shape of the hand.

### Claim 11 (depends on 10)

The method of claim 10 wherein controlling a display of the device comprises changing a color or transparency of an icon on the display.

### Claim 12 (depends on 11)

The method of claim 11 wherein the icon is a symbol or a representation of a hand.

### Claim 13 (depends on 1)

The method of claim 1 comprising
classifying a first set of selected features to groups;
determining multiple suspected locations according to locations of the groups;
applying a shape recognition algorithm at each suspected location;
determining, based on the shape recognition algorithm, a shape at one of the suspected locations, with the highest probability of being a shape of a hand;
selecting at least one newly selected feature to be tracked from within the determined shape; and
tracking the newly selected feature.

### Claim 14 (depends on 1)

The method of claim 1 wherein the newly detected shape of a hand comprises a shape of a hand in a specific posture.

### Claim 15

A method for computer vision based tracking of a hand, the method comprising:
receiving a sequence of images, said images comprising at least one object having a shape of a hand;
tracking a feature from within the shape of the object;
applying a shape recognition algorithm at a suspected location of the object in an image from the sequence of images to detect a newly detected shape of the object;
if the newly detected shape of the object is a shape of a hand then selecting a newly selected feature to be tracked from within the newly detected shape of the object and tracking the newly selected feature; and
terminating tracking if the shape of the newly detected shape of the object is not a hand shape.

### Claim 16 (depends on 15)

The method of claim 15 comprising moving an icon on a display according to the tracking of the newly selected feature.

### Claim 17 (depends on 16)

The method of claim 16 comprising stopping movement of the icon when tracking is terminated.

### Claim 18 (depends on 17)

The method of claim 17 comprising resuming movement of the icon according to tracking of the newly selected feature, the location of the icon when resuming movement being the last location of the icon when movement of the icon was stopped.

### Claim 19

A system for computer vision based tracking of a hand, the system comprising
a processor configured to:
detect within an image from a sequence of images an object having a shape of a hand;
track at least one first selected feature from within the shape of the object;
detect a newly detected shape of a hand at a suspected location of the object;
select at least one newly selected feature to be tracked from within the newly detected shape of the hand; and
track the newly selected feature; and
control a device based on the tracking of the newly selected feature.
