# US 2015/0146923 A1 · Systems and methods for tracking a model

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2015/0146923 A1
- Document ID: US20150146923A1
- Status: Published application
- Kind: A1
- Application number: 14/524,277
- Filed: 2014-10-27
- Published: 2015-05-28
- Earliest claimed: 2009-10-07
- Assignees: Microsoft Corporation
- Applicants: Microsoft Corporation
- Inventors: Johnny Chung Lee; Tommer Leyvand; Szymon Piotr Stachniak; Craig C. Peeper
- Web page: https://patentgrep.com/patent/US20150146923A1

## CPC

- H04N 13/20: Image signal generators
- G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)}
- G06T 7/251: {involving models}
- G06T 7/579: from motion
- G06T 15/00: Three-dimensional [3D] image rendering
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06T 2200/04: involving 3D image data
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30196: Human being; Person
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

## Claims

### Claim at position 1

-20. (canceled)

### Claim 21

A method for tracking a user, comprising:
receiving a first depth image, the first depth image being captured by a depth camera;
in response to identifying an estimated location of an extremity of a user in the first depth image, positioning a part of a model to correspond to the estimated location of the extremity in the first depth image; and
in response to determining that a location of an extremity has not been estimated, adjusting the part of the model to correspond to a portion of the depth image that is located close to the part of the model.

### Claim 22 (depends on 21)

The method of claim 21, wherein adjusting the part of the model to correspond to a portion of the depth image that is located close to the part of the model comprises adjusting the part of the model to a location of at least one voxel in the first depth image.

### Claim 23 (depends on 22)

The method of claim 22, wherein adjusting the part of the model to a location of at least one voxel in the first depth image comprises adjusting the part of the model to a location defined by an x-value, y-value, and depth value in the first depth image.

### Claim 24 (depends on 21)

The method of claim 21, wherein adjusting the part of the model to correspond to a portion of the depth image that is located close to the part of the model comprises adjusting the part of the model to correspond to a portion of the depth image that represents the user.

### Claim 25 (depends on 21)

The method of claim 21, further comprising:
in response to determining that a location of an extremity has not been estimated, positioning the model over the first depth image,
wherein adjusting the part of the model to correspond to a portion of the depth image that is located close to the part of the model comprises adjusting the part of the model to a portion of the depth image that is located closest to the part of the model.

### Claim 26 (depends on 21)

The method of claim 21, wherein adjusting the part of the model to correspond to a portion of the depth image that is located close to the part of the model comprises magnetizing the part of the model to a portion of the depth image.

### Claim 27 (depends on 26)

The method of claim 26, wherein magnetizing the part of the model to a portion of the depth image comprises magnetizing the part of the model to voxels defining a boundary or surface in the first depth image.

### Claim 28 (depends on 21)

The method of claim 21, wherein adjusting the part of the model to correspond to a portion of the depth image that is located close to the part of the model comprises adjusting the part of the model to correspond to voxels defining a boundary or surface in the first depth image.

### Claim 29 (depends on 21)

The method of claim 21, wherein adjusting the part of the model to correspond to a portion of the depth image comprises adjusting a joint of a skeletal model.

### Claim 30

A system, comprising:
at least one processor; and
computing memory communicatively coupled to the at least one processor, the computing memory comprising executable instructions that upon execution cause the system to perform operations comprising:
receiving a first depth image, the first depth image being captured by a depth camera;
identifying an estimated location or position of a first extremity of the user in the first depth image;
positioning a first part of a model to correspond to the estimated location or position of the first extremity in the first depth image; and
in response to determining that a location or position of a second extremity has not been estimated, adjusting a second part of the model to correspond to a portion of the depth image that is located close to the second part of the model.

### Claim 31 (depends on 30)

The system of claim 30, wherein adjusting a second part of the model to correspond to a portion of the depth image that is located close to the second part of the model comprises adjusting the second part of the model to a location of at least one voxel in the first depth image.

### Claim 32 (depends on 31)

The system of claim 31, wherein adjusting the second part of the model to a location of at least one voxel in the first depth image comprises adjusting the second part of the model to a location defined by an x-value, y-value, and depth value in the first depth image.

### Claim 33 (depends on 31)

The system of claim 31, wherein adjusting a second part of the model to correspond to a portion of the depth image that is located close to the second part of the model comprises adjusting the second part of the model to correspond to a portion of the depth image that represents the user.

### Claim 34 (depends on 30)

The system of claim 30, further comprising executable instructions that upon execution cause the system to perform operations comprising:
in response to determining that a location or position of a second extremity has not been estimated, positioning the model over the first depth image,
wherein adjusting a second part of the model to correspond to a portion of the depth image that is located close to the second part of the model comprises adjusting the second part of the model to a portion of the depth image that is located closest to the second part of the model.

### Claim 35 (depends on 30)

The system of claim 30, wherein adjusting a second part of the model to correspond to a portion of the depth image that is located close to the second part of the model comprises magnetizing the second part of the model to a portion of the depth image.

### Claim 36 (depends on 35)

The system of claim 35, wherein magnetizing the second part of the model to a portion of the depth image comprises magnetizing the second part of the model to voxels defining a boundary or surface in the first depth image.

### Claim 37 (depends on 30)

The system of claim 30, wherein adjusting a second part of the model to correspond to a portion of the depth image that is located close to the second part of the model comprises adjusting the second part of the model to correspond to voxels defining a boundary or surface in the first depth image.

### Claim 38 (depends on 30)

The system of claim 30, wherein a second part of the model to correspond to a portion of the depth image that is located close to the second part of the model comprises adjusting a joint of a skeletal model.

### Claim 39 (depends on 30)

The system of claim 30, further comprising executable instructions that upon execution cause the system to perform operations comprising:
downsampling pixels in the depth image before identifying an estimated location or position of a first extremity of the user in the first depth image.

### Claim 40

A method for tracking a user, comprising:
receiving a first depth image, the first depth image being captured by a depth camera;
in response to identifying an estimated location of an extremity of a user in the first depth image, positioning a part of a model to correspond to the estimated location of the extremity in the first depth image; and
in response to determining a failure to estimate a location of an extremity, adjusting the part of the model to correspond to a portion of the depth image that is located close to the part of the model.
