# US 2013/0293679 A1 · Upper-Body Skeleton Extraction from Depth Maps

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2013/0293679 A1
- Document ID: US20130293679A1
- Status: Published application
- Kind: A1
- Application number: 13/461,802
- Filed: 2012-05-02
- Published: 2013-11-07
- Earliest claimed: 2012-05-02
- Assignees: Primesense Ltd.
- Inventors: Amiad Gurman; Oren Tropp
- Web page: https://patentgrep.com/patent/US20130293679A1

## CPC

- G06T 7/73: using feature-based methods
- G06V 10/457: {by analysing connectivity, e.g. edge linking, connected component analysis or slices}
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30196: Human being; Person
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method for processing data includes receiving a depth map of a scene containing at least an upper body of a humanoid form. The depth map is processed so as to identify a head and at least one arm of the humanoid form in the depth map. Based on the identified head and at least one arm, and without reference to a lower body of the humanoid form, an upper-body pose, including at least three-dimensional (3D) coordinates of shoulder joints of the humanoid form, is extracted from the depth map.

## Claims

### Claim 1

A method for processing data, comprising:
receiving a depth map of a scene containing at least an upper body of a humanoid form, the depth map comprising a matrix of pixels having respective pixel depth values;
using a digital processor, processing the depth map so as to identify a head and at least one arm of the humanoid form in the depth map; and
based on the identified head and at least one arm, and without reference to a lower body of the humanoid form, extracting from the depth map an upper-body pose comprising at least three-dimensional (3D) coordinates of shoulder joints of the humanoid form.

### Claim 2 (depends on 1)

The method according to claim 1, wherein extracting the upper-body pose comprises finding the upper-body pose while the humanoid form in the depth map is seated.

### Claim 3 (depends on 1)

The method according to claim 1, wherein processing the depth map comprises locating one or more ridges in the depth map corresponding to parts of the humanoid form, each ridge corresponding to a chain of the pixels whose respective depth values are local minima.

### Claim 4 (depends on 1)

The method according to claim 1, wherein processing the depth map comprises locating depth edges in the depth map corresponding to parts of the humanoid form, and associating arcuate depth edges found in the depth map with the head and with a hand of the form.

### Claim 5 (depends on 1)

The method according to claim 1, wherein extracting the upper-body pose comprises finding 3D medial axes of the head and the at least one arm in the depth map, and finding the upper-body pose that includes estimated positions of the head and the at least one arm that satisfy constraints of a predefined body model while minimizing an error between the 3D medial axes and the estimated positions.

### Claim 6 (depends on 1)

The method according to claim 1, and comprising controlling a computerized system responsively to the extracted upper-body pose.

### Claim 7

A method for processing data, comprising:
receiving a depth map of a scene containing an object of interest, the depth map comprising a matrix of pixels having respective pixel depth values;
using a digital processor, processing the depth map so as to locate a single ridge on each limb of the object, the ridge corresponding to a chain of the pixels whose respective depth values are local minima; and
responsively to the ridge, extracting features of the object from the depth map.

### Claim 8 (depends on 7)

The method according to claim 7, wherein processing the depth map comprises finding each local minimum on a respective line oriented obliquely relative to the chain, while the respective depth values of the other pixels on the respective line are greater than the local minimum.

### Claim 9 (depends on 7)

The method according to claim 7, wherein extracting the features comprises processing the depth map so as to locate one or more depth edges of the object, and applying the depth edges together with the ridge in order to reconstruct a three-dimensional shape of the object.

### Claim 10 (depends on 7)

The method according to claim 7, wherein extracting the features comprises partitioning the chain into segments, fitting a line to each segment, and finding a location and orientation of the object based on the line.

### Claim 11 (depends on 7)

The method according to claim 7, wherein the object is a head of a humanoid form, and wherein extracting the features comprises finding an increase in the respective depth values of the pixels at an upper end of the ridge, and identifying the object as the head responsively to the increase.

### Claim 12 (depends on 11)

The method according to claim 11, wherein identifying the object as the head comprises processing the depth map so as to locate an arcuate depth edge around the upper end of the ridge.

### Claim 13

Apparatus for processing data, comprising:
an imaging assembly, which is configured to capture a depth map of a scene containing at least an upper body of a humanoid form, the depth map comprising a matrix of pixels having respective pixel depth values; and
a processor, which is configured to process the depth map so as to identify a head and at least one arm of the humanoid form in the depth map, and to extract from the depth map, based on the identified head and at least one arm, and without reference to a lower body of the humanoid form, an upper-body pose comprising at least three-dimensional (3D) coordinates of shoulder joints of the humanoid form.

### Claim 14 (depends on 13)

The apparatus according to claim 13, wherein the processor is configured to find the upper-body pose while the humanoid form in the depth map is seated.

### Claim 15 (depends on 13)

The apparatus according to claim 13, wherein the processor is configured to locate one or more ridges in the depth map corresponding to parts of the humanoid form, each ridge corresponding to a chain of the pixels whose respective depth values are local minima.

### Claim 16 (depends on 13)

The apparatus according to claim 13, wherein the processor is configured to locate depth edges in the depth map corresponding to parts of the humanoid form, and to associate arcuate depth edges found in the depth map with the head and with a hand of the form.

### Claim 17 (depends on 13)

The apparatus according to claim 13, wherein the processor is configured to find 3D medial axes of the head and the at least one arm in the depth map, and to find the upper-body pose that includes estimated positions of the head and the at least one arm that satisfy constraints of a predefined body model while minimizing an error between the 3D medial axes and the estimated positions.

### Claim 18 (depends on 13)

The apparatus according to claim 13, wherein the processor is configured to control a computerized system responsively to the extracted upper-body pose.

### Claim 19

Apparatus for processing data, comprising:
an imaging assembly, which is configured to capture a depth map of a scene containing an object of interest, the depth map comprising a matrix of pixels having respective pixel depth values; and
a processor, which is configured to process the depth map so as to locate a single ridge on each limb of the object, the ridge corresponding to a chain of the pixels whose respective depth values are local minima, and to extract, responsively to the ridge, features of the object from the depth map.

### Claim 20 (depends on 19)

The apparatus according to claim 19, wherein the processor is configured to find each local minimum on a respective line oriented obliquely relative to the chain, while the respective depth values of the other pixels on the respective line are greater than the local minimum.

### Claim 21 (depends on 19)

The apparatus according to claim 19, wherein the processor is configured to locate one or more depth edges of the object in the depth map, and to apply the depth edges together with the ridge in order to reconstruct a three-dimensional shape of the object.

### Claim 22 (depends on 19)

The apparatus according to claim 19, wherein the processor is configured to partition the chain into segments, to fit a line to each segment, and to find a location and orientation of the object based on the line.

### Claim 23 (depends on 19)

The apparatus according to claim 19, wherein the object is a head of a humanoid form, and wherein the processor is configured to find an increase in the respective depth values of the pixels at an upper end of the ridge, and to identify the object as the head responsively to the increase.

### Claim 24 (depends on 23)

The apparatus according to claim 23, wherein the processor is configured to process the depth map so as to locate an arcuate depth edge around the upper end of the ridge, and to identify the object as the head responsively to the arcuate depth edge.

### Claim 25

A computer software product, comprising a computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to receive a depth map of a scene containing at least an upper body of a humanoid form, the depth map comprising a matrix of pixels having respective pixel depth values, to process the depth map so as to identify a head and at least one arm of the humanoid form in the depth map, and to extract from the depth map, based on the identified head and at least one arm, and without reference to a lower body of the humanoid form, an upper-body pose comprising at least three-dimensional (3D) coordinates of shoulder joints of the humanoid form.

### Claim 26 (depends on 25)

The product according to claim 25, wherein the instructions cause the computer to find the upper-body pose while the humanoid form in the depth map is seated.

### Claim 27 (depends on 25)

The product according to claim 25, wherein the instructions cause the computer to locate one or more ridges in the depth map corresponding to parts of the humanoid form, each ridge corresponding to a chain of the pixels whose respective depth values are local minima.

### Claim 28 (depends on 25)

The product according to claim 25, wherein the instructions cause the computer to locate depth edges in the depth map corresponding to parts of the humanoid form, and to associate arcuate depth edges found in the depth map with the head and with a hand of the form.

### Claim 29 (depends on 25)

The product according to claim 25, wherein the instructions cause the computer to find 3D medial axes of the head and the at least one arm in the depth map, and to find the upper-body pose that includes estimated positions of the head and the at least one arm that satisfy constraints of a predefined body model while minimizing an error between the 3D medial axes and the estimated positions.

### Claim 30 (depends on 25)

The product according to claim 25, wherein the instructions cause the computer to control a computerized system responsively to the extracted upper-body pose.

### Claim 31

A computer software product, comprising a computer-readable medium in which program instructions are stored, which instructions, when read by a computer, cause the computer to receive a depth map of a scene containing an object of interest, the depth map comprising a matrix of pixels having respective pixel depth values, to process the depth map so as to locate a single ridge on each limb of the object, the ridge corresponding to a chain of the pixels whose respective depth values are local minima, and to extract, responsively to the ridge, features of the object from the depth map.

### Claim 32 (depends on 31)

The product according to claim 31, wherein the instructions cause the computer to find each local minimum on a respective line oriented obliquely relative to the chain, while the respective depth values of the other pixels on the respective line are greater than the local minimum.

### Claim 33 (depends on 31)

The product according to claim 31, wherein the instructions cause the computer to locate one or more depth edges of the object in the depth map, and to apply the depth edges together with the ridge in order to reconstruct a three-dimensional shape of the object.

### Claim 34 (depends on 31)

The product according to claim 31, wherein the instructions cause the computer to partition the chain into segments, to fit a line to each segment, and to find a location and orientation of the object based on the line.

### Claim 35 (depends on 31)

The product according to claim 31, wherein the object is a head of a humanoid form, and wherein the instructions cause the computer to find an increase in the respective depth values of the pixels at an upper end of the ridge, and to identify the object as the head responsively to the increase.

### Claim 36 (depends on 35)

The product according to claim 35, wherein the instructions cause the computer to process the depth map so as to locate an arcuate depth edge around the upper end of the ridge, and to identify the object as the head responsively to the arcuate depth edge.
