# US 12,762,735 B2 · Enhanced techniques for real-time multi-person three-dimensional pose tracking using a single camera

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,762,735 B2
- Document ID: US12762735B2
- Status: Granted
- Kind: B2
- Application number: 18/569,996
- Filed: 2021-12-10
- Published: 2024-09-19
- Granted: 2026-10-13
- Earliest claimed: 2021-12-10
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Shandong Wang; Yurong Chen; Ming Lu; Li Xu; Anbang Yao
- Web page: https://patentgrep.com/patent/US12762735B2

## CPC

- G06T 7/74: {involving reference images or patches}
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30196: Human being; Person
- G06T 2207/30221: Sports video; Sports image
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

This disclosure describes systems, methods, and devices related to real-time multi-person three-dimensional pose tracking using a single camera. A method may include receiving, by a device, two-dimensional image data from a camera, the two-dimensional image data representing a first person and a second person; generating, based on the two-dimensional image data, two-dimensional positions of body parts represented by the first person; generating, using a deep neural network, based on the two-dimensional positions, a three-dimensional pose regression of the body parts represented by the first person; identifying, based on the two-dimensional positions and the three-dimensional pose regression, contact between a ground plane and a foot of the first person; generating an absolute three-dimensional position of the contact between the ground plane and the foot of the first person; generating, based on the absolute three-dimensional position, a three-dimensional pose of the body parts represented by the first person.

## Claims

### Claim 1

A method comprising:
generating, by at least one processor circuit based on at least one instruction, first two-dimensional positions of body parts based on two-dimensional image data representing a first person and a second person, the first two-dimensional positions associated with the first person;
generating, by one or more of the at least one processor circuit, second two-dimensional positions of body parts based on the two-dimensional image data, the second two-dimensional positions associated with the second person;
generating, based on a deep neural network and the first two-dimensional positions, a first root-relative three-dimensional pose regression of the body parts associated with the first person;
generating, based on the deep neural network and the second two-dimensional positions, a second root-relative three-dimensional pose regression of the body parts associated with the second person;
identifying, by one or more of the at least one processor circuit, and based on the first two-dimensional positions and the first root-relative three-dimensional pose regression, contact between a ground plane and a foot of the first person;
identifying, by one or more of the at least one processor circuit, and based on the second two-dimensional positions and the second root-relative three-dimensional pose regression, contact between the ground plane and a foot of the second person;
generating, by one or more of the at least one processor circuit, a first absolute three-dimensional position of the contact between the ground plane and the foot of the first person;
generating, by one or more of the at least one processor circuit, a second absolute three-dimensional position of the contact between the ground plane and the foot of the second person;
identifying, by one or more of the at least one processor circuit, a two-dimensional position of the contact between the ground plane and the foot of the first person;
identifying, by one or more of the at least one processor circuit, two-dimensional positions of the ground plane based on the two-dimensional image data;
generating a homographic matrix to map the two-dimensional positions of the ground plane to three-dimensional positions of the ground plane;
generating a three-dimensional root position of the foot of the first person based on the homographic matrix;
generating, by one or more of the at least one processor circuit, a first three-dimensional pose of the body parts associated with the first person based on a difference between the first absolute three-dimensional position and the three-dimensional root position of the foot of the first person; and
generating, by one or more of the at least one processor circuit, and based on the second absolute three-dimensional position, a second three-dimensional pose of the body parts associated with the second person.

### Claim 2 (depends on 1)

The method of claim 1, wherein:
the generating of the first absolute three-dimensional position is based on the homographic matrix, and
the generating of the second absolute three-dimensional position is based on the homographic matrix.

### Claim 3 (depends on 2)

The method of claim 2, wherein the two-dimensional image data is from a camera, and including:
generating extrinsic parameters of the camera based on the homographic matrix and a focal length of the camera, the extrinsic parameters including a rotation matrix and a translation vector associated with the camera, wherein:
the generating of the first three-dimensional pose is based on the extrinsic parameters, and
the generating of the second three-dimensional pose is based on the extrinsic parameters.

### Claim 4 (depends on 1)

The method of claim 1, further comprising wherein the two-dimensional position of the contact between the ground plane and the foot of the first person is a first two-dimensional position, and including:
identifying a second two-dimensional position of the contact between the ground plane and the foot of the second person;
determining, based on the first two-dimensional position, that a first velocity of the foot of the first person is below a threshold velocity;
determining, based on the second two-dimensional position, that a second velocity of the foot of the second person is below the threshold velocity;
verifying the contact between the ground plane and the foot of the first person is further based on the first velocity of the foot of the first person being below the threshold velocity; and
verifying the contact between the ground plane and the foot of the second person is further based on the second velocity of the foot of the second person being below the threshold velocity.

### Claim 5 (depends on 1)

The method of claim 1, including:
determining a second difference between the first three-dimensional pose and the first root-relative three-dimensional pose regression; and
generating, based on the second difference, a third three-dimensional pose of the body parts associated with the first person.

### Claim 6 (depends on 1)

The method of claim 1, including:
determining an image difference between a first image including the two-dimensional image data and a second image including second two-dimensional image data representing the first person and the second person; and
generating, based on the image difference, a third three-dimensional pose of the body parts associated with the first person.

### Claim 7 (depends on 1)

The method of claim 1, wherein the two-dimensional image data is from a camera, and including:
generating extrinsic parameters of the camera based on the homographic matrix and a focal length of the camera, the extrinsic parameters including a rotation matrix and a translation vector for the camera; and
determining, based on the extrinsic parameters, differences between the first absolute three-dimensional position and the first two-dimensional positions,
wherein the generating of the first absolute three-dimensional position is based on the differences.

### Claim 8

A system comprising:
memory;
instructions; and
at least one processor circuit to be programmed based on the instructions to:
generate, based on two-dimensional image data representing a first person and a second person, first two-dimensional positions of body parts associated with the first person, the two-dimensional image data from a camera;
generate, based on the two-dimensional image data, second two-dimensional positions of body parts associated with the second person;
cause a deep neural network to generate, based on the first two-dimensional positions, a first root-relative three-dimensional pose regression of the body parts associated with the first person;
cause the deep neural network to generate, based on the second two-dimensional positions, a second root-relative three- dimensional pose regression of the body parts associated with the second person;
identify, based on the first two-dimensional positions and the first root-relative three-dimensional pose regression, contact between a ground plane and a foot of the first person;
identify, based on the second two-dimensional positions and the second root-relative three-dimensional pose regression, contact between the ground plane and a foot of the second person;
generate a first absolute three-dimensional position of the contact between the ground plane and the foot of the first person;
generate a second absolute three-dimensional position of the contact between the ground plane and the foot of the second person;
identify two-dimensional positions of the ground plane based on the two-dimensional image data;
generate a homographic matrix to map the two-dimensional positions of the ground plane to three-dimensional positions of the ground plane;
generate extrinsic parameters of the camera based on the homographic matrix and a focal length of the camera, the extrinsic parameters including a rotation matrix and a translation vector associated with the camera;
determine, based on the extrinsic parameters, differences between the first absolute three-dimensional position and at least some of the first two-dimensional positions;
generate, based on the first absolute three-dimensional position and the differences, a first three-dimensional pose of the body parts represented by the first person; and
generate, based on the second absolute three-dimensional position, a second three-dimensional pose of the body parts represented by the second person.

### Claim 9 (depends on 8)

The system of claim 8, wherein one or more of the at least one processor circuit is to:
generate the first absolute three-dimensional position based on the homographic matrix; and
generate the second absolute three-dimensional position based on the homographic matrix.

### Claim 10 (depends on 8)

The system of claim 8, wherein one or more of the at least one processor circuit is to:
identify a first two-dimensional position of the contact between the ground plane and the foot of the first person;
identify a second two-dimensional position of the contact between the ground plane and the foot of the second person;
determine, based on the first two-dimensional position, that a first velocity of the foot of the first person is below a threshold velocity;
determine, based on the second two-dimensional position, that a second velocity of the foot of the second person is below the threshold velocity;
verify the contact between the ground plane and the foot of the first person is further based on the first velocity of the foot of the first person being below the threshold velocity; and
verify the contact between the ground plane and the foot of the second person based on the second velocity of the foot of the second person being below the threshold velocity.

### Claim 11 (depends on 8)

The system of claim 8, wherein one or more of the at least one processor circuit is to:
identify a two-dimensional position of the contact between the ground plane and the foot of the first person; and
identify two-dimensional positions of the ground plane based on the two-dimensional image data; and
generate the first three-dimensional pose based on a mapping of the two-dimensional position of the contact between the ground plane and the foot of the first person to the first absolute three-dimensional position, the mapping based on the homographic matrix.

### Claim 12 (depends on 8)

The system of claim 8, wherein one or more of the at least one processor circuit is to:
determine a difference between the first three-dimensional pose and the first root-relative three-dimensional pose regression; and
generate, based on the difference, a third three-dimensional pose of the body parts associated with the first person.

### Claim 13 (depends on 8)

The system of claim 8, wherein one or more of the at least one processor circuit to:
determine an image difference between a first image including the two-dimensional image data and a second image including second two-dimensional image data representing the first person and the second person; and
generate, based on the image difference, a third three-dimensional pose of the body parts associated with the first person.

### Claim 14 (depends on 8)

The system of claim 8, wherein one or more of the at least one processor circuit to:
identify a two-dimensional position of the contact between the ground plane and the foot of the first person;
generate a three-dimensional root position of the foot of the first person based on the homographic matrix; and
generate the first three-dimensional pose based on a difference between the first absolute three-dimensional position and the three-dimensional root position of the foot of the first person.

### Claim 15

An apparatus comprising:
means for generating two dimensional body part positions, the means for generating two dimensional body part positions to:
generate, based on two-dimensional image data representing a first person and a second person, first two-dimensional positions of body parts associated with the first person; and
generate, based on the two-dimensional image data, second two-dimensional positions of body parts associated with the second person;
means for generating root-relative three-dimensional pose regression, the means for generating root-relative three-dimensional pose regression to:
generate, based on a deep neural network and the first two-dimensional positions, a first root-relative three-dimensional pose regression of the body parts associated with the first person; and
generate, based on the deep neural network and the second two-dimensional positions, a second root-relative three-dimensional pose regression of the body parts associated with the second person;
means for identifying ground plane contact, the means for identifying ground plane contact to:
identify, based on the first two-dimensional positions and the first root-relative three-dimensional pose regression, contact between a ground plane and a foot of the first person; and
identify, based on the second two-dimensional positions and the second root-relative three-dimensional pose regression, contact between the ground plane and a foot of the second person;
means for generating absolute three-dimensional ground contact positions, the means for generating absolute three-dimensional ground contact positions to:
generate a first absolute three-dimensional position of the contact between the ground plane and the foot of the first person; means for generating and
generate a second absolute three-dimensional position of the contact between the ground plane and the foot of the second person; and
means for generating poses, the means for generating poses to:
generate, based on the first absolute three-dimensional position, a first three-dimensional pose of the body parts associated with the first person; and
generate, based on the second absolute three-dimensional position, a second three-dimensional pose of the body parts represented by the second person.

### Claim 16 (depends on 15)

The apparatus of claim 15, including:
means for identifying two-dimensional positions of the ground plane based on the two-dimensional image data; and
means for generating a homographic matrix to map the two-dimensional positions of the ground plane to three-dimensional positions of the ground plane,
wherein the means for generating poses is to:
generate the first three-dimensional pose based on the homographic matrix, and
generate the second three-dimensional pose based on the homographic matrix.