# US 2021/0279506 A1 · Systems, methods, and devices for head pose determination

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279506 A1
- Document ID: US20210279506A1
- Status: Published application
- Kind: A1
- Application number: 17/191,718
- Filed: 2021-03-04
- Published: 2021-09-09
- Earliest claimed: 2020-12-18
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Parual DATTA; Nilesh AHUJA; Javier FELIP LEON
- Web page: https://patentgrep.com/patent/US20210279506A1

## CPC

- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06F 18/22: Matching criteria, e.g. proximity measures
- G06V 2201/12: Acquisition of 3D measurements of objects
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Estimating a head pose may include obtaining sensor data corresponding to a head and at least a portion of the body of a human subject and determining an estimate of a three-dimensional (3D) body pose using the obtained sensor data. The estimation can further include generating a first rendering of at least the human subject's head using the obtained sensor data and generating a plurality of head pose sample data sets by applying the estimated 3D body pose to a head-pose generative model. Further, the head pose estimation can include generating a plurality of second renderings respectively from each of the plurality of head pose sample data sets; determining which of the plurality of second renderings is closest to the first rendering; and selecting the second rendering determined to be closest to the first rendering.

## Claims

### Claim 1

A non-transitory computer readable medium comprising instructions that when executed by the at least one processor cause the at least one processor to:
obtain sensor data corresponding to a head and at least a portion of the body of a human subject;
determine an estimate of a body pose using the obtained sensor data;
generate a first rendering of at least the human subject's head using the obtained sensor data;
generate at least one head pose data set with a generative model based on the estimated body pose,
generate at least one second rendering using the at least one head pose dataset; and
determine at least one likeness factor between the first and the at least one second rendering.

### Claim 2 (depends on 1)

The computer readable medium of claim 1, wherein to determine the estimate of the body pose using the obtained sensor data comprises to use one or more of a machine learning technique, a neural network, a trajectory the human subject is on, historic pose data of the human subject, or a database or knowledge of human subject poses, an environment context the human subject and/or other actors is in.

### Claim 3 (depends on 1)

The computer readable medium of claim 1, wherein the body pose is a 3D body pose.

### Claim 4 (depends on 1)

The computer readable medium of claim 1, wherein the at least one head pose data set comprises a plurality of head pose data sets, and wherein to generate the at least one respective second rendering comprises to generate a plurality of second renderings respectively from using the plurality of the head pose data sets, and wherein to determine the at least one likeness factor comprises to determine a likeness factor for each of the plurality of second renderings.

### Claim 5 (depends on 4)

The computer readable medium of claim 4, wherein the instructions when executed, further cause the at least one processor to further determine one of the plurality of second renderings to be closest to the first rendering using the respective likeness factor.

### Claim 6 (depends on 1)

The computer readable medium of claim 1, wherein to determine which one of the plurality of second renderings is closest to the first rendering comprises to determine which one of the plurality of second renderings is closest to the first rendering using a likelihood function.

### Claim 7 (depends on 1)

The computer readable medium of claim 1, wherein the instructions when executed, further cause the at least one processor to:
select the head pose data set corresponding to the second rendering determined closest to the first rendering.

### Claim 8 (depends on 1)

The computer readable medium of claim 1, wherein to generate the head pose data set comprises to generate a random sampling of head pose data.

### Claim 9 (depends on 1)

The computer readable medium of claim 1, wherein to generate the first rendering comprises to generate the first rendering from a viewpoint corresponding to a sensor origin viewpoint of the obtained sensor data.

### Claim 10 (depends on 1)

The computer readable medium of claim 1, wherein the at least one processor renders the first and second renderings in a same render space.

### Claim 11 (depends on 1)

The computer readable medium of claim 1, wherein one or more facial features of human subject's head in the obtained sensor data are concealed or occluded.

### Claim 12 (depends on 1)

The computer readable medium of claim 1, wherein the generative model comprises a conditional probability distribution of head pose data sets.

### Claim 13 (depends on 12)

The computer readable medium of claim 12, wherein the conditional probability distribution of head pose data sets comprises a conditional probability distribution of one or more head pose parameters.

### Claim 14 (depends on 12)

The computer readable medium of claim 12, wherein the conditional probability distribution of head pose data sets comprises a conditional probability distribution of observed head-pose data sets.

### Claim 15 (depends on 1)

The computer readable medium of claim 1, wherein the obtained sensor data comprises video data.

### Claim 16 (depends on 15)

The computer readable medium of claim 15,
wherein determining an estimate of the body pose comprises determining a body pose for each of a plurality of sequential frames from the video data, and
wherein generating the at least one head pose data sets comprising successively applying each of the estimated 3D body poses and applying a corresponding previously generated head pose data to the generative model.

### Claim 17

A head pose apparatus comprising:
an interface, coupled to at least one sensor, to receive sensor data from the at least one sensor, the sensor data corresponding to a head and at least a portion of the body of a human subject;
a head pose estimator comprising:
at least one processor; and
a non-transitory computer readable medium comprising instructions that when executed by the at least one processor cause the at least one processor to:
determine an estimate of a body pose using the obtained sensor data;
generate a first rendering of at least the human subject's head using the obtained sensor data;
generate at least one head pose data set with a generative model based on the estimated body pose,
generate at least one second rendering using the at least one head pose dataset; and
determine at least one likeness factor between the first and the at least one second rendering.

### Claim 18 (depends on 18)

The head pose apparatus of claim 18, wherein the at least one head pose data set comprises a plurality of head pose data sets, and wherein to generate the at least one respective second rendering comprises to generate a plurality of second renderings respectively from using the plurality of the head pose data sets, and wherein to determine the at least one likeness factor comprises to determine a likeness factor for each of the plurality of second renderings.

### Claim 19 (depends on 19)

The head pose apparatus of claim 19, wherein the instructions when executed, further cause the at least one processor to further determine one of the plurality of second renderings to be closest to the first rendering using the respective likeness factor.

### Claim 20

A head pose estimator comprising:
means for obtaining sensor data corresponding to a head and at least a portion of the body of a human subject;
means for determining an estimate of a body pose using the obtained sensor data;
means for generating a first rendering of at least the human subject's head using the obtained sensor data;
means for generating head pose data;
means for generating at least one second rendering from the head pose data, and
means for selecting one of the plurality of the second renderings that is closest to the first rendering.
