# US 2023/0067442 A1 · Method of human pose estimation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0067442 A1
- Document ID: US20230067442A1
- Status: Published application
- Kind: A1
- Application number: 17/463,166
- Filed: 2021-08-31
- Published: 2023-03-02
- Earliest claimed: 2021-08-31
- Assignees: Black Sesame International Holding Limited
- Applicants: Black Sesame International Holding Limited
- Inventors: Xiaomin Liu; Lei Zhang; Qun Gu
- Web page: https://patentgrep.com/patent/US20230067442A1

## CPC

- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06T 7/73: using feature-based methods
- G06V 10/143: Sensing or illuminating at different wavelengths
- G06V 10/50: by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
- G06V 10/82: using neural networks
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30196: Human being; Person
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A method of human pose estimation, including, receiving an image frame, extracting in at least a first stage a first stage image feature based on the image frame, extracting in at least a second stage a second stage image feature based on the first stage image feature, extracting in at least a subsequent stage a subsequent stage image feature based on the second stage image feature, up-sampling the subsequent stage image feature, up-sampling the second stage image feature, concatenating the first stage image feature, the up-sampled second stage image feature and the up-sampled subsequent stage image feature and outputting a feature map based on an output of the concatenation.

## Claims

### Claim 1

A method of human pose estimation, comprising:
receiving an image frame;
extracting in at least a first stage a first stage image feature based on the image frame;
extracting in at least a second stage a second stage image feature based on the first stage image feature;
extracting in at least a subsequent stage a subsequent stage image feature based on the second stage image feature;
up-sampling the subsequent stage image feature;
up-sampling the second stage image feature;
concatenating the first stage image feature, the up-sampled second stage image feature and the up-sampled subsequent stage image feature; and
outputting a feature map based on an output of the concatenation.

### Claim 1 (depends on 1)

The method of human pose estimation of claim 1, wherein the feature map is based on a bottom-up determination.

### Claim 2 (depends on 1)

The method of human pose estimation of claim 1, wherein the image frame is at least one of red-blue-green and infrared.

### Claim 3 (depends on 1)

The method of human pose estimation of claim 1, wherein the extracting in at least the first stage utilizes a resnet-v1-50.

### Claim 4 (depends on 1)

The method of human pose estimation of claim 1, wherein the up-sampling of the subsequent stage image feature is by four.

### Claim 5 (depends on 1)

The method of human pose estimation of claim 1, wherein the up-sampling of the second stage image feature is by two.

### Claim 6 (depends on 1)

The method of human pose estimation of claim 1, further comprising a third stage to extract a third stage image feature.

### Claim 7

A method of human pose estimation, comprising:
receiving a feature map by an initial stage;
convoluting the feature map based on an initial part affinity map branch of the initial stage;
determining an initial part affinity map based on an output of the initial part affinity map branch;
convoluting the feature map based on an initial heat map branch of the initial stage;
determining an initial heat map based on an output of the initial heat map branch;
concatenating the initial part affinity map branch, the initial heat map branch and the feature map to output an initial stage concatenation;
receiving the initial stage concatenation by a refinement stage;
convoluting the initial stage concatenation via refinement part affinity map branch of the refinement stage;
determining a refinement part affinity map based on an output of the refinement part affinity map branch;
convoluting the initial stage concatenation via refinement heat map branch of the refinement stage; and
determining a refinement heat map based on an output of the refinement heat map branch.

### Claim 8 (depends on 7)

The method of human pose estimation of claim 7, wherein the feature map is based on a bottom-up determination.

### Claim 9 (depends on 7)

The method of human pose estimation of claim 7, wherein the heat map predicts a location of a body part and the part affinity map predicts an association of the body part.

### Claim 10 (depends on 7)

The method of human pose estimation of claim 7, wherein the convolution if the feature map utilizes a resnet-v1-50.

### Claim 11 (depends on 7)

The method of human pose estimation of claim 7, further comprising determining a body part confidence map based on the refinement part affinity map.

### Claim 12 (depends on 7)

The method of human pose estimation of claim 7, wherein the concatenating utilizes a residual connection of the feature map.

### Claim 13 (depends on 7)

The method of human pose estimation of claim 7, wherein the convolution of the feature map includes at least one 3×3 convolution and at least one 1×1 convolution.

### Claim 14

A method of human pose estimation, comprising:
receiving a feature map by an initial stage;
convoluting the feature map based on a first part affinity map branch of the initial stage;
determining a first stage part affinity map based on an output of the first part affinity map branch;
concatenating the first stage part affinity map and the feature map to output a first stage part affinity map concatenation;
convoluting the first stage part affinity map concatenation to output a second stage part affinity map convolution;
determining second stage part affinity map based on an output of the second stage part affinity map convolution;
concatenating the second stage part affinity map and the feature map to output a second stage part affinity map concatenation;
receiving the second stage part affinity map concatenation by a refinement stage;
convoluting the second stage part affinity map concatenation to output a third stage part affinity map convolution;
determining a third stage part affinity map based on an output of the third stage part affinity map convolution;
concatenating the third stage part affinity map and the feature map to output a concatenated third stage part affinity map;
convoluting the concatenated third stage part affinity map to output a refinement heatmap prediction; and
determining a refinement heat map based on an output of the refinement heatmap prediction.

### Claim 15 (depends on 14)

The method of human pose estimation of claim 14, wherein the feature map is based on a bottom-up determination.

### Claim 16 (depends on 14)

The method of human pose estimation of claim 14, wherein the heat map predicts a location of a body part and the part affinity map predicts an association of the body part.

### Claim 17 (depends on 14)

The method of human pose estimation of claim 14, wherein the convolution of the feature map utilizes a resnet-v1-50.

### Claim 18 (depends on 14)

The method of human pose estimation of claim 14, further comprising determining a body part confidence map based on the refinement part affinity map.

### Claim 19 (depends on 14)

The method of human pose estimation of claim 14, wherein the concatenating of the first stage, the second stage and the third stage utilizes a residual connection of the feature map.

### Claim 20 (depends on 14)

The method of human pose estimation of claim 14, wherein the convolution of the feature map includes at least one 3×3 convolution and at least one 1×1 convolution.
