# US 2026/0065565 A1 · Methods, systems, and computer program products for generating 3d human pose and movement estimation from monocular image information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065565 A1
- Document ID: US20260065565A1
- Status: Published application
- Kind: A1
- Application number: 19/318,526
- Filed: 2025-09-04
- Published: 2026-03-05
- Earliest claimed: 2024-09-04
- Assignees: The University of North Carolina at Charlotte
- Applicants: The University of North Carolina at Charlotte
- Inventors: Pu Wang; Muhammad Usama Saleem; Ekkasit Pinyoanuntapong; Parshwa Shah; Foram Shah
- Web page: https://patentgrep.com/patent/US20260065565A1

## CPC

- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06T 7/73: using feature-based methods
- G06T 13/205: {driven by audio data}
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06V 10/766: using regression, e.g. by projecting features on hyperplanes
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A computer-implemented method includes converting by a pose tokenizer, based on a learned codebook, pose parameters of a body into a sequence of discrete pose tokens; randomly masking a portion of the sequence of discrete pose tokens; predicting the randomly masked sequence of discrete pose tokens based on multi-scale features extracted from a monocular image by an image conditioned masked transformer; optimizing the sequence of discrete pose tokens by aligning a re-projected three-dimensional (3D) pose with an estimated two-dimensional (2D) pose; directly regressing, from the multi-scale features, a shape parameter of the body and a weak perspective camera parameter; and generating a 3D mesh reconstruction of the body based on the shape parameter and the weak perspective camera parameter.

## Claims

### Claim 1

A computer-implemented method comprises performing, by one or more processors, operations comprising:
converting by a pose tokenizer, based on a learned codebook, pose parameters of a body into a sequence of discrete pose tokens;
randomly masking a portion of the sequence of discrete pose tokens;
predicting the randomly masked sequence of discrete pose tokens based on multi-scale features extracted from a monocular image by an image conditioned masked transformer;
optimizing the sequence of discrete pose tokens by aligning a re-projected three-dimensional (3D) pose with an estimated two-dimensional (2D) pose;
directly regressing, from the multi-scale features, a shape parameter of the body and a weak perspective camera parameter; and
generating a 3D mesh reconstruction of the body based on the shape parameter and the weak perspective camera parameter.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the one or more processors comprise the pose tokenizer and the image conditional masked transformer; and
wherein the method further comprises:
training the pose tokenizer using Vector Quantized Variational Autoencoders (VQ-VAE).

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the post parameters comprise a representation of a continuous human pose and a representation of rotations of skeletal joints.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein the image conditional masked transformer comprises an image encoder and a masked transformer decoder with multi-scale deformable cross attention.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein optimizing the sequence comprises optimizing the sequence by using a two-dimensional pose-guided sampling strategy.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, further comprising:
training the image conditional masked transformer to predict the randomly masked sequence of discrete pose tokens by learning a conditional categorical distribution of sequences of discrete pose tokens.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein predicting the randomly masked sequence comprises:
predicting the randomly masked sequence using an iterative decoding process.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein the iterative decoding process comprises:
predicting high-confidence sequences of the discrete pose tokens;
progressively refining the high-confidence sequences by masking low-confidence sequences of the discrete pose tokens; and
leveraging both image semantics of the 2D image and inter-token dependencies.

### Claim at position 9

9-16. (canceled)

### Claim 17

A computer-implemented method comprises performing, by one or more processors, operations comprising:
receiving a text prompt and a spatial control signal indicating spatial control conditions for positions of each joint of a character at each frame in a motion sequence; and
creating, using a generative masked motion model, a physically plausible human motion sequence that aligns with the text prompt and follows the spatial control conditions.

### Claim 18 (depends on 17)

The computer-implemented method of claim 17, further comprising:
training the generative masked motion model using text training data and spatial control training data to learn a conditional distribution of motion tokens representing the joints of the character.

### Claim 19 (depends on 17)

The computer-implemented method of claim 17, further comprising:
controlling a robot to move according to the physically plausible human motion sequence.

### Claim 20 (depends on 17)

The computer-implemented method of claim 17, further comprising:
displaying animated graphics according to the physically plausible human motion sequence.

### Claim 21 (depends on 17)

The computer-implemented method of claim 17, wherein creating the physically plausible human motion sequence comprises:
processing a predicted conditional motion distribution of motion tokens representing the joints of the character so that generated motion, sampled from the plausible human motion sequence, adheres to the spatial control signal.

### Claim 22 (depends on 17)

The computer-implemented method of claim 17, wherein the text prompt includes descriptions of semantic guidance for motion generation.

### Claim 23 (depends on 17)

The computer-implemented method of claim 17, wherein the generative masked motion model includes:
a motion tokenizer; and
a text-conditioned masked transformer.

### Claim 24

A computer-implemented method comprises performing, by one or more processors, operations comprising:
receiving a text prompt and a music control signal indicating a rhythm and spatial control conditions for positions of each joint of a character at each frame in a motion sequence; and
creating, using a triple-stream masked motion model, a physically plausible human motion sequence that rhythmically aligns with the music control signal while maintaining spatial coherence.

### Claim 25 (depends on 24)

The computer-implemented method of claim 24, wherein the triple-stream masked motion model comprises a text-guided masked motion model, the method further comprising:
training the text-guided masked motion model using text training data to learn a conditional distribution of motion tokens representing the joints of the character based on the text training data.

### Claim 26 (depends on 25)

The computer-implemented method of claim 25, wherein the triple-stream masked motion model comprises a music-guided masked motion model, the method further comprising:
training the music-guided motion model using music control signal training data and the text training data to learn a conditional distribution of the motion tokens representing the joints of the character based on the music control signal training data and the text training data.

### Claim 27 (depends on 26)

The computer-implemented method of claim 26, wherein the triple-stream masked motion model comprises a pose-guided masked motion model, the method further comprising:
training the pose-guided motion model using pose control signal training data and the text training data to learn a conditional distribution of the motion tokens representing the joints of the character based on the pose control signal training data and the text training data.

### Claim 28 (depends on 27)

The computer-implemented method of claim 27, further comprising:
refining the motion tokens during interference to adjust rhythm synchronization and coherent alignment with multimodal inference inputs including an inference text prompt and an inference music control signal.

### Claim at position 22

29-36. (canceled)
