# US 2026/0065562 A1 · Generating motion from text in content generation systems and applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0065562 A1
- Document ID: US20260065562A1
- Status: Published application
- Kind: A1
- Application number: 18/819,169
- Filed: 2024-08-29
- Published: 2026-03-05
- Earliest claimed: 2024-08-29
- Assignees: NVIDIA Corporation
- Applicants: Nvidia Corporation
- Inventors: Xue Bin Peng; Jonathan Tseng; Davis Winston Rempe; Or Litany; Ye Yuan; Umar Iqbal; Sanja Fidler; Jan Kautz
- Web page: https://patentgrep.com/patent/US20260065562A1

## CPC

- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/0475: Generative networks
- G06N 3/088: Non-supervised learning, e.g. competitive learning
- G06N 3/092: Reinforcement learning
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

Approaches presented herein provide for the use of reinforcement learning to fine-tune a generative model, such as a motion diffusion model, for a specific objective, such as to generate representations of human motion corresponding to provided text input. A discriminator can be used to guide the training of the generative model. In at least one embodiment, the discriminator can compare the input text and generated motion representation (or embeddings of each) to determine an alignment value or match score, for example, which can then be used to adjust the network parameters or weights of the generative model to improve the alignment between input text and generated motion.

## Claims

### Claim 1

A computer-implemented method, comprising:
generating, using a generative model and based on input text, a representation of motion specified by the text;
comparing, using a discriminator, the text and the representation of motion to calculate an alignment value; and
updating one or more network parameters for the generative model based in part on the alignment value.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the discriminator includes a first network to generate a text embedding for the received text and a second network to generate a motion embedding for the representation of motion.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the discriminator is to calculate the alignment value based in part on a dot product of the text embedding and the motion embedding.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein one or more updates to the one or more network parameters are calculated using advantage-weighted regression.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, wherein the generative model is at least one of a diffusion model, a transformer-based model, a variational autoencoder-based model, or a generative adversarial network.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, wherein the representation is one of an image sequence, a video segment, or a virtual manipulation of a three-dimensional model.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, further comprising:
calculating a loss value for the generated representation of motion using a loss function having a loss term corresponding to the alignment value; and
determining one or more updates to the one or more network parameters based in part on the loss value.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein the text further includes at least one qualifier indicating a type, a style, or a rate of performance of the motion.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein the discriminator is a Contrastive Language-Image Pre-Training (CLIP)-based discriminator.

### Claim 10

A processor, comprising:
one or more circuits to:
generate, using a generative model, a representation of anthropomorphic motion associated with input text;
perform, using a discriminator, a comparison of the text and the representation of motion; and
update one or more network parameters for the generative model based in part on a result of the comparison.

### Claim 11 (depends on 10)

The processor of claim 10, wherein the one or more network parameters are updated as part of a reinforcement learning-based training process to fine-tune the generative model.

### Claim 12 (depends on 10)

The processor of claim 10, wherein the generative model is a motion diffusion model.

### Claim 13 (depends on 10)

The processor of claim 10, wherein the discriminator includes a first network to generate a text embedding for the received text and a second network to generate a motion embedding for the representation of motion.

### Claim 14 (depends on 13)

The processor of claim 13, wherein the discriminator is to calculate the alignment value based in part on a dot product of the text embedding and the motion embedding.

### Claim 15 (depends on 10)

The processor of claim 10, wherein the processor is comprised in at least one of:
a system for performing simulation operations;
a system for performing simulation operations to test or validate autonomous machine applications;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for rendering graphical output;
a system for performing deep learning operations;
a system implemented using an edge device;
a system for generating or presenting virtual reality (VR) content;
a system for generating or presenting augmented reality (AR) content;
a system for generating or presenting mixed reality (MR) content;
a system incorporating one or more Virtual Machines (VMs);
a system implemented at least partially in a data center;
a system for performing hardware testing using simulation;
a system for synthetic data generation;
a system for performing generative operations using a large language model (LLM);
a system for performing generative operations using a vision language model (VLM);
a collaborative content creation platform for 3D assets; or
a system implemented at least partially using cloud computing resources.

### Claim 16

A system, comprising:
one or more processors to use a motion diffusion model to generate a representation of anthropomorphic motion corresponding to a text input, the motion diffusion model being updated using a discriminator to calculate an alignment value, between the text input and the representation of human motion, to be used to determine an update to one or more network parameters for the motion diffusion model.

### Claim 17 (depends on 16)

The system of claim 16, wherein the one or more network parameters are updated as part of a reinforcement learning-based training process to fine-tune the motion diffusion model.

### Claim 18 (depends on 16)

The system of claim 16, wherein the alignment value is calculated based in part upon a first encoding generated for the text input and a second encoding generated for the representation of human motion.

### Claim 19 (depends on 16)

The system of claim 16, wherein the text further includes at least one qualifier indicating a type, a style, or a rate of performance of the motion.

### Claim 20 (depends on 16)

The system of claim 16, wherein the system comprises at least one of:
a system for performing simulation operations;
a system for performing simulation operations to test or validate autonomous machine applications;
a system for performing digital twin operations;
a system for performing light transport simulation;
a system for rendering graphical output;
a system for performing deep learning operations;
a system for performing generative operations using a large language model (LLM);
a system for performing generative operations using a vision language model (VLM);
a system implemented using an edge device;
a system for generating or presenting virtual reality (VR) content;
a system for generating or presenting augmented reality (AR) content;
a system for generating or presenting mixed reality (MR) content;
a system incorporating one or more Virtual Machines (VMs);
a system implemented at least partially in a data center;
a system for performing hardware testing using simulation;
a system for synthetic data generation;
a collaborative content creation platform for 3D assets; or
a system implemented at least partially using cloud computing resources.
