# US 12,277,662 B2 · Object alignment techniques for animation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,277,662 B2
- Document ID: US12277662B2
- Status: Granted
- Kind: B2
- Application number: 17/977,935
- Filed: 2022-10-31
- Published: 2024-05-02
- Granted: 2025-04-15
- Earliest claimed: 2022-10-31
- Assignees: Pixar; Disney Enterprises, Inc.
- Applicants: Pixar; Disney Enterprises, Inc.
- Inventors: Hayko Riemenschneider; Clara Fernandez; Joan Massich; Christopher Richard Schroers; Daniel Teece; Rasmus Tamstorf; Charles Tappan; Mark Meyer; Antoine Milliez
- Web page: https://patentgrep.com/patent/US12277662B2

## CPC

- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06T 7/33: using feature-based methods
- G06T 7/50: Depth or shape recovery
- G06T 13/20: Three-dimensional [3D] animation
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/82: using neural networks
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30196: Human being; Person
- G06T 2210/56: Particle system, point based geometry or rendering
- G06T 2219/2004: Aligning objects, relative positioning of parts
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Techniques for aligning object representations for animation include analyzing a source object representation and a target object representation to identify a category of the source object and a category of the target object. Based on the category or categories of the objects, a feature extraction machine learning models is selected. The source object representation and the target object representation are provided as input to the selected feature extraction machine learning model to generate respective semantic descriptors and shape vectors for the source and target objects. Based on the semantic descriptors and the shape vectors for the source and target objects, an alignment machine learning model generates an aligned target object representation that is aligned with the source object representation and usable for animating the target object.

## Claims

### Claim 1

A method for aligning objects for computer generated imagery, the method comprising performing, by a computer system:
obtaining a source object representation representing a source object and a target object representation representing a target object;
identifying a category of the source object and a category of the target object, wherein identifying the category of the source object and identifying the category of the target object comprises providing the object representations to a classifier machine learning model, wherein the classifier machine learning model is a neural network comprising:
an input layer for accepting an object representation corresponding to an object;
two or more mixed diffusion and convolution layers; and
an output layer for outputting a probability that the object is part of a particular category of objects;
based on the category of the source object and the category of the target object, selecting a feature extraction machine learning model from a set of feature extraction machine learning models for different object categories;
providing the source object representation and the target object representation as input to the selected feature extraction machine learning model;
generating, by the feature extraction machine learning model, information identifying parts of the source object, information identifying parts of the target object, a shape vector for the source object, and a shape vector for the target object; and
generating an aligned target object representation that is aligned with the source object representation and usable for animating the target object, the generating using an alignment machine learning model based on the information identifying parts of the source object, the information identifying parts of the target object, the shape vector for the source object, and the shape vector for the target object.

### Claim 2 (depends on 1)

The method of claim 1, wherein the feature extraction machine learning model further comprises:
a first branch that generates information identifying parts of the object; and
a second branch that generates a shape vector describing a shape of the object.

### Claim 3 (depends on 1)

The method of claim 1, wherein the two or more mixed diffusion and convolution layers are exactly four mixed diffusion and convolution layers.

### Claim 4 (depends on 1)

The method of claim 1, wherein the shape vector for the source object includes one or more of: a size of at least a portion of the source object, a volume of at least a portion of the source object, a curvature of at least a portion of the source object, and a length of at least a portion of the source object.

### Claim 5 (depends on 1)

The method of claim 1, wherein the category of the source object is one of: a human, a particular non-human species of animal, a biped, a quadruped, a set prop, furniture, or a tool.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
converting the source object representation from a first form to a third form; and
converting the target object representation from a second form to the third form.

### Claim 7 (depends on 6)

The method of claim 6, wherein the first form comprises a point cloud, a mesh, or a volume.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
identifying one or more user-defined weights, wherein
generating the aligned target object representation is further based on the weights.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
identifying information defined at locations of the source object representation;
modifying the aligned target object representation based on the information defined at the locations of the source object representation; and
using the modified aligned target object representation to generate an animation frame.

### Claim 10

A system for aligning objects for computer generated imagery, the system comprising one or more processors configured to:
obtain a source object representation representing a source object and a target object representation representing a target object;
identify a category of the source object and a category of the target object, wherein identifying the category of the source object and identifying the category of the target object comprises providing the object representations to a classifier machine learning model, wherein the classifier machine learning model is a neural network comprising:
an input layer for accepting an object representation corresponding to an object;
two or more mixed diffusion and convolution layers; and
an output layer for outputting a probability that the object is part of a particular category of objects;
based on the category of the source object and the category of the target object, select a feature extraction machine learning model from a set of feature extraction machine learning models for different object categories;
provide the source object representation and the target object representation as input to the selected feature extraction machine learning model;
generate, by the feature extraction machine learning model, information identifying parts of the source object, information identifying parts of the target object, a shape vector for the source object, and a shape vector for the target object; and
generate an aligned target object representation that is aligned with the source object representation and usable for animating the target object, the generating using an alignment machine learning model based on the information identifying parts of the source object, the information identifying parts of the target object, the shape vector for the source object, and the shape vector for the target object.

### Claim 11 (depends on 10)

The system of claim 10, wherein the feature extraction machine learning model further comprises:
a first branch that generates information identifying parts of the object; and
a second branch that generates a shape vector describing a shape of the object.

### Claim 12 (depends on 10)

The system of claim 10, wherein the two or more mixed diffusion and convolution layers are exactly four mixed diffusion and convolution layers.

### Claim 13 (depends on 10)

The system of claim 10, wherein the shape vector for the source object includes one or more of: a size of at least a portion of the source object, a volume of at least a portion of the source object, a curvature of at least a portion of the source object, and a length of at least a portion of the source object.

### Claim 14 (depends on 10)

The system of claim 10, wherein the category of the source object is one of: a human, a particular non-human species of animal, a biped, a quadruped, a set prop, furniture, or a tool.

### Claim 15 (depends on 10)

The system of claim 10, the one or more processors further configured to:
converting the source object representation from a first form to a third form; and
converting the target object representation from a second form to the third form.

### Claim 16 (depends on 15)

The system of claim 15, wherein the first form comprises a point cloud, a mesh, or a volume.

### Claim 17 (depends on 10)

The system of claim 10, the one or more processors further configured to:
identify one or more user-defined weights, wherein
generating the aligned target object representation is further based on the weights.

### Claim 18 (depends on 10)

The system of claim 10, the one or more processors further configured to:
identify information defined at locations of the source object representation;
modify the aligned target object representation based on the information defined at the locations of the source object representation; and
use the modified aligned target object representation to generate an animation frame.