# US 10,311,624 B2 · Single shot capture to animated vr avatar

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,311,624 B2
- Document ID: US10311624B2
- Status: Granted
- Kind: B2
- Application number: 15/631,576
- Filed: 2017-06-23
- Published: 2018-12-27
- Granted: 2019-06-04
- Earliest claimed: 2017-06-23
- Assignees: Disney Enterprises, Inc.
- Applicants: Disney Enterprises, Inc.
- Inventors: Kenneth Mitchell; Charles Malleson; Ivan Huerta Casado; Martin Klaudiny; Malgorzata Edyta Kosek
- Web page: https://patentgrep.com/patent/US10311624B2

## CPC

- G06T 13/40: of characters, e.g. humans, animals or virtual beings
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06V 40/166: {using acquisition arrangements}
- G06V 40/169: {Holistic features and representations, i.e. based on the facial image taken as a whole}
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- G06T 2219/2016: Rotation, translation, scaling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Presented herein are systems and methods configured to generate virtual entities representing real-world users. In some implementations, the systems and/or methods are configured to capture user appearance information with imaging devices and sensors, determines correspondence values conveying correspondences between the appearance of the user's body or user's head and individual ones of default body models and/or default head models, modifies a set of values defining a base body model and/or base head model based on determined correspondence values and sets of base values defining the default body models and/or default head models. The base body model and/or base head model may be modified to model the appearance of the body and/or head of the user.

## Claims

### Claim 1

A system to generate virtual entities representing real-world users, the system comprising:
a depth sensor configured to generate output signals conveying depth information, the depth information including range of surfaces of objects in a real-world environment;
non-transitory electronic storage storing multiple default body models and a base body model, an individual default body model modeling an individual body type, wherein body types include a pear body shape type, a reverse pear body shape type, and a big bone body shape type, the individual body type being defined by an individual set of base values for a set of body modeling parameters, the base body model being defined by a set of values for the set of body modeling parameters; and
one or more physical processors configured by machine-readable instructions:
obtain user body information, the user body information being defined by the depth information, the user body information specifying an appearance of a body of a real-world user;
determine correspondence values conveying correspondences between the appearance of the user's body and individual ones of the default body models using the user body information and the sets of base values of the default body models, such that the correspondence values convey correspondences between the appearance of the user's body and individual ones of the pear body shape type, the reverse pear body shape type, and the big bone body shape type; and
modify the set of values defining the base body model based on determined correspondence values and the sets of base values defining the default body models such that the base body model is modified to model the appearance of the body of the user.

### Claim 2 (depends on 1)

The system of claim 1, wherein the individual set of base values define human body characteristics, the human body characteristics include the size and shape an upper body, a lower body, and an overall body size.

### Claim 3 (depends on 1)

The system of claim 1, wherein the user body information includes:
a first set of values for a first set of torso parameters and a first value for a height parameter, the first set of values for the first set of torso parameter being associated with the first value for the height parameter, wherein the first set of values for the first set of torso parameters include a second value for a width parameter, and a third value for a depth parameter;
a second set of values for the first set of torso parameters and a fourth value for the height parameter, the second set of values for the first set of torso parameters being associated with the fourth value for the height parameter, wherein the second set of values for the first set of torso parameters include a fifth value for the width parameter, and a sixth value for a depth parameter; and
a third set of values for the first set of torso parameters and a seventh value for the height parameter, the third set of values for the first set of torso parameters being associated with the seventh value for the height parameter, wherein the third set of values for the first set of torso parameters include a eighth value for the width parameter and a ninth value for the depth parameter.

### Claim 4 (depends on 3)

The system of claim 3, wherein a first correspondence value is determined based on the first set of values for the first set of torso parameter and the first value for the height parameter, a second correspondence value is determined based on the second set of values for the first set of torso parameters and the fourth value for the height parameter, and a third correspondence value is determined based on the third set of values for the first set of torso parameters and the seventh value for the height parameter.

### Claim 5 (depends on 1)

The system of claim 1, wherein a gender of the real-word user is determined based on the depth information or the image information.

### Claim 6 (depends on 1)

The system of claim 1, wherein an overall height of the base body model is modified based on an overall height of the user.

### Claim 7

A system to generate virtual entities representing real-world users, the system comprising:
a first imaging device configured to generate output signals conveying first image information, the first image information defining visual content present within a field-of-view of the first imaging device;
a second imaging device configured to generate output signals conveying second image information, the second image information defining visual content present within a field-of-view of the second imaging device;
non-transitory electronic storage storing multiple default head models and a base head model, an individual default head model modeling an individual head shape type and/or facial feature type, wherein head shape types include a first head shape type and a second head shape type, and facial feature types include a first facial feature type and a second facial feature type, the individual head type being defined by an individual set of base values for a set of head modeling parameters, the base head model being defined by a set of values for the set of head modeling parameters; and
one or more physical processors configured by machine-readable instructions:
obtain user head information, the user head information specifying an appearance of a head and face of a real-world user, the user head information being determined by the first image information and the second image information;
determine correspondence values conveying correspondences between the appearance of the head and face and individual ones of the default head models, such that the correspondence values convey correspondences between the appearance of the head and face and individual ones of the first head shape type, the second head shape type, the first facial feature type, and the second facial feature type; and
modify the set of values defining the base head model based on determined correspondence values and the sets of base values defining the default head models such that the base head model is modified to model the appearance of the head of the user.

### Claim 8 (depends on 7)

The system of claim 7, wherein the user head information includes visual content including one or more of the user's facial features, head shape, facial textures, or facial expressions.

### Claim 9 (depends on 7)

The system of claim 7, wherein the set of head modeling parameters includes one or more of an eye parameter, an eyebrow parameter, a nose parameter, a mouth parameter, an ear parameter, a chin parameter, a cheek parameter, a forehead parameter, a jawline parameter, or a head shape parameter.

### Claim 10 (depends on 7)

The system of claim 7, wherein the correspondence values are determined based on one or more of a feature map of the user head, the individual default head models, or the base head model.

### Claim 11 (depends on 10)

The system of claim 10, wherein the one or more physical processors are further configured by machine-readable instructions to:
determine a first two-dimensional feature map from the first imaging information, the first two-dimensional feature map specifying two-dimensional positions of facial features of the human face;
determine a second two-dimensional feature map from the second image information, the second two-dimensional feature map specifying two-dimensional positions of facial features of the human face; and
generate a three-dimensional feature map based on the first two-dimensional feature map and the second two-dimensional feature map, the three-dimensional feature map specifying three-dimensional positions of facial features of the human face.

### Claim 12 (depends on 7)

The system of claim 7, wherein the first imaging device and the second imaging device are positioned to capture the user head information.

### Claim 13

A method for generating virtual entities representing real-world users, the method being implemented a system comprising a depth sensor, non-transitory electronic storage, and one or more physical processors, the method comprising:
generating output signals conveying depth information, the depth information including range of surfaces of objects in a real-world environment;
storing multiple default body models and a base body model, an individual default body model modeling an individual body type, wherein body types include a pear body shape type, a reverse pear body shape type, and a big bone body shape type, the individual body type being defined by an individual set of base values for a set of body modeling parameters, the base body model being defined by a set of values for the set of body modeling parameters;
obtaining user body information, the user body information being defined by the depth information, the user body information specifying an appearance of a body of a real-world user;
determining correspondence values conveying correspondences between the appearance of the user's body and individual ones of the default body models using the user body information and the set of values of the sets of base values of the default body models, such that the correspondence values convey correspondences between the appearance of the user's body and individual ones of the pear body shape type, the reverse pear body shape type, and the big bone body shape type; and
modifying the set of values defining the base body model based on determined correspondence values and the sets of base values defining the default body models such that the base body model is modified to model the appearance of the body of the user.

### Claim 14 (depends on 13)

The method of claim 13, wherein the individual set of base values define human body characteristics, the human body characteristics include the size and shape an upper body, a lower body, and an overall body size.

### Claim 15 (depends on 13)

The method of claim 13, wherein the user body information includes:
a first set of values for a first set of torso parameters and a first value for a height parameter, the first set of values for the first set of torso parameter being associated with the first value for the height parameter, wherein the first set of values for the first set of torso parameters include a second value for a width parameter, and a third value for a depth parameter;
a second set of values for the first set of torso parameters and a fourth value for the height parameter, the second set of values for the first set of torso parameters being associated with the fourth value for the height parameter, wherein the second set of values for the first set of torso parameters include a fifth value for the width parameter, and a sixth value for a depth parameter; and
a third set of values for the first set of torso parameters and a seventh value for the height parameter, the third set of values for the first set of torso parameters being associated with the seventh value for the height parameter, wherein the third set of values for the first set of torso parameters include a eighth value for the width parameter and a ninth value for the depth parameter.

### Claim 16 (depends on 15)

The method of claim 15, wherein a first correspondence value is determined based on the first set of values for the first set of torso parameter and the first value for the height parameter, a second correspondence value is determined based on the second set of values for the first set of torso parameters and the fourth value for the height parameter, and a third correspondence value is determined based on the third set of values for the first set of torso parameters and the seventh value for the height parameter.

### Claim 17 (depends on 13)

The method of claim 13, wherein a gender of the real-word user is determined based on the depth information or the image information.

### Claim 18 (depends on 13)

The method of claim 13, wherein an overall height of the base body model is modified based on an overall height of the user.

### Claim 19

A method for generating virtual entities representing real-world users, the method being implemented a system comprising a first imaging device, a second imaging device, non-transitory electronic storage, and one or more physical processors, the method comprising:
generating output signals conveying first image information, the first image information defining visual content present within a field-of-view of the first imaging device;
generating output signals conveying second image information, the second image information defining visual content present within a field-of-view of the second imaging device;
storing multiple default head models and a base head model, an individual default head model modeling an individual head shape type and/or facial feature type, wherein head shape types include a first head shape type and a second head shape type, and facial feature types include a first facial feature type and a second facial feature type, the individual head type being defined by an individual set of base values for a set of head modeling parameters, the base head model being defined by a set of values for the set of head modeling parameters;
obtaining user head information, the user head information specifying an appearance of a head and face of a real-world user, the user head information being determined by the first image information and the second image information;
determining correspondence values conveying correspondences between the appearance of the head and face and individual ones of the default head models, such that the correspondence values convey correspondences between the appearance of the head and face and individual ones of the first head shape type, the second head shape type, the first facial feature type, and the second facial feature type; and
modifying the set of values defining the base head model based on determined correspondence values and the sets of base values defining the default head models such that the base head model is modified to model the appearance of the head of the user.

### Claim 20 (depends on 19)

The method of claim 19, wherein the user head information includes visual content including one or more of the user's facial features, head shape, facial textures, or facial expressions.

### Claim 21 (depends on 19)

The method of claim 19, wherein the set of head modeling parameters includes one or more of an eye parameter, an eyebrow parameter, a nose parameter, a mouth parameter, an ear parameter, a chin parameter, a cheek parameter, a forehead parameter, a jawline parameter, or a head shape parameter.

### Claim 22 (depends on 19)

The method of claim 19, wherein the correspondence values are determined based on one or more of a feature map of the user head, the individual default head models, or the base head model.

### Claim 23 (depends on 22)

The method of claim 22, further comprising:
determining a first two-dimensional feature map from the first imaging information, the first two-dimensional feature map specifying two-dimensional positions of facial features of the human face;
determining a second two-dimensional feature map from the second image information, the second two-dimensional feature map specifying two-dimensional positions of facial features of the human face; and
generating a three-dimensional feature map based on the first two-dimensional feature map and the second two-dimensional feature map, the three-dimensional feature map specifying three-dimensional positions of facial features of the human face.

### Claim 24 (depends on 19)

The method of claim 19, wherein the first imaging device and the second imaging device are positioned to capture the user head information.