# US 10,503,963 B2 · System and method for image capture and modeling

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,503,963 B2
- Document ID: US10503963B2
- Status: Granted
- Kind: B2
- Application number: 15/836,578
- Filed: 2017-12-08
- Published: 2018-04-19
- Granted: 2019-12-10
- Earliest claimed: 2015-04-01
- Assignees: Take-Two Interactive Software, Inc.
- Applicants: Take-Two Interactive Software, Inc.
- Inventors: Timothy L. Meekins; Emre H. Yilmaz
- Web page: https://patentgrep.com/patent/US10503963B2

## CPC

- A63F 13/655: by importing photos, e.g. of the player
- A63F 13/00: Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence)
- A63F 13/25: Output arrangements for video game devices
- A63F 13/32: using local area network [LAN] connections
- A63F 13/327: using wireless networks, e.g. Wi-Fi® or piconet
- A63F 13/332: using wireless networks, e.g. cellular phone networks
- A63F 13/335: using Internet
- A63F 13/35: Details of game servers
- G06T 3/14: Transformations for image registration, e.g. adjusting or mapping for alignment of images
- G06T 7/20: Analysis of motion (motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51)
- G06T 7/55: from multiple images
- G06T 15/04: Texture mapping
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06V 40/165: {using facial parts and geometric relationships}
- G06V 40/168: {Feature extraction; Face representation}
- G06V 40/176: {Dynamic expression}
- G06T 2207/30201: Face
- G06V 20/64: Three-dimensional [3D] objects
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system and method for capturing a player's likeness on an in game model at runtime including geometry and texture.

## Claims

### Claim 1

A computer-implemented method for capturing a user's likeness on an in game model at runtime based on a set of images of the user, comprising:
isolating one or more features from the set of images to be matched, wherein each feature represents a group of pixels that can be found across at least one of the set of images;
matching a selected feature to said one or more isolated features;
generating a Fundamental Matrix for each image pair in the acquired set of images based on said matched features;
triangulating points of the set of images in a three-dimensional space based on said Fundamental Matrix;
aligning the triangulated points to a target model; and
generating texture for use on the aligned points to generate a first in game model at runtime.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
compressing the generated texture; and
uploading the compressed texture to a game server via a data processor.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, further comprising solving for a set of morph shapes on the target model based at least in part on said set of images.

### Claim 4 (depends on 1)

The computer-implemented method of claim 1, wherein said isolating one or more features includes limiting an area of each image of the set of images to a face tracking rectangle.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, further comprising eliminating said matched features that are outliers via a parameter estimation.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein said triangulating points further comprises a minimization selected from at least one of a bundle adjustment and a damped least-squares.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein said aligning the triangulated points further comprises at least one of rotating, scaling, translating, transforming, and sweeping the triangulated points to the target model.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein said generating texture further comprises blending said generated texture onto a preset head texture.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, wherein said isolating one or more features comprises defining features based on at least one of a Scale Invariant Feature Transform, a Speeded Up Robust Features, a Binary Robust Invariant Scalable Keypoints, a Fast Retina Keypoint, an Accelerated KAZE, a Gradient Location and Orientation Histogram, and a Histogram of Oriented Gradients.

### Claim 10

A system for capturing a user's likeness on an in game model at runtime based on a set of images of the user, comprising:
a database for maintaining the set of images of the user;
a platform being coupled to said database that executes a set of instructions stored on a non-transitory computer readable medium on the platform and executable by a data processor to cause the data processor to perform operations comprising:
isolating one or more features from the set of images to be matched, wherein each feature represents a group of pixels that can be found across at least one of the set of images;
matching a selected feature to said one or more isolated features;
triangulating points of the set of images in a three-dimensional space based on said matched features;
aligning the triangulated points to a target model;
generating texture for use on the aligned points to generate a first in game model at runtime; and
post-processing the aligned points on the first in game model.

### Claim 11 (depends on 10)

The system of claim 10, wherein said set of instructions further comprise instructions for:
compressing the generated texture; and
uploading the compressed texture to a game server in communication with the data processor.

### Claim 12 (depends on 10)

The system of claim 10, wherein said set of instructions further comprise instructions for solving for a set of morph shapes on the target model based at least in part on said set of images.

### Claim 13 (depends on 10)

The system of claim 10, wherein said platform is at least one of a dedicated game console, a biometrics system, a motion picture system, a medical application system, and a multimedia platform.

### Claim 14 (depends on 10)

The system of claim 10, wherein said set of instructions further comprise instructions for eliminating said matched features that are outliers via a parameter estimation.

### Claim 15 (depends on 10)

The system of claim 10, wherein said triangulating points further comprises a minimization selected from at least one of a bundle adjustment and a damped least-squares.

### Claim 16 (depends on 10)

The system of claim 10, wherein said aligning the triangulated points further comprises at least one of rotating, scaling, translating, transforming, and sweeping the triangulated points to the target model.

### Claim 17 (depends on 10)

The system of claim 10, wherein said generating texture further comprises blending said generated texture onto a preset head texture.

### Claim 18 (depends on 10)

The system of claim 10, wherein said isolating one or more features comprises defining features based on at least one of a Scale Invariant Feature Transform, a Speeded Up Robust Features, a Binary Robust Invariant Scalable Keypoints, a Fast Retina Keypoint, an Accelerated KAZE, a Gradient Location and Orientation Histogram, and a Histogram of Oriented Gradients.

### Claim 19

A computer-implemented method for capturing a user's likeness on an in game model at runtime based on a set of images of the user, comprising:
isolating one or more features from the set of images based on computer vision feature detection to obtain at least one feature descriptor, wherein a selected feature of a selected feature descriptor represents a group of pixels that can be found across at least one of the set of images;
matching a first feature of a selected image to said one or more isolated features to generate a Fundamental Matrix for each image pair in the set of images;
triangulating points of the set of images in a three-dimensional space based on the matched features and the Fundamental Matrix;
aligning the triangulated points to a target model based on at least one of robust alignment, coarse alignment, and an Iterative Closest Point translation; and
generating texture for use on the aligned points to generate a first in game model at runtime.