# US 12,602,889 B2 · Method and system of rendering a 3D image for automated facial morphing

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,602,889 B2
- Document ID: US12602889B2
- Status: Granted
- Kind: B2
- Application number: 17/001,682
- Filed: 2020-08-25
- Published: 2021-03-11
- Granted: 2026-04-14
- Earliest claimed: 2018-04-30
- Applicants: Mathew Powers; William A. Kennedy
- Inventors: Mathew Powers; William A. Kennedy
- Web page: https://patentgrep.com/patent/US12602889B2

## CPC

- G06T 7/344: {involving models}
- G06T 5/20: using local operators
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 15/04: Texture mapping
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G06V 40/162: {using pixel segmentation or colour matching}
- G06V 40/165: {using facial parts and geometric relationships}
- G06V 40/166: {using acquisition arrangements}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06T 2200/04: involving 3D image data
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30201: Face
- G06T 2210/44: Morphing
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

In one aspect, a computerized method for rendering a three-dimensional (3D) digital image for automated facial morphing includes the step of determining that a user's face is in a compliant state. The method includes the step of scanning of the user's face with a digital camera to obtain a set of digital images of the user's face. The method includes the step of implementing an analysis of the set of digital images and implementing a set of pre-rendering steps. Each digital image comprises a depth data, a red/green/blue (RGB) data, and a facemask data, implementing an iterative closest path (ICP) algorithm that correlates the set of digital images together by stitching together the cloud of points of the facemask data of each digital image and outputs a set of transformation matrices.

## Claims

### Claim 1

A computerized method for rendering a three-dimensional (3D) digital image for automated facial morphing comprising:
scanning of a user's face with a digital camera to obtain a set of digital images of the user's face;
determining that the user's face is in a compliant state;
implementing an analysis of the set of digital images and implementing a set of pre-rendering steps by:
wherein each digital image comprises a depth data, a red/green/blue (RGB) data, and a facemask data,
implementing an iterative closest path (ICP) algorithm that correlates the set of digital images together by stitching together a cloud of points of the facemask data of each digital image and outputs a set of transformation matrices,
implementing a truncated signed distance function (TSDF) algorithm on the set of the transformation matrices, wherein the TSDF algorithm represents each point of the transformation matrices in a regularized voxel grid and outputs a set of voxel representations as a one-dimension (1-D) array of voxels,
implementing a marching cubes algorithm that obtains each voxel representation of the 1-D array of voxels and creates a three-dimensional (3D) mesh out of the per-voxel values provided by the TSDF and outputs a mesh representation, wherein the mesh representation comprises a set of triangles and vertices,
implementing a cleaning algorithm that obtains the mesh representations and cleans a floating vertices and triangles and outputs a mesh, wherein the mesh comprises a set of scattered points with a normal per point,
implementing a Poisson algorithm on the mesh output and fills in any holes of the mesh, wherein the Poisson algorithm outputs a reconstructed mesh,
implementing a texture mapping algorithm that outputs a mask representation, wherein the mask representation illustrates each piece of the digital image with texture values and label values, and
implementing an image stitching algorithm that stitches the digital images with different lighting values to remove any seams; and outputs a single digital image with consistent lighting; and
rendering a 3D digital image of the user's face from the single digital image with consistent lighting,
wherein the compliant state comprises that a central axis of the user's face is perpendicular to a horizonal plane of the digital camera.

### Claim 2 (depends on 1)

The computerized method of claim 1, wherein the compliant state comprises that the user is looking straight ahead towards the digital camera, that there is no head hair in the user's face and that the user is not smiling.

### Claim 3 (depends on 1)

The computerized method of claim 1, wherein the points of each cloud of points are chosen by using an ARKit face mask to ensure that each of the points are only located on the user's face.

### Claim 4 (depends on 1)

The computerized method of claim 1, wherein the TSDF algorithm blends and matches together all the digital images.

### Claim 5 (depends on 4)

The computerized method of claim 4, wherein a set of meshes with triangles and vertices of the output of the ICP are blended and matched in one uniform object.

### Claim 6 (depends on 4)

The computerized method of claim 4, wherein the cleaning algorithm cleans a set of patches of a neck portion, and wherein the cleaning algorithm outputs a set of scattered points with a normal per point of the mesh.

### Claim 7 (depends on 1)

The computerized method of claim 1, wherein the cleaning algorithm cleans various patches on a back of a head portion.

### Claim 8 (depends on 1)

The computerized method of claim 1, wherein the texture mapping algorithm uses a cylinder to map a texture constructed by three digital images of a three different views.

### Claim 9 (depends on 1)

The computerized method of claim 1, wherein a 3D model is rendered using a custom library with an internal software development kit of a mobile device.

### Claim 10 (depends on 9)

The computerized method of claim 9, wherein the mobile device comprises a hardware-accelerated 3D graphic and compute shader application programming interface (API) to render the 3D digital image of the user's face.

### Claim 11 (depends on 10)

The computerized method of claim 10, wherein the 3D digital image of the user's face is displayed on the mobile device.

### Claim 12 (depends on 1)

The computerized method of claim 1 further comprises;
implementing one or more post rendering operations comprising at least one automated facial morphing process.

### Claim 13 (depends on 12)

The computerized method of claim 12, wherein the automated facial morphing process comprises an ability to segregate the user's face into regions for injectables; an under-eye bag removal; a blepharoplasty morphing; a jawline reduction; a beard detection; or an autocorrection of a crooked nose.