# US 2025/0009474 A1 · Photo realistic rendering of smile image after treatment

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0009474 A1
- Document ID: US20250009474A1
- Status: Published application
- Kind: A1
- Application number: 18/898,489
- Filed: 2024-09-26
- Published: 2025-01-09
- Earliest claimed: 2018-05-21
- Assignees: Align Technology, Inc.
- Applicants: Align Technology, Inc.
- Inventors: Dmitry Yurievich Chekh; Samuel Blanco; Vladislav Andreevich Miryaha; Boris Aleksandrovich Vysokanov; Chad Clayton Brown; Christopher E. Cramer
- Web page: https://patentgrep.com/patent/US20250009474A1

## CPC

- A61C 7/002: {Orthodontic computer assisted systems}
- G06T 5/70: Denoising; Smoothing
- G06T 15/83: Phong shading
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- H04N 1/3871: {the composed originals being of different kinds, e.g. low- and high-resolution originals}
- H04N 1/646: {Transmitting or storing colour television type signals, e.g. PAL, Lab; Their conversion into additive or subtractive colour signals or vice versa therefor (H04N1/642, H04N1/644 take precedence)}
- A61C 2203/00: Means for making patients feel at ease during dental treatment, e.g. dental instruments in the form of toys
- G06T 2210/41: Medical
- G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes
- A61C: DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE (non-driven toothbrushes A46B; {tongue scrapers A61B17/24;} preparations for dentistry A61K6/00; preparations for cleaning the teeth or mouth A61K8/00, A61Q11/00)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A system comprises a memory and one or more processors operatively connected to the memory. The one or more processors receive a facial image comprising a two-dimensional (2D) depiction of a person's teeth at a first arrangement and receive a tooth model comprising a three-dimensional (3D) depiction of the person's teeth at a second arrangement. The one or more processors obtain color information from the facial image, wherein the color information represents a color scheme of the person's teeth at the first arrangement. The one or more processors generate a photo-realistic 2D depiction of the person's teeth at the second arrangement using the 3D depiction of the person's teeth at the second arrangement and the color information obtained from the facial image. The one or more processors send the photo-realistic 2D depiction of the person's teeth at the second arrangement for display on a display device.

## Claims

### Claim 1

A system comprising:
a memory; and
one or more processors operatively connected to the memory, the one or more processors configured to:
receive a facial image comprising a two-dimensional (2D) depiction of a person's teeth at a first arrangement;
receive a tooth model comprising a three-dimensional (3D) depiction of the person's teeth at a second arrangement;
obtain color information from the facial image, wherein the color information represents a color scheme of the person's teeth at the first arrangement;
generate a photo-realistic 2D depiction of the person's teeth at the second arrangement using the 3D depiction of the person's teeth at the second arrangement and the color information obtained from the facial image; and
send the photo-realistic 2D depiction of the person's teeth at the second arrangement for display on a display device.

### Claim 2 (depends on 1)

The system of claim 1, wherein the system comprises a mobile phone, the system further comprising:
a camera to capture the facial image;
wherein the one or more processors are to receive the facial image from the camera or the memory.

### Claim 3 (depends on 1)

The system of claim 1, wherein the facial image is received from a mobile phone.

### Claim 4 (depends on 1)

The system of claim 1, wherein to obtain the color information the one or more processors are to blur the 2D depiction of the person's teeth at the first arrangement.

### Claim 5 (depends on 1)

The system of claim 1, wherein to obtain the color information the one or more processors are to obtain one or more average values of pixels in the 2D depiction of the person's teeth at the first arrangement.

### Claim 6 (depends on 1)

The system of claim 1, wherein the tooth model comprises a 3D model of the person's teeth derived from an intraoral scan of the person's teeth.

### Claim 7 (depends on 1)

The system of claim 1, further comprising:
a network interface, wherein the system is configured to send the photo-realistic 2D depiction of the person's teeth to a remote device via the network interface, and wherein the remote device is configured to output the photo-realistic 2D depiction of the person's teeth to the display device.

### Claim 8 (depends on 1)

The system of claim 1, wherein the person's teeth at the second arrangement corresponds to an estimated arrangement of the person's teeth after treatment according to a treatment plan, and wherein the one or more processors are further configured to:
receive approval of the treatment plan based at least in part on the photo-realistic 2D depiction of the person's teeth at the second arrangement; and
implement the treatment plan.

### Claim 9 (depends on 1)

The system of claim 1, wherein the one or more processors are further configured to:
design one or more dental appliances to be applied to the person's teeth to effect a treatment plan: and
cause a fabrication machine to fabricate the one or more dental appliances.

### Claim 10 (depends on 1)

The system of claim 1, wherein the tooth model comprises a 3D model of the person's teeth derived from a plurality of 2D images of the person's teeth.

### Claim 11 (depends on 1)

The system of claim 1, wherein the one or more processors are further configured to:
code a plurality of color channels for a plurality of pixels of a 2D rendering of the tooth model with structural attributes of the 3D depiction; and
use a result of coding the plurality of color channels for the plurality of pixels of the 2D rendering to generate the photo-realistic 2D depiction of the person's teeth at the second arrangement.

### Claim 12 (depends on 1)

The system of claim 1, wherein the one or more processors are further configured to perform at least one of:
whiten the person's teeth in the photo-realistic 2D depiction of the person's teeth:
brighten the person's teeth in the photo-realistic 2D depiction of the person's teeth; or
add specular highlights to the photo-realistic 2D depiction of the person's teeth.

### Claim 13 (depends on 1)

The system of claim 1, wherein the color information comprises values for a first plurality of color channels, and wherein structural attributes of the 3D depiction are coded into a second plurality of channels.

### Claim 14

A non-transitory computer readable medium comprising instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving a facial image comprising a two-dimensional (2D) depiction of a person's teeth at a first arrangement;
receiving a tooth model comprising a three-dimensional (3D) depiction of the person's teeth at a second arrangement;
obtaining color information from the facial image, wherein the color information represents a color scheme of the person's teeth at the first arrangement;
generating a photo-realistic 2D depiction of the person's teeth at the second arrangement using the 3D depiction of the person's teeth at the second arrangement and the color information obtained from the facial image; and
sending the photo-realistic 2D depiction of the person's teeth at the second arrangement for display on a display device.

### Claim 15 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the facial image is captured by a camera of a mobile phone and is received from the mobile phone.

### Claim 16 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein obtaining the color information comprises blurring the 2D depiction of the person's teeth at the first arrangement.

### Claim 17 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein obtaining the color information comprises obtaining one or more average values of pixels in the 2D depiction of the person's teeth at the first arrangement.

### Claim 18 (depends on 14)

The non-transitory computer readable medium of claim 14, wherein the person's teeth at the second arrangement corresponds to an estimated arrangement of the person's teeth after treatment according to a treatment plan, the operations further comprising:
receiving approval of the treatment plan based at least in part on the photo-realistic 2D depiction of the person's teeth at the second arrangement; and
implementing the treatment plan.

### Claim 19 (depends on 14)

The non-transitory computer readable medium of claim 14, the operations further comprising:
coding a plurality of color channels for a plurality of pixels of a 2D rendering of the tooth model with structural attributes of the 3D depiction; and
using a result of coding the plurality of color channels for the plurality of pixels of the 2D rendering to generate the photo-realistic 2D depiction of the person's teeth at the second arrangement.

### Claim 20 (depends on 14)

The non-transitory computer readable medium of claim 14, the operations further comprising performing at least one of:
whitening the person's teeth in the photo-realistic 2D depiction of the person's teeth;
brightening the person's teeth in the photo-realistic 2D depiction of the person's teeth; or
adding specular highlights to the photo-realistic 2D depiction of the person's teeth.
