# US 2024/0005567 A1 · Generation of images with tooth color determined using depth information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005567 A1
- Document ID: US20240005567A1
- Status: Published application
- Kind: A1
- Application number: 18/213,789
- Filed: 2023-06-23
- Published: 2024-01-04
- Earliest claimed: 2018-09-28
- Assignees: Align Technology, Inc.
- Applicants: Align Technology, Inc.
- Inventors: Yun Gao; Yingjie Li; Chao Shi
- Web page: https://patentgrep.com/patent/US20240005567A1

## CPC

- G06T 11/10: Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77)
- A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations
- A61C 9/004: {Means or methods for taking digitized impressions}
- A61C 9/0053: {Optical means or methods, e.g. scanning the teeth by a laser or light beam}
- A61C 13/0004: {Computer-assisted sizing or machining of dental prostheses}
- A61C 13/082: {Cosmetic aspects, e.g. inlays; Determination of the colour (A61C19/10 takes precedence)}
- A61C 13/34: Making or working of models, e.g. preliminary castings, trial dentures; Dowel pins [4]
- G01J 3/508: {measuring the colour of teeth}
- G06T 5/50: using two or more images, e.g. averaging or subtraction
- G06T 5/70: Denoising; Smoothing
- G06T 5/77: Retouching; Inpainting; Scratch removal
- G06T 7/50: Depth or shape recovery
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones}
- A61C 2201/002: using colour effect, e.g. for identification purposes
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30036: Dental; Teeth
- G06T 2210/41: Medical
- G06T 2219/2004: Aligning objects, relative positioning of parts
- G06T 2219/2016: Rotation, translation, scaling
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- A61C: DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE (non-driven toothbrushes A46B; {tongue scrapers A61B17/24;} preparations for dentistry A61K6/00; preparations for cleaning the teeth or mouth A61K8/00, A61Q11/00)
- G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N)

## Abstract

A method includes determining depth values associated with a first set of pixel locations in a first image of a mouth. One or more function is generated for one or more color channels based on intensities of the one or more color channels at the first set of pixel locations and depth values associated with the first set of pixel locations. Image data comprising a new representation of the teeth is received, wherein the image data comprises a second set of pixel locations and new depth values associated with the second set of pixel locations. A new image is generated based on the image data and the one or more functions.

## Claims

### Claim 1

(canceled)

### Claim 2

A system comprising:
a memory device; and
a processing device operatively coupled to the memory device, the processing device to:
receive or generate image data comprising a representation of teeth of an individual in a region of the image data, wherein the region comprises a set of pixel locations for the teeth, and wherein the image data further comprises depth values associated with pixel locations in the set of pixel locations; and
generate an image based at least in part on applying one or more functions to the image data, wherein the one or more functions output values for one or more color channels for the set of pixel locations based at least in part on the depth values.

### Claim 3 (depends on 2)

The system of claim 2, wherein the processing device is further to:
generate the one or more functions based on color information and depth information from a received image of a mouth of the individual.

### Claim 4 (depends on 3)

The system of claim 3, wherein the received image of the mouth is a two-dimensional (2D) image.

### Claim 5 (depends on 3)

The system of claim 3, wherein the depth information comprises depth values associated with pixels of the received image of the mouth that depict the teeth, and wherein the processing device is further to:
determine depth values associated with the pixels of the received image of the mouth that depict the teeth.

### Claim 6 (depends on 3)

The system of claim 3, wherein the color information comprises intensities of the one or more color channels at one or more pixels of the received image of the mouth that depict the teeth, and wherein the depth information comprises depth values associated with the pixels of the received image of the mouth that depict the teeth.

### Claim 7 (depends on 3)

The system of claim 3, wherein the received or generated image data comprises a first arrangement of the teeth of the individual, and wherein the received image of the mouth comprises a second arrangement of the teeth of the individual.

### Claim 8 (depends on 7)

The system of claim 7, wherein the first arrangement of the teeth is a post-treatment arrangement of the teeth, and wherein the second arrangement of the teeth is a current arrangement of the teeth.

### Claim 9 (depends on 3)

The system of claim 3, wherein the processing device is further to:
receive the received image of the mouth of the individual from a device comprising an image sensor.

### Claim 10 (depends on 2)

The system of claim 2, wherein the generated image is a blurred color image, and wherein the processing device is further to:
generate an additional image based on the generated image and the received or generated image data, wherein a shape of the teeth in the additional image is based on the received or generated image data and a color of the teeth in the additional image is based on the generated image.

### Claim 11 (depends on 10)

The system of claim 10, wherein generating the additional image comprises:
processing the image data and the generated image using a generative model, wherein the generative model outputs the additional image.

### Claim 12 (depends on 10)

The system of claim 10, wherein the processing device is further to:
transmit the additional image for display on a display device.

### Claim 13 (depends on 10)

The system of claim 10, wherein the processing device is further to:
output the additional image to a display device.

### Claim 14 (depends on 2)

The system of claim 2, wherein the image data is based on a three-dimensional (3D) model of an estimated post-treatment arrangement of the teeth.

### Claim 15 (depends on 14)

The system of claim 14, wherein the processing device is further to:
receive one or more additional two-dimensional (2D) images of a current arrangement of the teeth;
generate an initial 3D model of an estimation of the current arrangement of the teeth based on the one or more additional 2D images; and
perform a treatment simulation to generate the 3D model of the estimated post-treatment arrangement of the teeth based on the initial 3D model of the estimation of the current arrangement of the teeth.

### Claim 16 (depends on 14)

The system of claim 14, wherein the processing device is further to:
receive intraoral scan data of a current arrangement of the teeth;
generate an initial 3D model of the current arrangement of the teeth based on the intraoral scan data; and
perform a treatment simulation to generate the 3D model of the estimated post-treatment arrangement of the teeth based on the initial 3D model of the current arrangement of the teeth.

### Claim 17 (depends on 2)

The system of claim 2, wherein each of the one or more functions comprise a first variable for a first image axis, a second variable for a second image axis, and a third variable for a third image axis.

### Claim 18

A non-transitory computer readable medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:
receiving or generating image data comprising a representation of teeth of an individual in a region of the image data, wherein the region comprises a set of pixel locations for the teeth, and wherein the image data further comprises depth values associated with pixel locations in the set of pixel locations; and
generating an image based at least in part on applying one or more functions to the image data, wherein the one or more functions output values for one or more color channels for the set of pixel locations based at least in part on the depth values.

### Claim 19 (depends on 18)

The non-transitory computer readable medium of claim 18, wherein the generated image is a blurred color image, the operations further comprising:
generating an additional image based on the generated image and the received or generated image data, wherein a shape of the teeth in the additional image is based on the received or generated image data and a color of the teeth in the additional image is based on the generated image.

### Claim 20

A method comprising:
receiving or generating image data comprising a representation of teeth of an individual in a region of the image data, wherein the region comprises a set of pixel locations for the teeth, and wherein the image data further comprises depth values associated with pixel locations in the set of pixel locations; and
generating an image based at least in part on applying one or more functions to the image data, wherein the one or more functions output values for one or more color channels for the set of pixel locations based at least in part on the depth values.

### Claim 21 (depends on 20)

The method of claim 20, further comprising:
generating the one or more functions based on color information and depth information from a received image of a mouth of the individual.
