# US 2020/0404243 A1 · Intraoral 3d scanner employing multiple miniature cameras and multiple miniature pattern projectors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0404243 A1
- Document ID: US20200404243A1
- Status: Published application
- Kind: A1
- Application number: 16/910,042
- Filed: 2020-06-23
- Published: 2020-12-24
- Earliest claimed: 2019-06-24
- Assignees: Align Technology, Inc.
- Applicants: Align Technology, Inc.
- Inventors: Ofer Saphier; Tal Levy; Gal Peleg; Eliran Dafna; Sergei Ozerov; Tal Verker; Nir Makmel; Yossef Atiya
- Web page: https://patentgrep.com/patent/US20200404243A1

## CPC

- A61C 9/006: {projecting one or more stripes or patterns on the teeth}
- H04N 13/204: using stereoscopic image cameras (stereoscopic photography G03B35/00)
- A61C 9/0053: {Optical means or methods, e.g. scanning the teeth by a laser or light beam}
- G01B 11/245: using a plurality of fixed, simultaneously operating transducers ({G01B11/2408 - G01B11/2425, } G01B11/255 take precedence)
- G01B 11/25: by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50})
- G01B 11/2513: {with several lines being projected in more than one direction, e.g. grids, patterns}
- G01B 11/2545: {with one projection direction and several detection directions, e.g. stereo}
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- H04N 1/00018: {Scanning arrangements}
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- A61B 5/0088: {for oral or dental tissue}
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30036: Dental; Teeth
- H04N 2013/0081: {Depth or disparity estimation from stereoscopic image signals}
- Y02A 90/10: Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
- A61C: DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE (non-driven toothbrushes A46B; {tongue scrapers A61B17/24;} preparations for dentistry A61K6/00; preparations for cleaning the teeth or mouth A61K8/00, A61Q11/00)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- Y02A: TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE

## Abstract

A method for generating a 3D image includes driving structured light projector(s) to project a pattern of light on an intraoral 3D surface, and driving camera(s) to capture images, each image including at least a portion of the projected pattern, each one of the camera(s) comprising an array of pixels. A processor compares a series of images captured by each camera and determines which of the portions of the projected pattern can be tracked across the images. The processor constructs a three-dimensional model of the intraoral three-dimensional surface based at least in part on the comparison of the series of images. Other embodiments are also described.

## Claims

### Claim 1

A method for generating a digital three-dimensional image, the method comprising:
driving each one of one or more structured light projectors to project a pattern on an intraoral three-dimensional surface;
driving each one of one or more cameras to capture a plurality of images, each image including at least a portion of the projected pattern; and
using a processor to:
compare a series of images captured by the one or more cameras;
determine which of the portions of the projected pattern can be tracked across the series of images based on the comparison of the series of images; and
construct a three-dimensional model of the intraoral three-dimensional surface based at least in part on the comparison of the series of images.

### Claim 2 (depends on 1)

The method according to claim 1, wherein using the processor further comprises using the processor to:
solve a correspondence algorithm for the tracked portions of the projected pattern in at least one of the series of images; and
use the solved correspondence algorithm for the tracked portions in the at least one of the series of images to solve the correspondence algorithm for the tracked portions of the projected pattern in at least another image of the series of images, wherein the solution to the correspondence algorithm is used to construct the three-dimensional model.

### Claim 3 (depends on 1)

The method according to claim 1, wherein using the processor further comprises using the processor to:
solve a correspondence algorithm for the tracked portions of the projected pattern based on positions of the tracked portions in each image throughout the series of images, wherein constructing the three-dimensional model comprises using the solution to the correspondence algorithm to construct the three-dimensional model.

### Claim 4 (depends on 1)

The method according to claim 1, wherein the one or more structured light projectors project a pattern that is spatially fixed relative to the one or more cameras.

### Claim 5 (depends on 1)

The method according to claim 1, wherein the projected pattern comprises a plurality of projected spots of light, and wherein the portion of the projected pattern corresponds to a projected spots of the plurality of projected spots of light.

### Claim 6 (depends on 5)

The method according to claim 5, wherein using the processor to compare the series of images comprises using the processor to compare the series of images based on stored calibration values indicating (a) a camera ray corresponding to each pixel on a camera sensor of each one of the one or more cameras, and (b) a projector ray corresponding to each one of the projected spots of light from each one of the one or more structured light projectors, wherein each projector ray corresponds to a respective path of pixels on at least one of the camera sensors, wherein determining which portions of the projected pattern can be tracked comprises determining which of the projected spots s can be tracked across the series of images, and wherein each tracked spot s moves along a path of pixels corresponding to a respective projector ray r.

### Claim 7 (depends on 6)

The method according to claim 6, wherein using the processor further comprises using the processor to determine, for each tracked spot s, a plurality of possible paths p of pixels on a given one of the cameras, paths p corresponding to a respective plurality of possible projector rays r.

### Claim 8 (depends on 7)

The method according to claim 7, wherein using the processor further comprises using the processor to run a correspondence algorithm to:
for each of the possible projector rays r:
identify how many other cameras, on their respective paths p1 of pixels corresponding to projector ray r, detected respective spots q corresponding to respective camera rays that intersect projector ray r and the camera ray of the given one of the cameras corresponding to the tracked spot s;
identify a given projector ray r1 for which the highest number of other cameras detected respective spots q; and
identify projector ray r1 as the particular projector ray r that produced the tracked spot s.

### Claim 9 (depends on 6)

The method according to claim 6, wherein using the processor further comprises using the processor to:
run a correspondence algorithm to compute respective three-dimensional positions of a plurality of detected spots on the intraoral three-dimensional surface, as captured in the series of images; and
in at least one of the series of images, identify a detected spot as being from a particular projector ray r by identifying the detected spot as being a tracked spots moving along the path of pixels corresponding to the particular projector ray r.

### Claim 10 (depends on 6)

The method according to claim 6, wherein using the processor further comprises using the processor to:
run a correspondence algorithm to compute respective three-dimensional positions of a plurality of detected spots on the intraoral three-dimensional surface, as captured in the series of images, and
remove from being considered as a point on the intraoral three-dimensional surface a spot that (i) is identified as being from particular projector ray r based on the three-dimensional position computed by the correspondence algorithm, and (ii) is not identified as being a tracked spot s moving along the path of pixels corresponding to particular projector ray r.

### Claim 11 (depends on 6)

The method according to claim 6, wherein using the processor further comprises using the processor to:
run a correspondence algorithm to compute respective three-dimensional positions of a plurality of detected spots on the intraoral three-dimensional surface, as captured in the series of images; and
for a detected spot which is identified as being from two distinct projector rays r based on the three-dimensional position computed by the correspondence algorithm, identify the detected spot as being from one of the two distinct projector rays r by identifying the detected spot as a tracked spot s moving along the one of the two distinct projector rays r.

### Claim 12 (depends on 6)

The method according to claim 6, wherein using the processor further comprises using the processor to:
run a correspondence algorithm to compute respective three-dimensional positions of a plurality of detected spots on the intraoral three-dimensional surface, as captured in the series of images, and
identify a weak spot whose three-dimensional position was not computed by the correspondence algorithm as being a projected spot from a particular projector ray r, by identifying the weak spot as being a tracked spot s moving along the path of pixels corresponding to particular projector ray r.

### Claim 13 (depends on 6)

The method according to claim 6, wherein using the processor further comprises using the processor to compute respective three-dimensional positions on the intraoral three-dimensional surface at an intersection of the projector ray r and the respective camera rays corresponding to the tracked spot s in each of the series of images across which spot s was tracked.

### Claim 14 (depends on 1)

The method according to claim 1, wherein the three-dimensional model is constructed using a correspondence algorithm, wherein the correspondence algorithm uses, at least in-part, the portions of the projected pattern that are determined to be trackable across the series of images.

### Claim 15 (depends on 1)

The method according to claim 1, wherein using the processor further comprises using the processor to:
determine a parameter of a tracked portion of the projected pattern in at least two adjacent images from the series of images, the parameter selected from the group consisting of: a size of the portion, a shape of the portion, an orientation of the portion, an intensity of the portion, and a signal-to-noise ratio (SNR) of the portion; and
based on the parameter of the tracked portion of the projected pattern in the at least two adjacent images, predict the parameter of the tracked portion of the projected pattern in a later image.

### Claim 16 (depends on 15)

The method according to claim 15, wherein using the processor further comprises, based on the predicted parameter of the tracked portion of the projected pattern, using the processor to search for the portion of the projected pattern having substantially the predicted parameter in the later image.

### Claim 17 (depends on 15)

The method according to claim 15, wherein the selected parameter is the shape of the portion of the projected pattern, and wherein using the processor further comprises using the processor to, based on the predicted shape of the tracked portion of the projected pattern, determine a search space in a next image in which to search for the tracked portion of the projected pattern.

### Claim 18 (depends on 17)

The method according to claim 17, wherein using the processor to determine the search space comprises using the processor to determine a search space in the next image in which to search for the tracked portion of the projected pattern, the search space having a size and aspect ratio based on a size and aspect ratio of the predicted shape of the tracked portion of the projected pattern.

### Claim 19 (depends on 15)

The method according to claim 15, wherein the selected parameter is the shape of the portion of the projected pattern, and wherein using the processor further comprises using the processor to:
based on a direction and distance that the tracked portion of the projected pattern has moved between the at least two adjacent images from the series of images, determine a velocity vector of the tracked portion of the projected pattern;
in response to the shape of the tracked portion of the projected pattern in at least one of the at least two adjacent images, predict the shape of the tracked portion of the projected pattern in a later image; and
in response to (i) the determination of the velocity vector of the tracked portion of the projected pattern in combination with (ii) the predicted shape of the tracked portion of the projected pattern, determine a search space in the later image in which to search for the tracked portion of the projected pattern.

### Claim 20 (depends on 15)

The method according to claim 15, wherein the parameter is the shape of the portion of the projected pattern, and wherein using the processor further comprises using the processor to:
based on a direction and distance that the tracked portion of the projected pattern has moved between the at least two adjacent images from the series of images, determine a velocity vector of the tracked portion of the projected pattern;
in response to the determination of the velocity vector of the tracked portion of the projected pattern, predict the shape of the tracked portion of the projected pattern in a later image; and
in response to (i) the determination of the velocity vector of the tracked portion of the projected pattern in combination with (ii) the predicted shape of the tracked portion of the projected pattern, determine a search space in the later image in which to search for the tracked portion of the projected pattern.

### Claim 21 (depends on 20)

The method according to claim 20, wherein using the processor comprises using the processor to predict the shape of the tracked portion of the projected pattern in the later image in response to (i) the determination of the velocity vector of the tracked portion of the projected pattern in combination with (ii) the shape of the tracked portion of the projected pattern in at least one of the two adjacent images.

### Claim 22 (depends on 1)

The method according to claim 1, wherein using the processor further comprises using the processor to:
based on a direction and distance that a tracked portion of the projected pattern has moved between two consecutive images in the series of images, determine a velocity vector of the tracked portion of the projected pattern; and
in response to the determination of the velocity vector of the tracked portion of the projected pattern, determine a search space in a later image in which to search for the tracked portion of the projected pattern.
