# US 10,117,581 B2 · Identification of areas of interest during intraoral scans

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,117,581 B2
- Document ID: US10117581B2
- Status: Granted
- Kind: B2
- Application number: 15/347,620
- Filed: 2016-11-09
- Published: 2017-02-23
- Granted: 2018-11-06
- Earliest claimed: 2014-05-07
- Assignees: Align Technology, Inc.
- Applicants: Align Technology, Inc.
- Inventors: Avi Kopelman; Michael Sabina; Adi Levin; Mikhail Minchenkov; Maria Anufrienko
- Web page: https://patentgrep.com/patent/US10117581B2

## CPC

- A61B 5/0088: {for oral or dental tissue}
- A61B 1/24: for the mouth, i.e. stomatoscopes, e.g. with tongue depressors; Instruments for opening or keeping open the mouth
- A61B 5/4547: {Evaluating teeth}
- A61B 5/4552: {Evaluating soft tissue within the mouth, e.g. gums or tongue}
- A61B 5/486: {Biofeedback (using electroencephalography [EEG] A61B5/375)}
- A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems}
- A61B 6/032: {Transmission computed tomography [CT]}
- A61B 6/469: {for selecting a region of interest [ROI]}
- A61B 6/51: for dentistry
- A61B 6/512: {Intraoral means}
- A61B 6/5241: {combining overlapping images of the same imaging modality, e.g. by stitching}
- A61C 7/002: {Orthodontic computer assisted systems}
- A61C 9/0053: {Optical means or methods, e.g. scanning the teeth by a laser or light beam}
- A61C 13/34: Making or working of models, e.g. preliminary castings, trial dentures; Dowel pins [4]
- G06T 7/0016: {involving temporal comparison}
- G06T 7/344: {involving models}
- G06T 7/55: from multiple images
- G06T 7/579: from motion
- A61B 2576/02: specially adapted for a particular organ or body part
- G06T 2200/24: involving graphical user interfaces [GUIs]
- G06T 2207/10016: Video; Image sequence
- G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30036: Dental; Teeth
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- A61C: DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE (non-driven toothbrushes A46B; {tongue scrapers A61B17/24;} preparations for dentistry A61K6/00; preparations for cleaning the teeth or mouth A61K8/00, A61Q11/00)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

During an intraoral scan session, a processing device receives a first intraoral image of a dental site and identifies a candidate intraoral area of interest from the first intraoral image. The processing device receives a second intraoral image of the dental site and verifies the first candidate intraoral area of interest as an intraoral area of interest based on comparison of the second intraoral image to the first intraoral image. The processing device then provides an indication of the intraoral area of interest during the intraoral scan session.

## Claims

### Claim 1

A computer readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:
receiving a plurality of intraoral images, the plurality of intraoral images comprising a first intraoral image of a dental site;
performing image registration between the plurality of intraoral images;
identifying, by the processing device, a first candidate intraoral area of interest from the first intraoral image;
determining a value associated with the first candidate intraoral area of interest;
selecting a threshold based on one or more patient case details, wherein the one or more patient case details comprise a procedure for the dental site;
receiving a second intraoral image of the dental site;
comparing the first intraoral image to the second intraoral image;
verifying, by the processing device, the first candidate intraoral area of interest as an intraoral area of interest responsive to determining that a) the second intraoral image confirms the first candidate intraoral area of interest and b) the value is above the threshold; and
providing an indication of the intraoral area of interest, wherein the intraoral area of interest is hidden in one or more views of the dental site, and wherein the indication of the intraoral area of interest is visible in the one or more views.

### Claim 2 (depends on 1)

The computer readable storage medium of claim 1, wherein determining that the second intraoral image confirms the first candidate intraoral area of interest comprises:
determining an alignment between the first intraoral image and the second intraoral image based on geometric features shared by the first intraoral image and the second intraoral image; and
determining that the second intraoral image does not comprise a surface corresponding to the first candidate intraoral area of interest.

### Claim 3 (depends on 1)

The computer readable storage medium of claim 1, wherein the first candidate intraoral area of interest has a first shape and a first size, the operations further comprising:
making a determination that a portion of the first candidate intraoral area of interest corresponds to a portion of a surface of the second intraoral image based on a comparison of the second intraoral image to the first intraoral image; and
modifying the first candidate intraoral area of interest so that it has at least one of a second shape or a second size that is smaller than the first size.

### Claim 4 (depends on 1)

The computer readable storage medium of claim 1, wherein the first intraoral image and the second intraoral image are received during an intraoral scan session, and wherein the indication of the intraoral area of interest is provided during the intraoral scan session, the operations further comprising:
computing a virtual model of the dental site after the intraoral scan session is complete based on a plurality of intraoral images, the plurality of intraoral images including the first intraoral image and the second intraoral image.

### Claim 5 (depends on 1)

The computer readable storage medium of claim 1, the operations further comprising:
receiving a new intraoral image comprising a region corresponding to the intraoral area of interest;
determining an alignment between the new intraoral image and at least one of the first intraoral image or the second intraoral image;
determining that the intraoral area of interest corresponds to a surface of the region in the new intraoral image; and
removing the indication of the intraoral area of interest.

### Claim 6 (depends on 1)

The computer readable storage medium of claim 1, wherein the indication of the intraoral area of interest comprises a flag that points to the intraoral area of interest.

### Claim 7 (depends on 1)

The computer readable storage medium of claim 1, the operations further comprising:
providing feedback to a user of an intraoral scanner that generated the first intraoral image and the second intraoral image, wherein the feedback indicates a change in at least one of a motion of the intraoral scanner or a speed of the intraoral scanner that will improve a quality of a scan session.

### Claim 8 (depends on 1)

The computer readable storage medium of claim 1, wherein the intraoral area of interest comprises a conflicting surface between the first intraoral image and the second intraoral image.

### Claim 9 (depends on 1)

The computer readable storage medium of claim 1, wherein verifying the first candidate intraoral area of interest as an intraoral area of interest further comprises:
determining that the candidate intraoral area of interest has one or more dimensions that violate a geometric criterion.

### Claim 10

A computer readable storage medium, comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:
receiving a plurality of intraoral images, the plurality of intraoral images comprising a first intraoral image of a dental site;
performing image registration between the plurality of intraoral images;
identifying, by the processing device, a first candidate intraoral area of interest from the first intraoral image, wherein identifying the first candidate intraoral area of interest comprises:
identifying a plurality of voxels in the first intraoral image that satisfy a criterion;
determining a subset of the plurality of voxels that are proximate to one another; and
grouping the subset of the plurality of voxels into a volume that comprises the first candidate intraoral area of interest;
determining a value associated with the first candidate intraoral area of interest;
receiving a second intraoral image of the dental site;
comparing the first intraoral image to the second intraoral image;
verifying, by the processing device, the first candidate intraoral area of interest as an intraoral area of interest responsive to determining that a) the second intraoral image confirms the first candidate intraoral area of interest and b) the value is above a threshold; and
providing an indication of the intraoral area of interest, wherein the intraoral area of interest is hidden in one or more views of the dental site, and wherein the indication of the intraoral area of interest is visible in the one or more views.

### Claim 11 (depends on 10)

The computer readable storage medium of claim 10, wherein the criterion comprises lack of image data and wherein the first candidate intraoral area of interest comprises a void in the first intraoral image.

### Claim 12 (depends on 11)

The computer readable storage medium of claim 11, the operations further comprising:
interpolating a shape for the void based on geometric features surrounding the void in at least one of the first intraoral image or the second intraoral image, wherein the indication of the intraoral area of interest comprises the shape of the void.

### Claim 13

A computer readable storage medium comprising instructions that, when executed by a processing device, cause the processing device to perform operations comprising:
receiving a plurality of intraoral images of a dental site;
performing image registration between the plurality of intraoral images;
identifying, by the processing device and based on the plurality of intraoral images that have undergone image registration, a plurality of voxels satisfying a criterion;
determining a subset of the plurality of voxels that are proximate to one another;
grouping the subset of the plurality of voxels into an intraoral volume of interest; and
generating, by the processing device, an indication of the intraoral volume of interest, wherein the intraoral volume of interest is hidden in one or more views of the dental site, and wherein the indication of the intraoral volume of interest is visible in the one or more views.

### Claim 14 (depends on 13)

The computer readable storage medium of claim 13, the operations further comprising:
determining a classification for the intraoral volume of interest;
determining at least one of a text, a color or an icon based on the classification; and
presenting the classification for the intraoral image in the indication using at least one of the text, the color or the icon.

### Claim 15 (depends on 13)

The computer readable storage medium of claim 13, the operations further comprising:
generating a first virtual model of the dental site based on a first scan session of the dental site taken at a first time;
generating a second virtual model of the dental site based on a second scan session of the dental site taken at a second time, wherein the plurality of intraoral images of the dental site are associated with the second scan session; and
determining a change to the dental site based on a comparison of the second virtual model to the first virtual model, wherein the intraoral volume of interest comprises the change.

### Claim 16 (depends on 15)

The computer readable storage medium of claim 15, wherein the change comprises at least one of tooth decay, receding gums, tooth wear, a broken tooth, gum disease, gum color, moles, lesions, tooth shade, tooth color, an improvement in orthodontic alignment or degradation in orthodontic alignment.

### Claim 17 (depends on 13)

The computer readable storage medium of claim 13, the operations further comprising:
generating a virtual model of the dental site based on the plurality of intraoral images, wherein the intraoral volume of interest is determined from the virtual model.

### Claim 18 (depends on 13)

The computer readable storage medium of claim 13, the operations further comprising:
determining an alignment of two or more of the plurality of intraoral images without generation of a virtual model; and
identifying the intraoral volume of interest based on a comparison of the two or more intraoral images.

### Claim 19 (depends on 13)

The computer readable storage medium of claim 13, wherein grouping the subset of the plurality of voxels into the intraoral volume of interest comprises:
grouping the plurality of voxels into a candidate intraoral volume of interest;
determining whether the candidate intraoral volume of interest has clinical importance based on a geometry of the candidate intraoral volume of interest; and
responsive to determining that the candidate intraoral volume of interest does have clinical importance, verifying the candidate intraoral volume of interest as the intraoral volume of interest.

### Claim 20 (depends on 13)

The computer readable storage medium of claim 13, wherein:
the intraoral volume of interest comprises at least one of an intraoral volume to be rescanned, an intraoral volume suggestive of a dental lesion, an intraoral volume suggestive of dental improvement, an intraoral volume suggestive of dental deterioration, or an intraoral volume suggestive of a foreign object; and
the indication of the intraoral volume of interest comprises at least one of a rescan sequence, a clinical importance, or a size.

### Claim 21

A system comprising:
a scanner to generate a plurality of intraoral images of a dental site; and
a computing device operatively coupled to the scanner, the computing device to:
receive the plurality of intraoral images, the plurality of intraoral images comprising a first intraoral image of the plurality of intraoral images;
perform image registration between the plurality of intraoral images;
identify a first candidate intraoral area of interest from the first intraoral image;
determine a value associated with the first candidate intraoral area of interest;
select a threshold based on one or more patient case details, wherein the one or more patient case details comprise a procedure for the dental site;
receive a second intraoral image of the plurality of intraoral images;
compare the first intraoral image to the second intraoral image;
verify the first candidate intraoral area of interest as an intraoral area of interest responsive to determining that a) the second intraoral image confirms the first candidate intraoral area of interest and b) the value is above the threshold; and
provide an indication of the intraoral area of interest, wherein the intraoral area of interest is hidden in one or more views of the dental site, and wherein the indication of the intraoral area of interest is visible in the one or more views.

### Claim 22 (depends on 21)

The system of claim 21, wherein to determine that the second intraoral image confirms the first candidate intraoral area of interest the computing device is to:
determine an alignment between the first intraoral image and the second intraoral image based on geometric features shared by the first intraoral image and the second intraoral image; and
determine that the second intraoral image does not comprise a surface corresponding to the first candidate intraoral area of interest.

### Claim 23 (depends on 21)

The system of claim 21, wherein the computing device is further to:
identify a second candidate intraoral area of interest from the first intraoral image;
make a determination that the second candidate intraoral area of interest corresponds to a surface of the second intraoral image based on a comparison of the second intraoral image to the first intraoral image; and
conclude that the second candidate intraoral area of interest is not an additional intraoral area of interest responsive to making the determination.

### Claim 24 (depends on 21)

The system of claim 21, wherein the first intraoral image is to be generated during a first scan session and the second intraoral image is to be generated during a second scan session.

### Claim 25 (depends on 21)

The system of claim 21, wherein the plurality of intraoral images are to be generated during an intraoral scan session, wherein the indication of the intraoral area of interest is to be presented during the intraoral scan session, and wherein the computing device is further to:
compute a virtual model of the dental site after the intraoral scan session is complete based on the plurality of intraoral images.

### Claim 26 (depends on 21)

The system of claim 21, wherein the computing device is further to:
receive a new intraoral image comprising a region corresponding to the intraoral area of interest;
determine an alignment between the new intraoral image and at least one of the first intraoral image or the second intraoral image;
determine that the intraoral area of interest corresponds to a surface of the region in the new intraoral image; and
remove the indication of the intraoral area of interest.

### Claim 27 (depends on 21)

The system of claim 21, wherein the indication of the intraoral area of interest comprises a flag that points to the intraoral area of interest.

### Claim 28 (depends on 21)

The system of claim 21, wherein the computing device is further to:
provide feedback to a user of an intraoral scanner that generated the first intraoral image and the second intraoral image, wherein the feedback indicates a change in at least one of a motion of the intraoral scanner or a speed of the intraoral scanner that will improve a quality of a scan session.

### Claim 29 (depends on 21)

The system of claim 21, wherein the intraoral area of interest comprises a conflicting surface between the first intraoral image and the second intraoral image.

### Claim 30 (depends on 21)

The system of claim 21, wherein to verify the first candidate intraoral area of interest as an intraoral area of interest the computing device is to determine that the candidate intraoral area of interest has one or more dimensions that violate a geometric criterion.

### Claim 31

A system comprising:
a scanner to generate a plurality of intraoral images of a dental site; and
a computing device operatively coupled to the scanner, the computing device to:
receive the plurality of intraoral images;
perform image registration between the plurality of intraoral images;
identify a first candidate intraoral area of interest from a first intraoral image of the plurality of intraoral images, wherein to identify the first candidate area of interest the computing device is to:
identify a plurality of voxels in the first intraoral image that satisfy a criterion;
determine a subset of the plurality of voxels that are proximate to one another; and
group the subset of the plurality of voxels into a volume that comprises the first candidate intraoral area of interest;
determine a value associated with the first candidate intraoral area of interest;
receive a second intraoral image of the dental site;
compare the first intraoral image to the second intraoral image;
verify the first candidate intraoral area of interest as an intraoral area of interest responsive to determining that a) the second intraoral image confirms the first candidate intraoral area of interest and b) the value is above a threshold; and
provide an indication of the intraoral area of interest, wherein the intraoral area of interest is hidden in one or more views of the dental site, and wherein the indication of the intraoral area of interest is visible in the one or more views.

### Claim 32 (depends on 31)

The system of claim 31, wherein the criterion comprises lack of image data and wherein the first candidate intraoral area of interest comprises a void in the first intraoral image.

### Claim 33

A system comprising:
a scanner to generate a plurality of intraoral images of a dental site; and
a computing device operatively coupled to the scanner, the computing device to:
receive the plurality of intraoral images of the dental site;
perform image registration between the plurality of intraoral images;
identify, based on the plurality of intraoral images that have undergone image registration, a plurality of voxels satisfying a criterion;
determine a subset of the plurality of voxels that are proximate to one another;
group the subset of the plurality of voxels into an intraoral volume of interest; and
generate an indication of the intraoral volume of interest, wherein the intraoral volume of interest is hidden in one or more views of the dental site, and
wherein the indication of the intraoral volume of interest is visible in the one or more views.

### Claim 34 (depends on 33)

The system of claim 33, wherein the computing device is further to:
determine a classification for the intraoral volume of interest;
determine at least one of a text, a color or an icon based on the classification; and
present the classification for the intraoral image in the indication using at least one of the text, the color or the icon.

### Claim 35 (depends on 33)

The system of claim 33, wherein the computing device is further to:
generate a first virtual model of the dental site based on a first scan session of the dental site taken at a first time;
generate a second virtual model of the dental site based on a second scan session of the dental site taken at a second time, wherein the plurality of intraoral images of the dental site are associated with the second scan session; and
determine a change to the dental site based on a comparison of the second virtual model to the first virtual model, wherein the intraoral volume of interest comprises the change.

### Claim 36 (depends on 35)

The system of claim 35, wherein the change comprises at least one of tooth decay, receding gums, tooth wear, a broken tooth, gum disease, gum color, moles, lesions, tooth shade, tooth color, an improvement in orthodontic alignment or degradation in orthodontic alignment.

### Claim 37 (depends on 33)

The system of claim 33, wherein the computing device is further to generate a virtual model of the dental site based on the plurality of intraoral images, wherein the intraoral volume of interest is determined from the virtual model.

### Claim 38 (depends on 33)

The system of claim 33, wherein the computing device is further to:
determine an alignment of two or more of the plurality of intraoral images without generation of a virtual model; and
identify the intraoral volume of interest based on a comparison of the two or more intraoral images.

### Claim 39 (depends on 33)

The system of claim 33, wherein to group the subset of the plurality of voxels into the intraoral volume of interest the computing device is to:
group the plurality of voxels into a candidate intraoral volume of interest;
determine whether the candidate intraoral volume of interest has clinical importance based on a geometry of the candidate intraoral volume of interest; and
responsive to determining that the candidate intraoral volume of interest does has clinical importance, verify the candidate intraoral volume of interest as the intraoral volume of interest.

### Claim 40 (depends on 33)

The system of claim 33, wherein the intraoral volume of interest comprises at least one of an intraoral volume to be rescanned, an intraoral volume suggestive of a dental lesion, an intraoral volume suggestive of dental improvement, an intraoral volume suggestive of dental deterioration, or an intraoral volume suggestive of a foreign object.

### Claim 41 (depends on 33)

The system of claim 33, wherein the indication of the intraoral volume of interest comprises at least one of a rescan sequence, a clinical importance, or a size.