# US 2019/0174110 A1 · Method and apparatus for colour imaging a three-dimensional structure

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0174110 A1
- Document ID: US20190174110A1
- Status: Published application
- Kind: A1
- Application number: 16/270,419
- Filed: 2019-02-07
- Published: 2019-06-06
- Earliest claimed: 2004-06-17
- Assignees: Align Technology, Inc.
- Applicants: Align Technology, Inc.
- Inventors: Noam Babayoff
- Web page: https://patentgrep.com/patent/US20190174110A1

## CPC

- H04N 13/15: for colour aspects of image signals
- A61B 1/00009: {of image signals during a use of endoscope}
- A61B 1/00096: {Optical elements}
- A61B 1/0615: {for radial illumination}
- A61B 1/0638: {providing two or more wavelengths}
- A61B 1/0646: {with illumination filters}
- A61B 1/0676: {at distal tip of an endoscope}
- A61B 1/0684: {using light emitting diodes [LED]}
- A61B 1/24: for the mouth, i.e. stomatoscopes, e.g. with tongue depressors; Instruments for opening or keeping open the mouth
- A61B 1/247: with means for viewing areas outside the direct line of sight, e.g. dentists' mirrors
- A61B 5/0068: {Confocal scanning}
- A61B 5/0088: {for oral or dental tissue}
- A61B 5/1077: {Measuring of profiles}
- A61B 5/1079: {using optical or photographic means}
- A61C 9/0053: {Optical means or methods, e.g. scanning the teeth by a laser or light beam}
- A61C 9/0066: {Depth determination through adaptive focusing}
- A61C 19/04: Measuring instruments specially adapted for dentistry ({A61C5/44 takes precedence; testing vitality of teeth A61B5/0534;} radiation diagnosis specially adapted for dentistry A61B6/51)
- G01B 11/24: for measuring contours or curvatures
- G01B 11/25: by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50})
- G01J 3/02: Details
- G01J 3/0205: {Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows}
- G01J 3/0208: {using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction}
- G01J 3/0216: {using light concentrators or collectors or condensers}
- G01J 3/0218: {using optical fibers}
- G01J 3/0224: {using polarising or depolarising elements}
- G01J 3/0243: {having a through-hole enabling the optical element to fulfil an additional optical function, e.g. a mirror or grating having a throughhole for a light collecting or light injecting optical fiber}
- G01J 3/0256: {Compact construction}
- G01J 3/10: Arrangements of light sources specially adapted for spectrometry or colorimetry
- G01J 3/50: using electric radiation detectors
- G01J 3/501: {Colorimeters using spectrally-selective light sources, e.g. LEDs}
- G01J 3/508: {measuring the colour of teeth}
- G01N 21/255: {Details, e.g. use of specially adapted sources, lighting or optical systems}
- G06T 7/0012: {Biomedical image inspection}
- G06T 7/12: Edge-based segmentation
- G06T 7/90: Determination of colour characteristics
- H04N 13/207: using a single two-dimensional [2D] image sensor
- H04N 13/257: Colour aspects
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- H04N 13/296: Synchronisation thereof; Control thereof
- H10F 39/156: {CCD or CID colour image sensors}
- G01J 3/462: {Computing operations in or between colour spaces; Colour management systems}
- G01J 3/51: using colour filters
- G06T 2207/10024: Color image
- G06T 2207/10028: Range image; Depth image; 3D point clouds
- G06T 2207/30036: Dental; Teeth
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- A61C: DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE (non-driven toothbrushes A46B; {tongue scrapers A61B17/24;} preparations for dentistry A61K6/00; preparations for cleaning the teeth or mouth A61K8/00, A61Q11/00)
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N)
- G01N: INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES (measuring or testing apparatus or processes other than immunoassay, involving enzymes or microorganisms C12M, C12Q)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H10F: INORGANIC SEMICONDUCTOR DEVICES SENSITIVE TO INFRARED RADIATION, LIGHT, ELECTROMAGNETIC RADIATION OF SHORTER WAVELENGTH OR CORPUSCULAR RADIATION

## Abstract

A device for determining the surface topology and associated color of a structure, such as a teeth segment, includes a scanner for providing depth data for points along a two-dimensional array substantially orthogonal to the depth direction, and an image acquisition means for providing color data for each of the points of the array, while the spatial disposition of the device with respect to the structure is maintained substantially unchanged. A processor combines the color data and depth data for each point in the array, thereby providing a three-dimensional color virtual model of the surface of the structure. A corresponding method for determining the surface topology and associate color of a structure is also provided.

## Claims

### Claim 1

A system for determining surface topology and associated color of an intraoral structure, the system comprising:
a handheld imaging device comprising a probe, focusing optics configured to scan a focal plane of the handheld imaging device over a range of depths, an illuminator, and an image sensor configured to capture depth image data over the range of depths and color image data of the intraoral structure, wherein a time interval between capture of the depth image data and the color image data is within an interval between about 0 to about 20 milliseconds; and
one or more processors operably coupled to the hand-held imaging device, the one or more processors configured to cause the system to:
generate depth data of the intraoral structure portion using the depth image data from the handheld imaging device corresponding to a first spatial disposition relative to the intraoral structure,
generate color data of the intraoral structure using the color image data from the handheld imaging device corresponding to a second spatial disposition relative to the intraoral structure, and
provide a color three-dimensional numerical entity based on the depth data and the color data.

### Claim 2 (depends on 1)

The system of claim 1, wherein the second spatial disposition differs from the first spatial disposition by one or more of a translation along a depth direction or a rotation about the depth direction.

### Claim 3 (depends on 1)

The system of claim 1, wherein the one or more processors are further configured to cause the system to determine a spatial transformation between the depth data and the color data, the spatial transformation is determined by performing an alignment procedure.

### Claim 4 (depends on 1)

The system of claim 1, wherein the color three-dimensional numerical entity comprises a plurality of data points, each data point comprising three-dimensional surface coordinate data and color data associated therewith

### Claim 5 (depends on 1)

The system of claim 1, wherein the image sensor is a monochromatic image sensor.

### Claim 6 (depends on 1)

The system of claim 1, wherein the image sensor is a color image sensor.

### Claim 7 (depends on 1)

The system of claim 1, wherein the illuminator includes a plurality of LEDs.

### Claim 8 (depends on 1)

The system of claim 1, wherein the first and second spatial disposition are the same.

### Claim 9

A method for determining surface topology and associated color of an intraoral structure, the method comprising:
illuminating the intraoral structure with white light;
capturing depth image data and color image data of the structure portion using an image sensor of an imaging device, wherein the capturing occurs within a time interval between about 0 seconds and bout 20 milliseconds and includes scanning a focal plane of the imaging device over a range of depths;
generating, using the imaging device and the depth image data, depth data of the intraoral structure corresponding to a first spatial disposition of the imaging device relative to the intraoral structure;
generating, using the imaging device and the color image data, color data of the intraoral structure corresponding to a second spatial disposition of the imaging device relative to the intraoral structure; and
providing, using the processors, a color three-dimensional numerical entity based on the depth data and the color data.

### Claim 10 (depends on 9)

The method of claim 9, wherein the second spatial disposition differs from the first spatial disposition by one or more of a translation along a depth direction or a rotation about the depth direction.

### Claim 11 (depends on 9)

The method of claim 9, further comprising determining, using one or more processors, a spatial transformation between the depth data and the color data, wherein determining the spatial transformation comprises performing an alignment procedure.

### Claim 12 (depends on 9)

The method of claim 9, wherein the first and second spatial dispositions are substantially the same relative to the intraoral structure.

### Claim 13 (depends on 11)

The method of claim 11, wherein performing the alignment procedure comprises:
determining a first shape profile based on the depth data;
determining a second shape profile based on the color data; and
aligning the first shape profile to the second shape profile.

### Claim 14 (depends on 9)

The method of claim 9, wherein the first and second spatial dispositions are substantially fixed with respect to the structure portion.

### Claim 15 (depends on 9)

The method of claim 9, wherein the image sensor is a color image sensor.

### Claim 16

A system for determining surface topology and associated color of an intraoral structure, the system comprising:
a handheld imaging device comprising a probe, focusing optics configured to scan a focal plane of the handheld imaging device over a range of depths, an illuminator, and an image sensor configured to capture depth image data over the range of depths and color image data of the intraoral structure within an intraoral cavity of a patient, and further configured to capture the depth image data and the color image data at the same angle and orientation with respect to the intraoral structure portion; and
one or more processors operably coupled to the imaging device, the one or more processors configured to cause the system to:
generate depth data of the intraoral structure portion using the depth image data from the imaging device,
generate color data of the intraoral structure using the color image data from the imaging device, and
provide a color three-dimensional numerical entity based on the depth data and the color data.

### Claim 17 (depends on 16)

The system of claim 16, wherein the color three-dimensional numerical entity comprises a plurality of data points, each data point comprising three-dimensional surface coordinate data and color data associated therewith

### Claim 18 (depends on 16)

The system of claim 16, wherein a time interval between capture of the depth image data and the color image data is within an interval between 0 to 20 milliseconds.

### Claim 19 (depends on 16)

The system of claim 16, wherein the image sensor is a color image sensor.

### Claim 20 (depends on 16)

The system of claim 16, wherein the illuminator includes a plurality of LEDs.

### Claim 21 (depends on 16)

The system of claim 16, wherein the illuminator is a white light source.

### Claim 22 (depends on 16)

The system of claim 16, wherein the handheld imaging device is configured to capture the color image data and the depth data in parallel planes.

### Claim 23

A method for determining surface topology and associated color of an intraoral structure, the method comprising:
illuminating the intraoral structure;
scanning a focus of a handheld imaging device at a plurality of focal planes of a range of distances;
capturing depth image data and color image data of the structure portion over the range of distances using an image sensor of the handheld imaging device, wherein the capturing of the depth image data and the color image data occurs at the same first angle and orientation with respect to the intraoral structure;
generating, using the imaging device and the depth image data, the depth data at the first angle and orientation with respect to the intraoral structure;
generating, using the imaging device and the color image data, the color data at the first angle and orientation with respect to the intraoral structure; and
providing, using the processors, a color three-dimensional numerical entity based on the depth data and the color data.

### Claim 24 (depends on 23)

The method of claim 23 wherein providing color three-dimensional numerical entity based on the depth data and the color data comprises generating a plurality of data points of the numerical entity, each data point comprising three-dimensional surface coordinate data and color data associated therewith

### Claim 25 (depends on 23)

The method of claim 23, wherein a time interval between capture of the depth image data and the color image data is within an interval between 0 to 20 milliseconds.

### Claim 26 (depends on 23)

The method of claim 23, wherein the image sensor is a color image sensor.

### Claim 27 (depends on 23)

The method of claim 23, wherein illuminating the intraoral structure includes illuminating the intraoral structure with a while light source.

### Claim 28 (depends on 23)

The method of claim 23, wherein capturing the depth image data comprises:
capturing depth image data at a plurality of first of the plurality of focal planes.

### Claim 29 (depends on 28)

The method of claim 28, wherein capturing the color image data comprises:
capturing color image data at a plurality of second of the plurality of focal planes.

### Claim 30 (depends on 29)

The method of claim 29, wherein the first of the plurality of focal planes and the second of the plurality of focal planes are parallel.
