# US 2021/0174585 A1 · Three-dimensional modeling toolkit

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0174585 A1
- Document ID: US20210174585A1
- Status: Published application
- Kind: A1
- Application number: 17/179,344
- Filed: 2021-02-18
- Published: 2021-06-10
- Earliest claimed: 2019-05-17
- Assignees: Standard Cyborg, Inc.
- Applicants: Standard Cyborg, Inc.
- Inventors: Jeffrey Huber; Aaron Thompson; Ricky Reusser; Dustin Dorroh; Eric Arneback; Garrett Spiegel
- Web page: https://patentgrep.com/patent/US20210174585A1

## CPC

- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06N 3/04: Architecture, e.g. interconnection topology
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06N 5/02: Knowledge representation; Symbolic representation
- G06T 7/0012: {Biomedical image inspection}
- G06T 17/10: Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/647: {by matching two-dimensional images to three-dimensional objects}
- G06V 40/171: {Local features and components; Facial parts (eye characteristics G06V40/18); Occluding parts, e.g. glasses; Geometrical relationships}
- G06T 2200/08: involving all processing steps from image acquisition to 3D model generation
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30036: Dental; Teeth
- G06T 2210/41: Medical
- G06V 2201/033: of skeletal patterns
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A 3D scanning toolkit to perform operations that include: accessing a first data stream at a client device, wherein the first data stream comprises at least image data; applying a bit mask to the first data stream, the bit mask identifying a portion of the image data; accessing a second data stream at the client device, the second data stream comprising depth data associated with the portion of the image data; generating a point cloud based on the depth data, the point cloud comprising a set of data points that define surface features of an object depicted in the first data stream; and causing display of a visualization of the point cloud upon a presentation of the first data stream at the client device.

## Claims

### Claim 1

A method comprising:
accessing an image depicting a dental arch of a user;
identifying, from the image, a set of features describing the dental arch of the user;
generating a voxel grid based on the set of features identified from the image, the voxel grid including occupancy probabilities representing a three-dimensional (3D) surface of the dental arch of the user; and
generating a 3D model of the dental arch of the user based on the voxel grid.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
generating a dental aligner customized for the user based on the 3D model of the dental arch of the user.

### Claim 3 (depends on 1)

The method of claim 1, wherein identifying the set of features describing the dental arch of the user comprises:
identifying regions of the image that each depict at least one respective tooth from the dental arch; and
using a neural network to determine features describing the at least one respective tooth depicted in each respective region of the image.

### Claim 4 (depends on 3)

The method of claim 3, further comprising:
assigning a respective label to each respective region of the image based on the features identified from the region, each respective label identifying a tooth type of the at least one respective tooth of the dental arch that is depicted in the respective region of the image.

### Claim 5 (depends on 1)

The method of claim 1, wherein a shape of the voxel grid is generated based on a camera projection matrix.

### Claim 6 (depends on 1)

The method of claim 1, wherein generating the 3D model of the dental arch comprises:
generating a triangle mesh based on the occupancy probabilities included in the voxel grid, the triangle mesh representing the 3D surface of the dental arch of the user.

### Claim 7 (depends on 6)

The method of claim 6, wherein generating the 3D model of the dental arch further comprises:
iteratively processing the triangle mesh through a mesh refinement process, the mesh refinement process comprising of a vertex alignment stage, graph convolution stage, and a vertex refinement stage.

### Claim 8

A system comprising:
one or more computer processors; and
one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause the system to perform operations comprising:
accessing an image depicting a dental arch of a user;
identifying, from the image, a set of features describing the dental arch of the user;
generating a voxel grid based on the set of features identified from the image, the voxel grid including occupancy probabilities representing a three-dimensional (3D) surface of the dental arch of the user; and
generating a 3D model of the dental arch of the user based on the voxel grid.

### Claim 9 (depends on 8)

The system of claim 8, the operations further comprising:
generating a dental aligner customized for the user based on the 3D model of the dental arch of the user.

### Claim 10 (depends on 8)

The system of claim 8, wherein identifying the set of features describing the dental arch of the user comprises:
identifying regions of the image that each depict at least one respective tooth from the dental arch; and
using a neural network to determine features describing the at least one respective tooth depicted in each respective region of the image.

### Claim 11 (depends on 10)

The system of claim 10, the operations further comprising:
assigning a respective label to each respective region of the image based on the features identified from the region, each respective label identifying a tooth type of the at least one respective tooth of the dental arch that is depicted in the respective region of the image.

### Claim 12 (depends on 8)

The system of claim 8, wherein a shape of the voxel grid is generated based on a camera projection matrix.

### Claim 13 (depends on 8)

The system of claim 8, wherein generating the 3D model of the dental arch comprises:
generating a triangle mesh based on the occupancy probabilities included in the voxel grid, the triangle mesh representing the 3D surface of the dental arch of the user.

### Claim 14 (depends on 13)

The system of claim 13, wherein generating the 3D model of the dental arch further comprises:
iteratively processing the triangle mesh through a mesh refinement process, the mesh refinement process comprising of a vertex alignment stage, graph convolution stage, and a vertex refinement stage.

### Claim 15

A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of one or more computing devices, cause the one or more computing devices to perform operations comprising:
accessing an image depicting a dental arch of a user;
identifying, from the image, a set of features describing the dental arch of the user;
generating a voxel grid based on the set of features identified from the image, the voxel grid including occupancy probabilities representing a three-dimensional (3D) surface of the dental arch of the user; and
generating a 3D model of the dental arch of the user based on the voxel grid.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, the operations further comprising:
generating a dental aligner customized for the user based on the 3D model of the dental arch of the user.

### Claim 17 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein identifying the set of features describing the dental arch of the user comprises:
identifying regions of the image that each depict at least one respective tooth from the dental arch; and
using a neural network to determine features describing the at least one respective tooth depicted in each respective region of the image.

### Claim 18 (depends on 17)

The non-transitory computer-readable medium of claim 17, the operations further comprising:
assigning a respective label to each respective region of the image based on the features identified from the region, each respective label identifying a tooth type of the at least one respective tooth of the dental arch that is depicted in the respective region of the image.

### Claim 19 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein a shape of the voxel grid is generated based on a camera projection matrix.

### Claim 20 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein generating the 3D model of the dental arch comprises:
generating a triangle mesh based on the occupancy probabilities included in the voxel grid, the triangle mesh representing the 3D surface of the dental arch of the user; and
iteratively processing the triangle mesh through a mesh refinement process, the mesh refinement process comprising of a vertex alignment stage, graph convolution stage, and a vertex refinement stage.
