# US 12,178,672 B2 · Method for exporting a three-dimensional esthetic dental design model from an augmented reality application to a computer-aided design application > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,178,672 B2 - Document ID: US12178672B2 - Status: Granted - Kind: B2 - Application number: 17/720,683 - Filed: 2022-04-14 - Published: 2022-10-27 - Granted: 2024-12-31 - Earliest claimed: 2021-04-23 - Assignees: Ivoclar Vivadent AG - Applicants: Ivoclar Vivadent AG - Inventors: Marcel Lancelle - Web page: https://patentgrep.com/patent/US12178672B2 ## CPC - A61C 13/0004: {Computer-assisted sizing or machining of dental prostheses} - G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration - A61B 5/0088: {for oral or dental tissue} - A61B 5/4547: {Evaluating teeth} - A61B 90/36: {Image-producing devices or illumination devices not otherwise provided for} - A61C 7/002: {Orthodontic computer assisted systems} - A61C 9/0053: {Optical means or methods, e.g. scanning the teeth by a laser or light beam} - G06F 30/10: Geometric CAD - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - A61B 2090/365: {augmented reality, i.e. correlating a live optical image with another image} - G06F 2111/18: using virtual or augmented reality - G06F 2111/20: Configuration CAD, e.g. designing by assembling or positioning modules selected from libraries of predesigned modules - G06T 2207/30036: Dental; Teeth - G06T 2219/2004: Aligning objects, relative positioning of parts - G06T 2219/2016: Rotation, translation, scaling - A61C: DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE (non-driven toothbrushes A46B; {tongue scrapers A61B17/24;} preparations for dentistry A61K6/00; preparations for cleaning the teeth or mouth A61K8/00, A61Q11/00) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A method for exporting a three-dimensional dental design model based on a three-dimensional dental library model (4) from an augmented reality application (6) which is adapted to visualize an image of the dental library model (4) rendered by a virtual camera with a preliminary pose and scale in a photo (2) of a face of a patient taken by a camera under a viewing axis to the face of the patient, the photo (2) including a mouth opening showing at least part of the present situation of the patient's dentition. ## Claims ### Claim 1 A method for exporting a three-dimensional dental design model based on a three-dimensional dental library model (4) from an augmented reality application (6), the augmented reality application (6) being adapted to visualize on a display (7) an image of the dental library model (4) rendered by a virtual camera with a preliminary pose and scale in a photo (2) of a face of a patient taken by a camera under a viewing axis to the face of the patient, the photo (2) including a mouth opening showing at least part of the present situation of the patient's dentition, the method comprising: placing within the augmented reality application (6) the rendered image of the dental library model (4) on the photo (2) and providing tools to modify and/or move the dental library model (4) on the display (7) to obtain a desired dental design model (8), providing a three-dimensional scan model (10) of the present situation of the patient's dentition, aligning the scan model (10) and the present situation of the patient's dentition in the photo (2) to obtain a pose of the scan model (10), calculating, based on the preliminary pose and scale and the pose of the scan model (10), at least one transformation to bring the desired dental design model (8) and the scan model (10) into a common coordinate system to obtain a positioned dental model (12), adapting the positioned dental design model (12) to the scan model (10) with non-rigid deformations by minimizing a measure of shape difference between the positioned dental design model (12) and the scan model (10) to obtain a modified dental design model (14) as the dental design model for export, and determining for the desired dental design model (12) visible and non-visible regions thereof by taking into account the position in respect to the mouth opening and the viewing axis, wherein weighted non-rigid deformations are used to adapt the desired dental design model (12) to the scan model (10), wherein movements caused by the non-rigid deformations and which are perpendicular to the viewing axis are restricted more for the visible regions than such movements for the non-visible regions which is achieved by applying low weights to movements perpendicular to the viewing axis of the visible regions and higher weights to movements perpendicular to the viewing axis of the non-visible regions. ### Claim 2 (depends on 1) The method according to claim 1, comprising wherein the visible regions comprise front regions including incisors and the non-visible regions comprise rear regions including molars, wherein weighted non-rigid deformations are used to adapt the positioned dental design model (12) to the scan model (10), movements caused by the non-rigid deformations and which are perpendicular to the viewing axis are restricted more for the front regions including incisors than such movements for the rear regions including molars which is achieved by applying low weights to movements perpendicular to the viewing axis of the front regions and higher weights to movements perpendicular to the viewing axis of the rear regions. ### Claim 3 (depends on 2) The method according to claim 2, comprising wherein the weights used in the weighted non-rigid deformations and applied to movements perpendicular to the viewing axis of the positioned dental design model (12), gradually increase from the front regions including incisors to the rear regions including molars. ### Claim 4 (depends on 1) The method according to claim 1, comprising wherein the weights used in the weighted non-rigid deformations and applied to movements perpendicular to the viewing axis of the positioned dental design model (12), gradually increase from the visible regions to the non-visible regions. ### Claim 5 (depends on 1) The method according to claim 1, comprising applying weighted non-rigid deformations, wherein by applying weighted non-rigid deformations, movements along the viewing axis are allowed for the positioned dental design model (12) by allowing to apply the same high weights to the movements along the viewing axis of the positioned dental design model (12). ### Claim 6 (depends on 1) The method according to claim 1, comprising, wherein the non-rigid deformations comprise shearing, skewing, uniform and/or non-uniform scaling. ### Claim 7 (depends on 1) The method according to claim 1, comprising, exporting the modified dental design model (14) to an other application, wherein exporting comprises one or more of the following: generating an export file comprising the modified dental design model (14), the modified dental design model (14) is exported in a file together with the scan model (10), the other application is an external application, the other application is a computer-aided design application (16), and/or the export file is in STL file format. ### Claim 8 (depends on 1) A computer programme product comprising programme code which is stored on a non-transitory machine-readable medium, the non-transitory machine-readable medium comprising computer-executable instructions for performing the method according to claim 1. ### Claim 9 A method for exporting a three-dimensional dental design model based on a three-dimensional dental library model (4) from an augmented reality application (6), the method comprising: retrieving a three-dimensional scan model (10) of the present situation of a patient's dentition, adapting the three-dimensional dental library model (4) to the scan model (10) with non-rigid deformations by minimizing a measure of shape difference between the dental library model (4) and the scan model (10) to obtain a modified dental library model (5), placing within the augmented reality application (6) an image of the modified dental library model (5), rendered by a virtual camera with a preliminary pose and scale in a photo (2) of the face of the patient taken by a camera under a viewing axis to the face of the patient, the photo (2) including a mouth opening showing at least part of the present situation of the patient's dentition, on the photo (2) and providing tools to modify and/or move the modified dental library model (5) on a display (7) to obtain a desired dental design model (8), aligning the scan model (10) and the present situation of the patient's dentition in the photo (2) to obtain a pose of the scan model (10), calculating, based on the preliminary pose and scale and the pose of the scan model (10), at least one transformation to bring the desired dental design model (8) and the scan model (10) into a common coordinate system, wherein the desired dental design model (8) is brought to a coordinate system of the scan model (10) to obtain a positioned dental design model (12) wherein the non-rigid deformations are weighted and are used to adapt the three-dimensional dental library model (4) to the scan model (10). ### Claim 10 (depends on 9) The method according to claim 9, comprising wherein the non-rigid deformations comprise shearing, skewing, uniform and/or non-uniform scaling. ### Claim 11 (depends on 9) The method according to claim 9, comprising exporting the positioned dental design model (12) to an other application, wherein exporting comprises one or more of the following: generating an export file comprising the positioned dental design model (12), the positioned dental design model (12) is exported in a file together with the scan model (10), the other application is an external application, the other application is a computer-aided design application (16), and/or the export file is in STL file format. ### Claim 12 (depends on 9) A computer programme product comprising programme code which is stored on a non-transitory machine-readable medium, the non-transitory machine-readable medium comprising computer-executable instructions for performing the method according to claim 9. ### Claim 13 A method for exporting a three-dimensional dental design model based on a three-dimensional dental library model (4) from an augmented reality application (6), the augmented reality application (6) being adapted to visualize on a display (7) an image of the dental library model (4) rendered by a virtual camera with a preliminary pose and scale in a photo (2) of a face of a patient taken by a camera under a viewing axis to the face of the patient, the photo (2) including a mouth opening showing at least part of the present situation of the patient's dentition, the method comprising: placing within the augmented reality application (6) the rendered image of the dental library model (4) on the photo (2) and providing tools to modify and/or move the dental library model (4) on the display (7) to obtain a desired dental design model (8), providing a three-dimensional scan model (10) of the present situation of the patient's dentition, aligning the scan model (10) and the present situation of the patient's dentition in the photo (2) to obtain a pose of the scan model (10), calculating, based on the preliminary pose and scale and the pose of the scan model (10), at least one transformation to bring the desired dental design model (8) and the scan model (10) into a common coordinate system to obtain a positioned dental model (12), adapting the positioned dental design model (12) to the scan model (10) with non-rigid deformations by minimizing a measure of shape difference between the positioned dental design model (12) and the scan model (10) to obtain a modified dental design model (14) as the dental design model for export, and applying weighted non-rigid deformations, wherein by applying weighted non-rigid deformations, movements along the viewing axis are allowed for the positioned dental design model (12) by allowing to apply the same high weights to the movements along the viewing axis of the positioned dental design model (12).