# US 9,775,491 B2 · Visualising a 3D dental restoration on a 2D image > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,775,491 B2 - Document ID: US9775491B2 - Status: Granted - Kind: B2 - Application number: 14/773,564 - Filed: 2014-03-07 - Published: 2016-01-21 - Granted: 2017-10-03 - Earliest claimed: 2013-03-08 - Assignees: 3Shape A/S - Applicants: 3Shape A/S - Inventors: Tais Clausen; Rune Fisker - Web page: https://patentgrep.com/patent/US9775491B2 ## CPC - G06T 7/33: using feature-based methods - A61B 1/000095: {for image enhancement} - A61B 1/0005: {combining images e.g. side-by-side, superimposed or tiled} - A61B 1/24: for the mouth, i.e. stomatoscopes, e.g. with tongue depressors; Instruments for opening or keeping open the mouth - A61C 13/0004: {Computer-assisted sizing or machining of dental prostheses} - A61C 13/34: Making or working of models, e.g. preliminary castings, trial dentures; Dowel pins [4] - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 2207/30036: Dental; Teeth - G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - A61C: DENTISTRY; APPARATUS OR METHODS FOR ORAL OR DENTAL HYGIENE (non-driven toothbrushes A46B; {tongue scrapers A61B17/24;} preparations for dentistry A61K6/00; preparations for cleaning the teeth or mouth A61K8/00, A61Q11/00) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract A method for visualizing a 3D dental restoration on a 2D image of the mouth of a patient, the method including obtaining a 3D dental model of at least a part of the patient's oral cavity, designing the 3D dental restoration, obtaining the 2D image of the mouth of the patient, estimating a virtual camera comprising at least one virtual camera property corresponding to at least one physical camera property of the physical camera used to obtain the 2D image, viewing the 3D dental restoration using the virtual camera, determining the visible area of the 3D dental restoration, which is not overlapped by surrounding anatomic features when viewed with the virtual camera, imaging the visible area of the 3D dental restoration with the 2D image. This provides an image which with high accuracy gives the dentist and the patient a visual presentation of the final result of a dental treatment. ## Claims ### Claim 1 A method for visualising a 3D dental restoration on a 2D image of a mouth of a patient, wherein the method comprises: obtaining a 3D dental model of at least a part of the patient's oral cavity, designing the 3D dental restoration on the 3D dental model, obtaining the 2D image of the mouth of the patient, estimating a virtual camera comprising at least one virtual camera property corresponding to at least one physical camera property of a physical camera used to obtain the 2D image, viewing the 3D dental restoration using the virtual camera, selecting only a portion of the 3D dental model that corresponds to the 3D dental restoration which is visible to the virtual camera, and imaging only the selected portion of the 3D dental model with the 2D image, wherein the 2D image is not visible at the selected portion of the 3D dental model. ### Claim 2 (depends on 1) A method according to claim 1, wherein the method further comprises: determining the visible area of the 3D dental restoration with the 3D dental restoration placed on the 3D dental model. ### Claim 3 (depends on 2) A method according to claim 2, wherein the virtual camera is estimated by marking at least a first, second, third and fourth reference point on the 2D image and corresponding at least first, second, third and fourth reference point on the 3D dental model, and aligning the at least first, second, third and fourth reference point with the respective corresponding at least first, second, third and fourth reference point. ### Claim 4 (depends on 1) A method according to claim 1, wherein the selected portion of the 3D dental model is determined by identifying overlapping anatomic features in the 2D image of the mouth and subtracting the overlapping anatomic features from the 3D dental model. ### Claim 5 (depends on 4) A method according to claim 4, wherein the overlapping anatomic features include a gingiva, a posterior tooth and/or a lip. ### Claim 6 (depends on 1) A method according to claim 1, wherein the at least one physical camera property is a field of view. ### Claim 7 (depends on 1) A method according to claim 1, wherein the selected portion of the 3D dental model is imaged with the 2D image by projecting the selected portion of the 3D dental model onto the 2D image. ### Claim 8 (depends on 1) A method according to claim 1, wherein the selected portion of the 3D dental model is imaged with the 2D image by generating a mask aligned with the 2D image, said mask having a first value in the areas of the 2D image overlapping the 3D dental restorations and a second value in the areas where no overlap occur. ### Claim 9 (depends on 8) A method according to claim 8, wherein the 2D image is transparent in the areas where the mask have the first value. ### Claim 10 (depends on 1) A method according to claim 1, wherein the method further comprises adjusting a transparency of the 2D image. ### Claim 11 (depends on 1) A method according to claim 1, wherein the method further comprises defining an edge of the lip in the 2D image. ### Claim 12 (depends on 1) A method according to claim 1, wherein the selected portion of the 3D dental model is a portion of the 3D dental model which is not overlapped by surrounding portions of the 3D dental model that were a part of the 3D dental model prior to designing the 3D dental restoration on the 3D dental model. ### Claim 13 A system for visualising a 3D dental restoration on a 2D image of a mouth of a patient, wherein the system enables: loading a 2D image of the mouth of the patient, providing a 3D dental model of at least a part of the patient's oral cavity, providing a 3D dental restoration on the 3D dental model, estimating a virtual camera comprising at least one virtual camera property corresponding to at least one physical camera property of a physical camera used to obtain the 2D image, and selecting only the portion of the 3D dental model that corresponds to the 3D dental restoration which is visible to the virtual camera, and the system includes a view area for imaging only the selected portion of the 3D dental model with the 2D image. ### Claim 14 (depends on 13) A system according to claim 13, wherein the system further enables marking at least a first, second, third and fourth reference point on the 2D image and corresponding at least first, second, third and fourth reference point on the 3D dental model. ### Claim 15 (depends on 13) A system according to claim 13, wherein the system further enables adjusting a transparency of the 2D image. ### Claim 16 (depends on 13) A system according to claim 13, wherein the system further enables defining an edge of the lip in the 2D image. ### Claim 17 (depends on 13) A system according to claim 13, wherein the selected portion of the 3D dental model is a portion of the 3D dental restoration which is not overlapped by surrounding portions of the 3D dental model that were a part of the 3D dental model prior to designing the 3D dental restoration on the 3D dental model.