# US 2021/0241885 A1 · Tooth virtual editing method and system, computer device, and storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0241885 A1 - Document ID: US20210241885A1 - Status: Published application - Kind: A1 - Application number: 17/054,068 - Filed: 2019-05-30 - Published: 2021-08-05 - Earliest claimed: 2018-10-10 - Assignees: Beijing Keeyoo Technologies Co., Ltd. - Applicants: Beijing Keeyoo Technologies Co., Ltd. - Inventors: Congxing OUYANG - Web page: https://patentgrep.com/patent/US20210241885A1 ## CPC - G16H 30/40: for processing medical images, e.g. editing - G06T 7/62: of area, perimeter, diameter or volume - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06V 20/20: in augmented reality scenes - G06V 20/64: Three-dimensional [3D] objects - G16H 40/67: for remote operation - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G16H 50/50: for simulation or modelling of medical disorders - G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30036: Dental; Teeth - G06T 2219/004: Annotating, labelling - G06T 2219/2016: Rotation, translation, scaling - G06T 2219/2021: Shape modification - G06V 2201/033: of skeletal patterns - G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A tooth virtual editing method comprises: constructing a three-dimensional image basic database of a human oral cavity; acquiring three-dimensional image data of a user oral cavity, performing image pattern recognition on the three-dimensional image data according to the stored three-dimensional image basic database, recognizing an object to which the image data belongs, and establishing an object annotation system of the user oral cavity, wherein the object contains a block and/or an entity; and performing virtual editing on the object according to the established object annotation system of the current user, and displaying an updated three-dimensional image of the teeth. ## Claims ### Claim 1 A tooth virtual editing method, comprising the steps of: Step S1, constructing a three-dimensional image basic database of a human oral cavity; Step S2, acquiring three-dimensional image data of a user oral cavity, performing image pattern recognition on the three-dimensional image data according to the stored three-dimensional image basic database, recognizing an object to which the image data belongs, and establishing an object annotation system of the user oral cavity; and Step S3, performing virtual editing on the object according to the established object annotation system of a current user, and displaying an updated three-dimensional image of user's teeth. ### Claim 2 (depends on 1) The tooth virtual editing method according to claim 1, wherein the object comprises: a block and/or an entity, wherein the block is formed by division according to curved surface images of different parts of the oral cavity, and the entity comprises: a tooth entity in a three-dimensional volume shape formed by enclosure of a plurality of blocks associated with a tooth part, and a dentition entity formed by combination of a plurality of tooth entities and gaps between the tooth entities. ### Claim 3 (depends on 2) The tooth virtual editing method according to claim 2, wherein the Step S2 further comprises: Step S21, performing image pattern recognition on the three-dimensional image data, recognizing a block to which the image data belongs, outputting a recognition result, and establishing a block annotation system of the user oral cavity; and Step S22, establishing an entity annotation system covering the user's teeth according to image information of each block in the block annotation system and a spatial relationship between the blocks. ### Claim 4 (depends on 3) The tooth virtual editing method according to claim 3, wherein the three-dimensional image basic database of the human oral cavity stores three-dimensional curved surface image data and three-dimensional entity image data covering all regions of a human oral cavity surface, and editing item list information for performing editing on the object. ### Claim 5 (depends on 4) The tooth virtual editing method according to claim 4, wherein said editing item list information comprises: a serial number of an editing object, an editing content and an appearance image after editing, wherein the editing content contains an editing action for the operation object and parameter settings for the editing action. ### Claim 6 (depends on 5) The tooth virtual editing method according to claim 5, wherein the editing action comprises: removing dental plaque on a block, removing dental calculus on a block, removing pigment on a block, repairing a curved surface shape of a block, rotating a tooth entity, moving a tooth entity, stripping a tooth entity, performing veneering or cosmetic crown treatment on a tooth entity, and repairing an inlay of a tooth entity; and the parameter settings include: specific quantized numerical parameters and value ranges of the editing action. ### Claim 7 (depends on 4) The tooth virtual editing method according to claim 4, wherein said acquiring three-dimensional image data of the teeth of the user oral cavity comprises: performing three-dimensional true color scanning on the user oral cavity to obtain a spatial arrangement of the user's teeth and a three-dimensional true color curved surface image of the user's teeth, wherein the spatial arrangement comprises relationships between the teeth and an occlusal relationship between a maxillary dentition and mandibular a dentition in an occlusal state. ### Claim 8 (depends on 7) The tooth virtual editing method according to claim 7, wherein the method further comprises: performing scanning on face and oral fissure regions near the lips when the user lips are closed or opened to obtain an oral cavity surface image visible from the outside in the user oral fissure region, performing image pattern recognition on the oral cavity image visible in the user oral fissure region according to the stored three-dimensional image basic database, recognizing a block to which the oral cavity surface visible in the oral fissure region belongs, outputting a recognition result, and establishing a block annotation system of the user face and oral fissure regions. ### Claim 9 (depends on 8) The tooth virtual editing method according to claim 8, wherein the three-dimensional image basic database pre-stores three-dimensional curved surface image data of human teeth in various scenarios, the three-dimensional curved surface image is pre-divided into a plurality of mutually connected blocks, and annotation information is established for each said block, the annotation information containing image information of the block, a spatial position relationship between adjacent blocks, and a spatial position relationship between a certain maxillary dentition block and a corresponding mandibular dentition block abutting the maxillary dentition block, as well as a spatial position relationship between a certain mandibular dentition block and a corresponding maxillary dentition block abutting the mandibular dentition block in the occlusal state; and the three-dimensional image basic database further pre-stores three-dimensional entity image data of human teeth in various scenarios, and the three-dimensional entity image data contains annotation information of a corresponding entity, the annotation information containing three-dimensional image data of the entity, a spatial position relationship between adjacent entities, and a spatial position relationship between a certain maxillary dentition entity and a corresponding mandibular dentition entity abutting the maxillary dentition entity, as well as a spatial position relationship between a certain mandibular dentition entity and a corresponding maxillary dentition entity abutting the mandibular dentition entity in the occlusal state. ### Claim 10 (depends on 9) The tooth virtual editing method according to claim 9, wherein pre-dividing the three-dimensional curved surface image into a plurality of mutually connected blocks comprises: firstly dividing the oral cavity surface into a series of mutually connected regions, and then dividing a curved surface image of a region into one or more mutually connected blocks. ### Claim 11 (depends on 4) The tooth virtual editing method according to claim 4, wherein according to the three-dimensional image basic database, the acquired image data is matched with image data of a corresponding object stored in the three-dimensional image basic database according to a preset image pattern recognition algorithm to obtain a mapping relationship therebetween, an offending object is determined in the three-dimensional image of the user's teeth according to the mapping relationship, and editing is performed on the offending object. ### Claim 12 (depends on 4) The tooth virtual editing method according to claim 4, wherein when the recognition for the acquired image data cannot be achieved based on the current basic database, the image data is uploaded to a cloud server for artificial recognition to input the annotation information or revise the editing item list information, and then the basic database is updated according to the recognition result, the annotation information and the editing item list information. ### Claim 13 (depends on 12) The tooth virtual editing method according to claim 12, wherein in a case of adding a new object and annotation information thereof into the basic database after artificial recognition, the editing list information of the object is added into the basic database by manual operation, and in a case of revising the annotation information of an existing object in the basic database without adding a new object after artificial recognition, whether to adaptively revise original editing item list information of the existing object in the basic database is determined by manual operation. ### Claim 14 A tooth virtual editing device, comprising: a three-dimensional image basic database established according to big data of human oral cavities; an image data acquisition device for acquiring and obtaining three-dimensional image data of a user oral cavity and sending the three-dimensional image data to an image pattern recognition device; the pattern recognition device for receiving the three-dimensional image data sent by the image data acquisition device, retrieving from the basic database of the system, and recognizing an object to which the three-dimensional image data belongs; a virtual editing device for, according to the recognized object, performing virtual editing on the object; and a display device for displaying appearance of the edited three-dimensional image of user's teeth. ### Claim 15 (depends on 14) The tooth virtual editing device of claim 14, wherein the pattern recognition device performs pattern recognition on the three-dimensional image data of the user oral cavity according to image information stored in the basic database and determines a serial number of an object to which the image data belongs; and the virtual editing device indexes the serial number of the object to a specific object so as to obtain the image information of the object and retrieve editing item list information of the object in the basic database. ### Claim 16 (depends on 14) The tooth virtual editing device of claim 14, wherein the system further comprises a cloud server connected with both the pattern recognition device and the three-dimensional image basic database, and when the recognition for the image data send by the image acquisition device cannot be achieved by the pattern recognition device, the image data is sent to the cloud server for artificial recognition to input the annotation information and revise the editing item list information, and then the basic database is updated according to the recognition result, the annotation information and the editing item list information; and the cloud server performs iterative training on the pattern recognition device based on the updated basic database. ### Claim 17 (depends on 1) A computer device comprising a memory and one or more processors, the memory having stored therein computer readable instructions which, when executed by the one or more processors, the one or more processors perform the method of claim 1. ### Claim 18 (depends on 1) A non-transistory computer-readable storage medium having stored therein computer-readable instructions that, when executed by one or more processors, the one or more processors perform the method of claim 1.