# US 10,499,993 B2 · Systems and methods for non-rigid deformation of tissue for virtual navigation of interventional tools > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,499,993 B2 - Document ID: US10499993B2 - Status: Granted - Kind: B2 - Application number: 16/405,438 - Filed: 2019-05-07 - Published: 2019-08-22 - Granted: 2019-12-10 - Earliest claimed: 2014-02-04 - Assignees: Intuitive Surgical Operations, Inc. - Applicants: Intuitive Surgical Operations, Inc. - Inventors: Prashant Chopra; Vincent Duindam; Lei Xu; Tao Zhao - Web page: https://patentgrep.com/patent/US10499993B2 ## CPC - A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61B 34/30: Surgical robots - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones} - A61B 2034/107: {Visualisation of planned trajectories or target regions} - A61B 2034/2051: {Electromagnetic tracking systems} - A61B 2034/2061: {using shape-sensors, e.g. fiber shape sensors with Bragg gratings} - A61B 2034/301: {for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes} - G06T 2210/41: Medical - G06T 2219/2021: Shape modification - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A processing system comprises a processor and a memory having computer readable instructions stored thereon. The computer readable instructions, when executed by the processor, cause the system to receive a reference three-dimensional volumetric representation of a branched anatomical formation in a reference state and obtain a reference tree of nodes and linkages based on the reference three-dimensional volumetric representation. The computer readable instructions also cause the system to obtain a reference three-dimensional geometric model based on the reference tree and detect deformation of the branched anatomical formation due to anatomical motion based on measurements from a shape sensor. The computer readable instructions also cause the system to obtain a deformed tree of nodes and linkages based on the detected deformation, create a three-dimensional deformation field that represent the detected deformation of branched anatomical, and apply the three-dimensional deformation field to the reference three dimensional geometric model. ## Claims ### Claim 1 A processing system comprising: a processor; and a memory having computer readable instructions stored thereon, the computer readable instructions, when executed by the processor, cause the system to: receive a reference three dimensional volumetric representation of a branched anatomical formation in a reference state; obtain a reference tree of nodes and linkages based on the reference three dimensional volumetric representation; obtain a reference three dimensional geometric model based on the reference tree; detect deformation of the branched anatomical formation due to anatomical motion based on measurements from a shape sensor; obtain a deformed tree of nodes and linkages based on the detected deformation; create a three dimensional deformation field that represents the detected deformation of the branched anatomical formation; and apply the three dimensional deformation field to the reference three dimensional geometric model to create a deformed three dimensional geometric model of the branched anatomical formation in a deformed state. ### Claim 2 (depends on 1) The processing system of claim 1, wherein the computer readable instructions, when executed by the processor, cause the system to perform operations including: dynamically display an image of the deformed three dimensional geometric model of the branched anatomical formation in the deformed state. ### Claim 3 (depends on 2) The processing system of claim 2, wherein displaying the image of the deformed three dimensional geometric model of the branched anatomical formation in the deformed state includes displaying a sequence of images of intermediate deformations of the branched anatomical formation. ### Claim 4 (depends on 2) The processing system of claim 2, wherein the computer readable instructions, when executed by the processor, cause the system to perform operations including: display a composite image including an image of an interventional instrument positioned within the branched anatomical formation and the dynamically displayed image of the deformed three dimensional geometric model of the branched anatomical formation in the deformed state. ### Claim 5 (depends on 4) The processing system of claim 4, wherein the computer readable instructions, when executed by the processor, cause the system to perform operations including: display the composite image in real time. ### Claim 6 (depends on 1) The processing system of claim 1 wherein the reference three dimensional geometric model is a reference mesh model having vertices and the deformed three dimensional geometric model is a deformed mesh model having vertices, and wherein applying the three dimensional deformation field to the reference three dimensional geometric model includes interpolating the three dimensional deformation field for at least one of the vertices on the reference mesh model over time. ### Claim 7 (depends on 6) The processing system of claim 6 wherein the computer readable instructions, when executed by the processor, cause the system to perform operations including: applying weights to the vertices of the reference mesh model. ### Claim 8 (depends on 6) The processing system of claim 6 wherein applying the three dimensional deformation field to the reference three dimensional geometric model further includes transforming at least one of the vertices from the reference mesh model to a corresponding vertex on the deformed mesh model of the branched anatomical formation in the deformed state. ### Claim 9 (depends on 8) The processing system of claim 8, wherein transforming the at least one of the vertices on the reference mesh model to the corresponding vertex on the deformed three dimensional model further includes assigning a weight to the at least one of the vertices on the reference mesh model for each linkage in a plurality of linkages that influences the at least one of the vertices on the reference mesh model. ### Claim 10 (depends on 1) The processing system of claim 1, including a graphical processing unit to apply the three dimensional deformation field to the reference three dimensional geometric model. ### Claim 11 (depends on 1) The processing system of claim 1, wherein the detected deformation is indicative of at least a periodic anatomical motion. ### Claim 12 (depends on 1) The processing system of claim 1 wherein the detected deformation is indicative of a force applied by an interventional instrument positioned within the branched anatomical formation. ### Claim 13 (depends on 1) The processing system of claim 1 wherein the detected deformation is indicative of a periodic anatomical motion and a force applied by an interventional instrument positioned within the branched anatomical formation. ### Claim 14 A method comprising: receiving a reference three dimensional volumetric representation of a branched anatomical formation in a reference state; creating a reference tree of nodes and linkages based on the reference three dimensional volumetric representation; creating a reference three dimensional geometric model based on the reference tree; detecting deformation of the branched anatomical formation due to anatomical motion based on measurements from a shape sensor; creating a deformed tree of nodes and linkages based on the detected deformation; creating a three dimensional deformation field that represents the detected deformation of the branched anatomical formation from the reference state to a deformed state; and applying the three dimensional deformation field to the reference three dimensional geometric model to create a deformed three dimensional geometric model of the branched anatomical formation in the deformed state. ### Claim 15 (depends on 14) The method of claim 14, including: dynamically displaying an image of the deformed three dimensional geometric model of the branched anatomical formation in the deformed state. ### Claim 16 (depends on 15) The method of claim 15, wherein displaying the image of the deformed three dimensional geometric model of the branched anatomical formation in the deformed state includes displaying a sequence of images of intermediate deformations of the branched anatomical formation. ### Claim 17 (depends on 16) The method of claim 16, including: displaying a composite image including an image of an interventional instrument positioned within the branched anatomical formation and the dynamically displayed image of the deformed three dimensional geometric model of the branched anatomical formation in the deformed state. ### Claim 18 (depends on 14) The method of claim 14, wherein the reference three dimensional geometric model is a reference mesh model having vertices and the deformed three dimensional geometric model is a deformed mesh model having vertices, and wherein applying the three dimensional deformation field to the reference three dimensional geometric model includes interpolating the three dimensional deformation field for at least one of the vertices on the reference mesh model over time. ### Claim 19 (depends on 18) The method of claim 18 wherein applying the three dimensional deformation field to the reference three dimensional geometric model further includes transforming at least one of the vertices from the reference mesh model to a corresponding vertex on the deformed mesh model of the branched anatomical formation in the deformed state. ### Claim 20 (depends on 18) The method of claim 18, including: generating an image of the reference three dimensional volumetric representation in the deformed state by deforming the reference three dimensional volumetric representation into alignment with the deformed mesh model.