# US 10,765,371 B2 · Method to project a two dimensional image/photo onto a 3D reconstruction, such as an epicardial view of heart > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,765,371 B2 - Document ID: US10765371B2 - Status: Granted - Kind: B2 - Application number: 15/476,097 - Filed: 2017-03-31 - Published: 2018-10-04 - Granted: 2020-09-08 - Earliest claimed: 2017-03-31 - Assignees: Biosense Webster (Israel) Ltd. - Applicants: Biosense Webster (Israel) Ltd.; Takehiro Kimura - Inventors: Gal Hayam; Natan Sharon Katz; Takehiro Kimura - Web page: https://patentgrep.com/patent/US10765371B2 ## CPC - A61B 5/6852: {Catheters} - A61B 5/0084: {for introduction into the body, e.g. by catheters (A61B5/1459 takes precedence)} - A61B 5/065: {Determining position of the probe employing exclusively positioning means located on or in the probe, e.g. using position sensors arranged on the probe} - A61B 5/28: for electrocardiography [ECG] - A61B 5/363: Detecting tachycardia or bradycardia - A61B 18/1482: {having a long rigid shaft for accessing the inner body transcutaneously in minimal invasive surgery, e.g. laparoscopy} - G16H 30/40: for processing medical images, e.g. editing - G16H 50/50: for simulation or modelling of medical disorders - A61B 2018/00351: {Heart} - A61B 2018/00577: {Ablation} - A61B 2018/00642: {with feedback, i.e. closed loop control} - A61B 2576/023: for the heart - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract A three-dimensional (3D) electrical mapping system and method may be used to generate a 3D image of the epicardial surface of a heart by integrating one or more epicardial images with a 3D image of the cardiac structure that may be generated by real-time 3D location and mapping system for cardiac mapping and ablation. The visual textural representation of the epicardial surface of the heart may be reconstructed using, for example, an image sensor or camera-based catheter to collect images of the epicardial surface. For each image that is captured, the system and method may store the image data along with the corresponding catheter location, orientation and/or distance information relative to the cardiac structure. The location, orientation, and/or distance information may be used to reconstruct a 3D textural model of the epicardial surface of the cardiac structure. ## Claims ### Claim 1 A system configured to generate a three-dimensional (3D) texture map of an epicardial surface of a cardiac structure, the system comprising: a processing device configured to acquire anatomical data of the cardiac structure and generate a 3D model of the cardiac structure; a catheter configured to be inserted into an epicardium of the cardiac structure, the catheter comprising: an image sensor configured to capture a plurality of images of the epicardial surface of the cardiac structure and provide the plurality of images to the processing device, wherein the plurality of images include textural details of the epicardial surface; at least one location sensor configured to generate, for each of the plurality images, corresponding location, orientation and distance information and provide the corresponding location, orientation and distance information to the processing device; the processing device further configured to generate the 3D texture map of the epicardial surface of the cardiac structure by selecting a subset of images from the plurality of images of the epicardial surface of the cardiac structure based on the corresponding location, orientation and distance information to minimize redundant information in the subset of images, and stitching the subset of images to the 3D model of the cardiac structure using the corresponding location, orientation and distance information; the processing device configured to provide the 3D texture map of the epicardial surface of the cardiac structure to a visual display device; and the visual display device configured to visually display the 3D texture map of the epicardial surface of the cardiac structure. ### Claim 2 (depends on 1) The system of claim 1, wherein the processing device is further configured to store the plurality of images and the corresponding location, orientation and distance information in a local storage. ### Claim 3 (depends on 1) The system of claim 1, wherein the 3D texture map of the epicardial surface of the cardiac structure shows at least one of: a coronary artery, small vessels, adipose tissue, or a scar lesion. ### Claim 4 (depends on 1) The system of claim 1, wherein the corresponding location, orientation and distance information is relative to an external reference. ### Claim 5 (depends on 1) The system of claim 1, wherein: the processing device is configured to generate the 3D model of the cardiac structure using a multi-view 3D reconstruction algorithm. ### Claim 6 (depends on 1) The system of claim 1, wherein the image sensor is a charge-coupled device (CCD) image sensor. ### Claim 7 (depends on 1) The system of claim 1, wherein the catheter includes one or more electrodes for performing ablation on the epicardium of the cardiac structure7 based on the 3D texture map of the epicardial surface of the cardiac structure. ### Claim 8 (depends on 7) The system of claim 7, wherein the one or more electrodes are configured to perform the ablation to treat ventricular tachycardia (VT). ### Claim 9 (depends on 1) The system of claim 1 further comprising: extra-cardiac sensors configured to detect electrical activity of the cardiac structure and provide information on the electrical activity of the cardiac structure to the processing device. ### Claim 10 A method for generating a three-dimensional (3D) texture map of an epicardial surface of a cardiac structure, the method comprising: acquiring anatomical data of the cardiac structure and generating a 3D model of the cardiac structure; capturing a plurality of images of the epicardial surface of the cardiac structure, wherein the plurality of images include textural details of the epicardial surface; generating, for each of the plurality images, corresponding location, orientation and distance information; generating the 3D texture map of the epicardial surface of the cardiac structure by selecting a subset of images from the plurality of images of the epicardial surface of the cardiac structure based on the corresponding location, orientation and distance information to minimize redundant information in the subset of images, and stitching the subset of images to the 3D model of the cardiac structure using the corresponding location, orientation and distance information; and visually displaying the 3D texture map of the epicardial surface of the cardiac structure. ### Claim 11 (depends on 10) The method of claim 10, further comprising: storing the plurality of images and the corresponding location, orientation and distance information. ### Claim 12 (depends on 10) The method of claim 10, wherein the 3D texture map of the epicardial surface of the cardiac structure shows at least one of: a coronary artery, small vessels, adipose tissue, or a scar lesion. ### Claim 13 (depends on 10) The method of claim 10, wherein the corresponding location, orientation and distance information is relative to an external reference. ### Claim 14 (depends on 10) The method of claim 10, wherein: the generating the 3D model of the cardiac structure uses a multi-view 3D reconstruction algorithm. ### Claim 15 (depends on 10) The method of claim 10, wherein the plurality of images of the epicardial surface of the cardiac structure are captured using a charge-coupled device (CCD) image sensor mounted to a catheter inserted into an epicardium of the cardiac structure. ### Claim 16 (depends on 10) The method of claim 10, further comprising performing ablation on an epicardium of the cardiac structure based on the 3D texture map of the epicardial surface of the cardiac structure. ### Claim 17 (depends on 16) The method of claim 16, wherein the ablation is used to treat ventricular tachycardia (VT). ### Claim 18 (depends on 16) The method of claim 16, further comprising: detecting electrical activity of the cardiac structure; and providing information on the electrical activity of the cardiac structure for performing the ablation.