# US 2021/0366135 A1 · Method and system for registering images containing anatomical structures > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0366135 A1 - Document ID: US20210366135A1 - Status: Published application - Kind: A1 - Application number: 17/324,633 - Filed: 2021-05-19 - Published: 2021-11-25 - Earliest claimed: 2020-05-20 - Assignees: Therenva - Applicants: Therenva - Inventors: Florent Lalys; Mathieu Colleaux; Vincent Gratsac - Web page: https://patentgrep.com/patent/US20210366135A1 ## CPC - G06T 7/33: using feature-based methods - G06F 18/2431: Multiple classes - G06T 2200/04: involving 3D image data - G06T 2207/10064: Fluorescence image - G06T 2207/10081: Computed x-ray tomography [CT] - G06T 2207/10088: Magnetic resonance imaging [MRI] - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/20221: Image fusion; Image merging - G06T 2207/30004: Biomedical image processing - G06T 2207/30101: Blood vessel; Artery; Vein; Vascular - G06V 2201/03: Recognition of patterns in medical or anatomical images - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A method for registration between a first three-dimensional image acquired according to a first acquisition mode, and including anatomical structures of a patient, and a second two-dimensional image, acquired according to a second acquisition mode by a rotatable and translatable device, the second image including a portion of the anatomical structures of the patient. The registration implements a rigid spatial transformation defined by rotation and translation parameters. The method includes automatically detecting the anatomical structures in the two-dimensional image by applying a first detection neural network trained on a generic database; estimating, from the anatomical structures automatically detected in the second two-dimensional image, by applying at least one classification neural network trained beforehand on a generic database, the rotation and translation parameters of the rigid spatial transformation; and 3D/2D iconic registration between the first three-dimensional image and the second two-dimensional image starting from an initialization with the rigid spatial transformation. ## Claims ### Claim 1 A method for registration between a first three-dimensional image acquired according to a first acquisition mode, and comprising anatomical structures of a patient, and a second two-dimensional image, acquired according to a second acquisition mode by an image acquisition device mounted on a scoping arch movable in rotation and in translation, the second image comprising a portion of the anatomical structures of said patient, the registration implementing a rigid spatial transformation defined by rotation and translation parameters, the method being implemented by a processor of a programmable electronic device and comprising: automatic detection of the anatomical structures in the two-dimensional image by application of a first detection neural network trained on a generic database, estimation, from the anatomical structures automatically detected in said second two-dimensional image, by application of at least one classification neural network trained beforehand on a generic database, of the rotation and translation parameters of said rigid spatial transformation, and 3D/2D iconic registration between the first three-dimensional image and the second two-dimensional image starting from an initialization with said rigid spatial transformation. ### Claim 2 (depends on 1) The method according to claim 1, wherein the anatomical structures are classified into a plurality of predetermined anatomical structure categories, and wherein the estimation of the parameters of the rigid transformation includes the implementation of a second classification neural network per anatomical structure category to obtain transformation parameters estimated per anatomical structure category. ### Claim 3 (depends on 1) The method according to claim 1, wherein the estimation includes: a first estimation, from anatomical structures automatically detected in said second two-dimensional image, by application of at least one classification neural network, of a first angle of rotation and of a second angle of rotation of said rigid spatial transformation, characterizing the position of the scoping arch, a second estimation of translational parameters, of a third angle of rotation and of a zoom factor of said rigid spatial transformation, the second estimation using the result of the first estimation. ### Claim 4 (depends on 3) The method according to claim 3, wherein a classification neural network is applied per anatomical structure category to obtain a first angle of rotation and a second angle of rotation estimated per anatomical structure category, the method further including a stitching of the first angles of rotation estimated per anatomical structure category to obtain said estimated first angle of rotation, and a stitching of the second angles of rotation estimated per anatomical structure category to obtain said estimated second parameter. ### Claim 5 (depends on 4) The method according to claim 4, wherein each stitch consists of an average value calculation or of a median value calculation. ### Claim 6 (depends on 3) The method according to claim 3, wherein the second estimation comprises: a generation of a third two-dimensional image obtained from the first three-dimensional image by applying said first and second angles of rotation, an automatic detection of anatomical structures on said third image, and a pairing between the structures of said second and third two-dimensional images to obtain said translation parameters, the third angle of rotation and the zoom factor of said rigid spatial transformation. ### Claim 7 (depends on 6) The method according to claim 6, wherein the detection of anatomical structures in said third two-dimensional image is performed by application of said first detection neural network. ### Claim 8 (depends on 1) The method according to claim 1, wherein said anatomical structures are bone structures obtained on a second two-dimensional image acquired by fluoroscopy. ### Claim 9 (depends on 1) The method according to claim 1, wherein said anatomical structures are vascular structures obtained on a second two-dimensional image acquired by angiography. ### Claim 10 (depends on 1) The method according to claim 1, wherein the first three-dimensional image is acquired according to a first acquisition mode among tomography, MRI, 3D sonography or cone beam computed tomography. ### Claim 11 A device for registration between a first three-dimensional image acquired according to a first acquisition mode, and comprising anatomical structures of a patient, and a second two-dimensional image, acquired according to a second acquisition mode by an image acquisition device mounted on a scoping arch movable in rotation and in translation, the second image comprising a portion of the anatomical structures of said patient, the registration implementing a rigid spatial transformation defined by rotation and translation parameters, the device including a processor configured to implement: automatic detection of the anatomical structures in the two-dimensional image by application of a first detection neural network trained on a generic database, estimation, from the anatomical structures automatically detected in said second two-dimensional image, by application of at least one classification neural network trained beforehand on a generic database, of the rotation and translation parameters of said rigid spatial transformation, and 3D/2D iconic registration between the first three-dimensional image and the second two-dimensional image starting from an initialization with said rigid spatial transformation. ### Claim 12 (depends on 11) The registration device according to claim 11, wherein the estimation includes: a first estimation, from anatomical structures automatically detected in said second two-dimensional image, by application of at least one neural network trained beforehand on a generic database, of a first angle of rotation and of a second angle of rotation of said rigid spatial transformation, characterizing the position of the scoping arch, a second estimation of translation parameters, of a third angle of rotation and of a zoom factor of said rigid spatial transformation, the second estimation using the result of the first estimation. ### Claim 13 A system for registration between a first three-dimensional image acquired according to a first acquisition mode, and comprising anatomical structures of a patient, and a second two-dimensional image, acquired according to a second acquisition mode, the system comprising: an image acquisition device mounted on a scoping arch movable in rotation and in translation, configured to acquire the second two-dimensional image, the second two-dimensional image comprising a portion of the anatomical structures of said patient, and a device for registration between the first three-dimensional image and the second two-dimensional image, the registration implementing a rigid spatial transformation defined by rotation and translation parameters, the device including a processor configured to implement: automatic detection of the anatomical structures in the two-dimensional image by application of a first detection neural network trained on a generic database, estimation, from the anatomical structures automatically detected in said second two-dimensional image, by application of at least one classification neural network trained beforehand on a generic database, of the rotation and translation parameters of said rigid spatial transformation, and 3D/2D iconic registration between the first three-dimensional image and the second two-dimensional image starting from an initialization with said rigid spatial transformation.