# US 12,217,427 B2 · Method of visualizing a dynamic anatomical structure > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,217,427 B2 - Document ID: US12217427B2 - Status: Granted - Kind: B2 - Application number: 17/423,157 - Filed: 2020-01-10 - Published: 2022-04-28 - Granted: 2025-02-04 - Earliest claimed: 2019-01-17 - Assignees: Koninklijke Philips N.V. - Applicants: Koninklijke Philips N.V. - Inventors: Marcus Schreckenberg; Niklas Hitschrich - Web page: https://patentgrep.com/patent/US12217427B2 ## CPC - G06T 7/0016: {involving temporal comparison} - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 7/11: Region-based segmentation - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 15/08: Volume rendering - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 10/62: relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking - G06T 2207/10016: Video; Image sequence - G06T 2207/10076: 4D tomography; Time-sequential 3D tomography - G06T 2207/30048: Heart; Cardiac - G06T 2210/41: Medical - G06V 2201/031: of internal organs - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract The invention relates to a method of visualising a dynamic anatomical structure (1), a computer program and a user interface. The method comprises (a) providing a sequence of three-dimensional medical images (M1, M2, M3, . . . MZ) of a dynamic anatomical structure (1) spanning a time period (T), (b) providing a dynamic model (14), in particular surface of the anatomical structure, (c) determining a volume of interest (40) containing an anatomical feature of interest (3) within each of the three-dimensional images, wherein the volume of interest (40) follows the position and/or the shape of the anatomical feature of interest (3) across the time period and wherein the volume of interest (40) is smaller than the complete field of view of the three-dimensional medical images (M1, M2, M3, . . . MZ), and (d) providing a three-dimensional visualisation environment (50, 70), wherein a visualisation (45) corresponding to a particular point in time comprises (i) a volume rendering of the volume of interest (40) of the three-dimensional image; and (ii) a visualisation of the dynamic model (14) in the same coordinate system. Preferably, the three-dimensional visualisation environment (50, 70) allows for displaying the dynamic model (14) and the volume rendered volume of interest (40) for each three-dimensional image across the time period in cine mode. ## Claims ### Claim 1 A method of visualising a dynamic anatomical structure, the method comprising: a) providing a sequence of three-dimensional medical images spanning a time period, each three-dimensional medical image of the sequence showing a dynamic anatomical structure at a point in time during the time period; b) providing a dynamic model of at least a part of the dynamic anatomical structure, wherein the dynamic model has been derived from and is registered with the sequence of three-dimensional medical images; c) determining a volume of interest containing an anatomical feature of interest within each of the three-dimensional medical images, wherein the volume of interest follows a position and/or a shape of the anatomical feature of interest across the time period, and wherein the volume of interest is smaller than a complete field of view of the three-dimensional medical images; and d) providing a three-dimensional visualisation environment for displaying the dynamic anatomical structure across the time period, wherein a visualisation corresponding to a particular point in time within the time period comprises (i) a volume rendering of the volume of interest of the three-dimensional medical image corresponding to the particular point in time; and (ii) a visualisation of the dynamic model at the particular point in time and in the same coordinate system as the volume rendering of the volume of interest. ### Claim 2 (depends on 1) The method of claim 1, wherein the three-dimensional visualisation environment is a virtual reality environment. ### Claim 3 (depends on 2) The method of claim 2, wherein the virtual reality environment comprises at least one input tool, wherein the input tool is realised by a virtual reality controller and allows a user to grab and move an object within the virtual reality environment using hand gestures. ### Claim 4 (depends on 2) The method of claim 2, wherein the virtual reality environment comprises at least one input tool, wherein the input tool is realised by a virtual reality controller and allows a user to adjust parameters used in the visualisation of the dynamic model via gesture control. ### Claim 5 (depends on 4) The method of claim 4, wherein the gesture control is to adjust settings for a threshold used in performing volume rendering on the volume of interest. ### Claim 6 (depends on 1) The method of claim 1, wherein the dynamic anatomical structure is a human heart or animal heart, and the anatomical feature of interest is a part of the human heart or animal heart. ### Claim 7 (depends on 6) The method of claim 6, wherein the dynamic model is a dynamic surface model of one heart chamber or several heart chambers, and the anatomical feature is a heart valve. ### Claim 8 (depends on 1) The method of claim 1, wherein the dynamic model is a dynamic surface model and is derived from the sequence of three-dimensional medical images by segmentation over all three-dimensional medical images, or by segmentation in one three-dimensional medical image and speckle or feature tracking in subsequent three-dimensional medical images. ### Claim 9 (depends on 1) The method of claim 1, wherein a position and/or orientation of the volume of interest is determined across the time period by identifying a corresponding position and/or orientation of the anatomical feature of interest in the dynamic model. ### Claim 10 (depends on 1) The method of claim 1, wherein the volume of interest is determined by identifying a set of landmark points of the anatomical feature in the dynamic model or in the three-dimensional medical images, wherein the landmark points follow the position and/or the shape of the anatomical feature of interest across the time period, and by defining a approximating surface spanning the set of landmark points for each three-dimensional medical image, and by determining the volume of interest as a volume extending above and/or below and/or to a side of the approximation surface. ### Claim 11 (depends on 1) The method of claim 1, wherein providing a three-dimensional visualisation environment comprises: displaying a computer graphical object corresponding to a medical device in the same coordinate system as the volume rendering of the volume of interest, and wherein the computer graphical object is locked to a position in the dynamic model when the dynamic model and the volume rendered volume of interest are displayed in cine-mode, and providing an input tool to a user, the input tool allowing the user to move and tilt the computer graphical object corresponding to the medical device in relation to the volume rendering and the visualisation of the dynamic model. ### Claim 12 (depends on 11) The method of claim 11, wherein the medical device comprises an implant. ### Claim 13 (depends on 1) The method of claim 1, wherein providing a three-dimensional visualisation environment comprises: providing an input tool to a user, the input tool allowing the user to select points within the dynamic anatomical structure and to take measurements on the dynamic anatomical structure. ### Claim 14 (depends on 1) A tangible, non-transitory computer readable medium that stores a computer program comprising program code instructions which, when executed by a processor, enables the processor to perform the method according to claim 1. ### Claim 15 A user interface configured for visualising a dynamic anatomical structure, the dynamic anatomical structure having been captured on a sequence of three-dimensional medical images spanning a time period, each three-dimensional medical image of the sequence showing the dynamic anatomical structure at a point in time during the time period, the user interface comprising: a) a three-dimensional visualisation environment configured to provide a three-dimensional visualisation of the dynamic anatomical structure across a time period, and a processor configured for generating a visualisation corresponding to a particular point in time within the time period, the visualisation comprising: (i) a display of a volume rendering of a volume of interest within the three-dimensional medical image corresponding to the particular point in time, the volume of interest containing an anatomical feature of interest; and (ii) a display of a dynamic model of at least a part of the dynamic anatomical structure at the particular point in time and in the same coordinate system as the volume rendering of the volume of interest, wherein the volume of interest follows a position and/or a shape of the anatomical feature of interest across the time period, and wherein the volume of interest is smaller than a complete field of view of the three-dimensional medical images. ### Claim 16 (depends on 15) The user interface of claim 15, wherein the three-dimensional visualisation environment is a virtual reality environment and the display of the volume rendering of the volume of interest and the dynamic model is a virtual reality display. ### Claim 17 (depends on 16) The user interface of claim 16, wherein the virtual reality environment comprises at least one input tool, wherein the input tool is a virtual reality controller allowing a user to perform one or more of the following actions: to grab and move an object displayed in the virtual reality environment; to take measurements on the dynamic anatomical structure; to adjust parameters used in the visualisation of the dynamic model via gesture control; and to make annotations to the dynamic anatomical structure, wherein the annotations are locked to a position in the dynamic model when the dynamic model and the volume rendered volume of interest are displayed in cine-mode. ### Claim 18 (depends on 16) The user interface of claim 16, wherein the virtual reality display is provided on a virtual reality headset. ### Claim 19 (depends on 15) The user interface of claim 15, wherein the dynamic model is a dynamic surface model of one heart chamber or several heart chambers, and the anatomical feature is a heart valve. ### Claim 20 (depends on 15) The user interface of claim 15, wherein the dynamic model is a dynamic surface model and is derived from the sequence of three-dimensional medical images by segmentation over all three-dimensional medical images, or by segmentation in one three-dimensional medical image and speckle or feature tracking in subsequent three-dimensional medical images.