# US 2021/0330387 A1 · Visualization of three-dimensional image data on two- dimensional images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0330387 A1 - Document ID: US20210330387A1 - Status: Published application - Kind: A1 - Application number: 17/237,668 - Filed: 2021-04-22 - Published: 2021-10-28 - Earliest claimed: 2020-04-22 - Assignees: Canon U.S.A., Inc. - Applicants: Canon U.S.A., Inc. - Inventors: Hitoshi Nakamura; Antonio Bonillas Vaca - Web page: https://patentgrep.com/patent/US20210330387A1 ## CPC - A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations - A61B 34/20: Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis - A61B 90/37: {Surgical systems with images on a monitor during operation} - A61B 2034/107: {Visualisation of planned trajectories or target regions} - A61B 2034/2065: {Tracking using image or pattern recognition} - A61B 2034/2068: {using pointers, e.g. pointers having reference marks for determining coordinates of body points} - A61B 2090/363: {Use of fiducial points} - A61B 2090/367: {creating a 3D dataset from 2D images using position information} - A61B 2090/372: {Details of monitor hardware} - A61B 2090/3762: {using computed tomography systems [CT]} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) ## Abstract The present disclosure relates generally to medical imaging and, more particularly to systems, methods, and devices for planning and carrying out minimally invasive procedures using external devices for needle guidance and the display and manipulation of the image set when planning and performing the procedure. ## Claims ### Claim 1 An image processing apparatus comprising a processor that operates to: acquire a three-dimensional image set comprising a stack of image slices; cause a display to display a first image slice from the three-dimensional image set in a first image mode; cause the display to display, concurrent with the first image slice, a first indicator element associated with the first image mode; cause a display to display a second image slice from the three-dimensional image set in a second image mode; cause the display to display, concurrent with the first image slice, a second indicator element associated with the second image mode; wherein the first and second indicator elements each indicate: the location of the first and second image slice within the three-dimensional image set, respectively, and the location of the at least one region of interest and/or at least one target point. ### Claim 2 (depends on 1) The image processing apparatus of claim 1, wherein at least one of the first and second image modes is an imaging mode that is out-of-plane compared to the three-dimensional image set. ### Claim 3 (depends on 1) The image processing apparatus of claim 1, wherein the first and second image mode are selected from: an axial image mode, a sagittal image mode, a coronal image mode, an arc image mode, a trajectory image mode, a base image mode, and a flythrough image mode. ### Claim 4 (depends on 1) The image processing apparatus of claim 1, wherein the first indicator element indicates the relative location of the second image slice as related to the first image. ### Claim 5 (depends on 1) The image processing apparatus of claim 1, wherein, in response to receiving a user input for selecting an image slice within the indicator element associated with the first image mode, the processing apparatus is adapted to: cause the display displaying a first image slice in the first image mode to change the displayed image slice to an image slice that corresponds to selected slice within the first indicator element, and cause the display displaying a second image slide in the second image mode to change the displayed image slice to an image slice that corresponds to a location intersecting with the displayed first image slice. ### Claim 6 (depends on 5) The image processing apparatus of claim 5, wherein the processor is adapted to: cause a display to display a third image slice from the three-dimensional image set in a third image mode; cause the display to display, concurrent with the third image slice, a third indicator element associated with the third image mode; cause the display displaying the third image slide in the third image mode to change the displayed image slice to an image slice that corresponds to a location intersecting with the displayed first or second image slice. ### Claim 7 (depends on 1) The image processing apparatus of claim 1, wherein the processor generates and displays at least one of: a reference trajectory connecting the target point and the insertion point, a reachable zone, and one or more fiducial markers. ### Claim 8 (depends on 1) The image processing apparatus of claim 1, wherein the at least one region of interest or at least one target point in the image and the at least one insertion point are determined in response to receiving a user input for selecting a position in a displayed image slice. ### Claim 9 (depends on 1) The image processing apparatus of claim 1, wherein the processor displays the region of interest and the insertion point superimposed on at least one of the first or second image slice. ### Claim 10 (depends on 1) The image processing apparatus of claim 1, further comprising an input means for inputting a desired viewing location within the three dimensional data set. ### Claim 11 A method of performing planning or treatment for a percutaneous probe treatment, comprising: acquiring a three-dimensional image set comprising a stack of image slices; causing a display to display a first image slice from the three-dimensional image set in a first image mode; causing the display to display, concurrent with the first image slice, a first indicator element associated with the first image mode; causing a display to display a second image slice from the three-dimensional image set in a second image mode; causing the display to display, concurrent with the first image slice, a second indicator element associated with the second image mode; wherein the first and second indicator elements each indicate: the location of the first and second image slice within the three-dimensional image set, respectively, and the location of the at least one region of interest and/or at least one target point. ### Claim 12 (depends on 11) The method of claim 11, further comprising causing the processor to display, on the image slice, when applicable, at least one of: a reference trajectory connecting the target point and the insertion point, a reachable zone, and one or more fiducial markers. ### Claim 13 (depends on 11) The method of claim 11, further comprising receiving at least one user input to acquire the location of at least one region of interest or at least one target point in the image and the at least one insertion point. ### Claim 14 An image processing apparatus comprising a processor that operates to: acquire a three-dimensional image set comprising a stack of image slices; cause a display to display: (a) a first image slice from the three-dimensional image set in a first image mode and a first indicator element associated with the first image mode and containing a first marker that indicates the location of at least one of the first image slice within the three-dimensional image set, an insertion point, a target point, or a region of interest; and (b) a second image slice from the three-dimensional image set in a second image mode and a second indicator element associated with the second image mode and containing a second marker that indicates at least one of the location of the second image slice within the three-dimensional image set, an insertion point, a target point, or a region of interest; wherein the processor further operates, based on a user selecting and moving the first marker or the second marker, to: (a) update the display of the first image slice to an updated first image slice in the first image mode, wherein the first marker indicates the location of the updated first image slice, the insertion point, the target point, or the region of interest; and (b) update the display of the second image slice to an updated second image slice in the second image mode, wherein the second marker indicates the location of the updated second image slice, the insertion point, the target point, or the region of interest. ### Claim 15 (depends on 14) The image processing apparatus of claim 14, wherein at least one of the first and second image modes is an imaging mode that is out-of-plane compared to the three-dimensional image set. ### Claim 16 (depends on 14) The image processing apparatus of claim 14, wherein the first and second image mode are selected from: an axial image mode, a sagittal image mode, a coronal image mode, an arc image mode, a trajectory image mode, a base image mode, and a flythrough image mode. ### Claim 17 (depends on 14) The image processing apparatus of claim 14, wherein the processor is adapted to: cause a display to display (c) a third image slice from the three-dimensional image set in a third image mode; and wherein the processor further operates, based on a user selecting and moving the first marker or the second marker, to: (c) update the display of the third image slice to an updated third image slice in the third image mode. ### Claim 18 (depends on 14) The image processing apparatus of claim 14, wherein the processor generates and displays at least one of: a reference trajectory connecting the target point and the insertion point, a reachable zone, and one or more fiducial markers, on at least the first indicator element and the second indicator element. ### Claim 19 A method of performing planning or treatment for a percutaneous probe treatment, comprising: acquiring a three-dimensional image set comprising a stack of image slices; causing a display to display: (a) a first image slice from the three-dimensional image set in a first image mode and a first indicator element associated with the first image mode and containing a first marker that indicates the location of at least one of the first image slice within the three-dimensional image set, an insertion point, a target point, or a region of interest; and (b) a second image slice from the three-dimensional image set in a second image mode and a second indicator element associated with the second image mode and containing a second marker that indicates at least one of the location of the second image slice within the three-dimensional image set, an insertion point, a target point, or a region of interest; wherein the processor further operates, based on a user selecting and moving the first marker or the second marker, to: (a) update the display of the first image slice to an updated first image slice in the first image mode, wherein the first marker indicates the location of the updated first image slice, the insertion point, the target point, or the region of interest; and (b) update the display of the second image slice to an updated second image slice in the second image mode, wherein the second marker indicates the location of the updated second image slice, the insertion point, the target point, or the region of interest. ### Claim 20 (depends on 19) A imaging server storing an imaging application having instructions that, when executed by a processor, cause the ablation server to perform the method according to claim 19. ### Claim 21 (depends on 19) A non-transitory computer-readable storage medium storing an imaging application to cause an ablation server to perform the method according to claim 19.