Imaging system and methods displaying a fused multidimensional reconstructed image

2:

• An imaging system comprising:
• at least one photosensor configured to record a visualization of a surgical site;
• a visualization generation system configured to create a multidimensional visualization based on the recorded visualization of the surgical site;
• an interface configured to receive image data of the surgical site from at least one of a medical device or a server that stores the image data from a medical device;
• a processor communicatively coupled to a display device and a memory, the memory storing instructions, which when executed by the processor, cause the processor to:
• create a three-dimensional image or model based on the image data,
• display the multidimensional visualization of the surgical site,
• receive a selection corresponding to a portion of the displayed multidimensional visualization of the surgical site,
• select a portion of the three-dimensional image or model that corresponds to the selected portion of the multidimensional visualization,
• combine the portion of the three-dimensional image or model with the multidimensional visualization, and
• cause the display device to display the portion of the three-dimensional image or model in addition to the multidimensional visualization.

10:

• The imaging system of claim 2, wherein the memory includes additional instructions, which when executed by the processor, cause the processor to select the portion of the three-dimensional image or model by:
• determining at least one of an orientation and coordinates of the selected portion of the multidimensional visualization;
• determining at least one of a corresponding orientation and coordinates of the three-dimensional image or model;
• and
• selecting the portion of the three-dimensional image or model that is associated with the at least one of the corresponding orientation and coordinates.

11:

• The imaging system of claim 2, wherein the memory includes additional instructions, which when executed by the processor, cause the processor to combine the portion of the three-dimensional image or model with the multidimensional visualization by:
• identifying at least one of a feature, a structure, and an object inside of or within proximity of the selected portion of the multidimensional visualization;
• identifying at least one of a corresponding feature, structure, and object within the three-dimensional image or model;
• and
• aligning the portion of the three-dimensional image or model with the selected portion of the multidimensional visualization such that the at least one identified feature, structure, and object of the multidimensional visualization is aligned with the at least one corresponding identified feature, structure, and object of the three-dimensional image or model.

13:

• An imaging apparatus comprising:
• a visualization generation system configured to create a multidimensional visualization based on a recorded visualization of a surgical site;
• an interface configured to receive image data of the surgical site from at least one of a medical device or a server that stores the image data from a medical device;
• a processor communicatively coupled to a display device and a memory, the memory storing instructions, which when executed by the processor, cause the processor to:
• create a reconstructed image based on the image data,
• display the multidimensional visualization of the surgical site,
• receive a selection corresponding to a portion of the displayed multidimensional visualization of the surgical site,
• select a portion of the reconstructed image that corresponds to the selected portion of the multidimensional visualization,
• combine the portion of the reconstructed image with the multidimensional visualization, and
• cause the display device to display the portion of the reconstructed image in addition to the multidimensional visualization.

15:

• The imaging apparatus of claim 14, wherein the memory includes additional instructions, which when executed by the processor, cause the processor to create the three-dimensional image or model by:
• determining a sequence of the two-dimensional image slices;
• identifying a common structure between the image slices;
• and
• forming corresponding three dimensional shapes based on the identified common structure.

16:

• The imaging apparatus of claim 13, wherein the memory includes additional instructions, which when executed by the processor, cause the processor to:
• create the reconstructed image at a first time before the visualization of the surgical site is recorded;
• and
• display the multidimensional visualization of the surgical site at a second time after the visualization of the surgical site is recorded.

19:

• The imaging apparatus of claim 18, wherein the memory includes additional instructions, which when executed by the processor, cause the processor to adjust the visual property by at least one of:
• (a) increasing a transparency of the selected portion of a multidimensional visualization so that the portion of the reconstructed image replaces the selected portion of the multidimensional visualization;
• (b) adjusting a contrast around edges or fringes of at least one of the selected portion of a multidimensional visualization and the portion of the reconstructed image;
• (c) adjusting a focus around edges or fringes of at least one of the selected portion of a multidimensional visualization and the portion of the reconstructed image;
• and
• (d) applying a spline function around edges or fringes of at least one of the selected portion of a multidimensional visualization and the portion of the reconstructed image.

20:

• The imaging apparatus of claim 13, wherein the memory includes additional instructions, which when executed by the processor, cause the processor to combine the portion of the reconstructed image with the multidimensional visualization by:
• identifying a target structure within the multidimensional visualization;
• identifying the same target structure within the reconstructed image;
• aligning the reconstructed image with the multidimensional visualization such that the target structure within the multidimensional visualization is aligned with the target structure within the reconstructed image;
• and
• identifying the portion of the reconstructed image that is located at a same location as the selected portion of the multidimensional visualization.