Augmented medical vision systems and methods

A system comprising:

• a memory storing instructions;
• and
• a processor communicatively coupled to the memory and configured to execute the instructions to:
• generate a visible light image stream based on visible light from a surgical area captured by an imaging device and generate a fluorescence image stream based on fluorescence illumination from the surgical area captured by the imaging device,
• operate in a first display mode by directing a display device to display a first video stream based on a first set of at least one of the visible light image stream and the fluorescence image stream,
• determine, while operating in the first display mode, that an event occurs within the surgical area, the event comprising at least one of a fluorescence event or an operation of a surgical instrument other than the imaging device,
• switch, in response to the determining that the event occurs within the surgical area, from operating in the first display mode to operating in a second display mode, and
• disable a surgical instrument functionality of a surgical instrument located within the surgical area while operating in the second display mode,
• wherein the operating in the second display mode comprises directing the display device to display, in place of the first video stream, a second video stream based on a second set of at least one of the visible light image stream and the fluorescence image stream, the first set being different than the second set and the second set including the fluorescence image stream.

A system comprising:

• a memory storing instructions;
• and
• a processor communicatively coupled to the memory and configured to execute the instructions to:
• generate a visible light image stream based on visible light from a surgical area captured by an imaging device and generate a fluorescence image stream based on fluorescence illumination from the surgical area captured by the imaging device,
• direct a display device to display only the visible light image stream,
• determine, while the visible light image stream is displayed, that an event occurs within the surgical area, the event comprising at least one of a fluorescence event or an operation of a surgical instrument other than the imaging device, and
• switch, in response to the determining that the event occurs within the surgical area, from directing the display device to display only the visible light image stream to directing the display device to display a second video stream based at least in part on the fluorescence image stream,
• determine, while the display device displays the second video stream, that an additional event occurs within the surgical area, and
• switch, in response to the determining that the additional event occurs with in the surgical area, from directing the display device to display the second video stream to directing the display device to display a third video stream based at least in part on the fluorescence image stream, wherein the third video stream is different than the second video stream.

A method comprising:

• generating, by an augmented medical vision system, a visible light image stream based on visible light from a surgical area captured by an imaging device and generate a non-visible light image stream based on non-visible light from the surgical area captured by the imaging device;
• operating, by the augmented medical vision system, in a first display mode by directing a display device communicatively coupled with the augmented medical vision system to display a first video stream based on a first set of at least one of the visible light image stream and the non-visible light image stream;
• determining, by the augmented medical vision system while operating in the first display mode, that an event occurs within the surgical area, the event comprising at least one of a fluorescence event or an operation of a surgical instrument other than the imaging device;
• switching, by the augmented medical vision system in response to the determining that the event occurs within the surgical area, from operating in the first display mode to operating in a second display mode;
• determining, by the augmented medical vision system while operating in the second display mode, that an additional event occurs within the surgical area;
• and
• switching, by the augmented medical vision system in response to the determining that the additional event occurs within the surgical area, from operating in the second display mode to operating in the first display mode or a third display mode,
• wherein the operating in the second display mode comprises directing the display device to display, in place of the first video stream, a second video stream based on a second set of at least one of the visible light image stream and the non-visible light image stream, the first set being different than the second set;
• and
• wherein the operating in the third display mode comprises directing the display device to display, in place of the second video stream, a third video stream based on a third set of at least one of the visible light image stream and the non-visible light image stream, the third set being different than the first set and the second set.