Surgery assisting system, surgery assisting method, and surgery assisting program

A surgery assisting method that causes a computer used in a surgery assisting system to perform:

• an image acquisition step of acquiring an image captured by an endoscope;
• a surgical technique selection reception step of accepting a selection of a surgical technique of a surgery;
• an organ area estimation step of inputting the image acquired in the image acquisition step and identification information indicating the surgical technique accepted in the surgical technique selection reception step into an organ area estimation model that has learned in advance a relationship among identification information indicating the surgical technique of the surgery, the image captured by the endoscope during the surgery, and a position and range of an organ that is likely to be damaged during the surgery associated with the surgical technique in the image to estimate the position and range of the organ that is likely to be damaged during the surgery in the image acquired in the image acquisition step;
• an estimated area display step of superimposing and displaying information indicating the position and range of the organ that is likely to be damaged during the surgery estimated in the organ area estimation step on the image acquired in the image acquisition step;
• a surgical instrument tip detection step of detecting a position of a tip portion of a surgical instrument in the image acquired in the image acquisition step using a surgical instrument detection model that has learned in advance a shape of the tip portion of the surgical instrument;
• an organ damage determination step of determining whether the position of the tip portion detected in the surgical instrument tip detection step has reached the organ that is likely to be damaged during the surgery estimated in the organ area estimation step;
• and
• an organ damage notification step of notifying a surgeon of a possibility of organ damage when it is determined in the organ damage determination step that the position of the tip portion has reached the estimated organ.

A non-transitory computer readable medium storing therein a surgery assisting program that causes a computer used in a surgery assisting system to embody:

• an image acquisition function that acquires an image captured by an endoscope;
• a surgical technique selection reception function that accepts a selection of a surgical technique of a surgery;
• an organ area estimation function that inputs the image acquired by the image acquisition function and identification information indicating the surgical technique accepted by the surgical technique selection reception function into an organ area estimation model that has learned in advance a relationship among identification information indicating the surgical technique of the surgery, the image captured by the endoscope during the surgery, and a position and range of an organ that is likely to be damaged during the surgery associated with the surgical technique in the image to estimate the position and range of the organ that is likely to be damaged during the surgery in the image acquired by the image acquisition function;
• an estimated area display function that superimposes and displays information indicating the position and range of the organ that is likely to be damaged during the surgery estimated by the organ area estimation function on the image acquired by the image acquisition function;
• a surgical instrument tip detection function that detects a position of a tip portion of a surgical instrument in the image acquired by the image acquisition function using a surgical instrument detection model that has learned in advance a shape of the tip portion of the surgical instrument;
• an organ damage determination function that determines whether the position of the tip portion detected by the surgical instrument tip detection function has reached the organ that is likely to be damaged during the surgery estimated by the organ area estimation function;
• and
• an organ damage notification function that notifies a surgeon of a possibility of organ damage when the organ damage determination function determines that the position of the tip portion has reached the estimated organ.

A surgery assisting system comprising:

• an image acquisition system configured to acquire an image captured by an endoscope;
• a surgical technique selection reception system configured to accept a selection of a surgical technique of a surgery;
• an organ area estimation system configured to input the image acquired by the image acquisition system and identification information indicating the surgical technique accepted by the surgical technique selection reception system into an organ area estimation model that has learned in advance a relationship among identification information indicating the surgical technique of the surgery, the image captured by the endoscope during the surgery, and a position and range of an organ that is likely to be damaged during the surgery associated with the surgical technique in the image to estimate the position and range of the organ that is likely to be damaged during the surgery in the image acquired by the image acquisition system;
• and
• an estimated area display system configured to superimpose and display information indicating the position and range of the organ that is likely to be damaged during the surgery estimated by the organ area estimation system on the image acquired by the image acquisition system, wherein when the organ area estimation system estimates a position and range of each of a plurality of organs with a high probability of being damaged during the surgery in the image acquired by the image acquisition system, the estimation area display system displays information indicating the organs with a high probability of being damaged during the surgery estimated by the organ area estimation system by superimposing the information on the image acquired by the image acquisition system in different display forms, respectively.

A surgery assisting method that causes a computer used in a surgery assisting system to perform:

• an image acquisition step of acquiring an image captured by an endoscope;
• a surgical technique selection reception step of accepting a selection of a surgical technique of a surgery;
• an organ area estimation step of inputting the image acquired in the image acquisition step and identification information indicating the surgical technique accepted in the surgical technique selection reception step into an organ area estimation model that has learned in advance a relationship among identification information indicating the surgical technique of the surgery, the image captured by the endoscope during the surgery, and a position and range of an organ that is likely to be damaged during the surgery associated with the surgical technique in the image to estimate the position and range of the organ that is likely to be damaged during the surgery in the image acquired in the image acquisition step;
• and
• an estimated area display step of superimposing and displaying information indicating the position and range of the organ that is likely to be damaged during the surgery estimated in the organ area estimation step on the image acquired in the image acquisition step, wherein when the organ area estimation step estimates a position and range of each of a plurality of organs with a high probability of being damaged during the surgery in the image acquired in the image acquisition step, the estimated area display step displays information indicating the organs with a high probability of being damaged during the surgery estimated in the organ area estimation step by superimposing the information on the image acquired in the image acquisition step in different display forms, respectively.

A non-transitory computer readable medium storing therein a surgery assisting program that causes a computer used in a surgery assisting system to embody:

• an image acquisition function that acquires an image captured by an endoscope;
• a surgical technique selection reception function that accepts a selection of a surgical technique of a surgery;
• an organ area estimation function that inputs the image acquired by the image acquisition function and identification information indicating the surgical technique accepted by the surgical technique selection reception function into an organ area estimation model that has learned in advance a relationship among identification information indicating the surgical technique of the surgery, the image captured by the endoscope during the surgery, and a position and range of an organ that is likely to be damaged during the surgery associated with the surgical technique in the image to estimate the position and range of the organ that is likely to be damaged during the surgery in the image acquired by the image acquisition function;
• and
• an estimated area display function that superimposes and displays information indicating the position and range of the organ that is likely to be damaged during the surgery estimated by the organ area estimation function on the image acquired by the image acquisition function, wherein when the organ area estimation function estimates a position and range of each of a plurality of organs with a high probability of being damaged during the surgery in the image acquired by the image acquisition function, the estimated area display function displays information indicating the organs with a high probability of being damaged during the surgery estimated by the organ area estimation function by superimposing the information on the image acquired by the image acquisition function in different display forms, respectively.

A surgery assisting system comprising:

• an image acquisition system configured to acquire an image captured by an endoscope;
• a surgical technique selection reception system configured to accept a selection of a surgical technique of a surgery;
• an organ area estimation system configured to input the image acquired by the image acquisition system and identification information indicating the surgical technique accepted by the surgical technique selection reception system into an organ area estimation model that has learned in advance a relationship among identification information indicating the surgical technique of the surgery, the image captured by the endoscope during the surgery, and a position and range of an organ that is likely to be damaged during the surgery associated with the surgical technique in the image to estimate the position and range of the organ that is likely to be damaged during the surgery in the image acquired by the image acquisition system;
• and
• an estimated area display system configured to superimpose and display information indicating the position and range of the organ that is likely to be damaged during the surgery estimated by the organ area estimation system on the image acquired by the image acquisition system, wherein the image acquisition system acquires a plurality of consecutive images, and the organ area estimation system inputs the consecutive images acquired by the image acquisition system into the organ area estimation model to estimate a trajectory of a position and range of an organ with a high probability of being damaged during the surgery.

A surgery assisting method that causes a computer used in a surgery assisting system to perform:

• an image acquisition step of acquiring an image captured by an endoscope;
• a surgical technique selection reception step of accepting a selection of a surgical technique of a surgery;
• an organ area estimation step of inputting the image acquired in the image acquisition step and identification information indicating the surgical technique accepted in the surgical technique selection reception step into an organ area estimation model that has learned in advance a relationship among identification information indicating the surgical technique of the surgery, the image captured by the endoscope during the surgery, and a position and range of an organ that is likely to be damaged during the surgery associated with the surgical technique in the image to estimate the position and range of the organ that is likely to be damaged during the surgery in the image acquired in the image acquisition step;
• and
• an estimated area display step of superimposing and displaying information indicating the position and range of the organ that is likely to be damaged during the surgery estimated in the organ area estimation step on the image acquired in the image acquisition step, wherein the image acquisition step acquires a plurality of consecutive images, and the organ area estimation step inputs the consecutive images acquired by the image acquisition step into the organ area estimation model to estimate a trajectory of a position and range of an organ with a high probability of being damaged during the surgery.

A non-transitory computer readable medium storing therein a surgery assisting program that causes a computer used in a surgery assisting system to embody:

• an image acquisition function that acquires an image captured by an endoscope;
• a surgical technique selection reception function that accepts a selection of a surgical technique of a surgery;
• an organ area estimation function that inputs the image acquired by the image acquisition function and identification information indicating the surgical technique accepted by the surgical technique selection reception function into an organ area estimation model that has learned in advance a relationship among identification information indicating the surgical technique of the surgery, the image captured by the endoscope during the surgery, and a position and range of an organ that is likely to be damaged during the surgery associated with the surgical technique in the image to estimate the position and range of the organ that is likely to be damaged during the surgery in the image acquired by the image acquisition function;
• and
• an estimated area display function that superimposes and displays information indicating the position and range of the organ that is likely to be damaged during the surgery estimated by the organ area estimation function on the image acquired by the image acquisition function, wherein the image acquisition function acquires a plurality of consecutive images, and the organ area estimation function inputs the consecutive images acquired by the image acquisition function into the organ area estimation model to estimate a trajectory of a position and range of an organ with a high probability of being damaged during the surgery.