Prostate puncture guidance method, apparatus, device and system based on multi-modal fusion

A prostate puncture guidance method based on multi-modal fusion, comprising steps of:

• obtaining a medical image sequence of an object to be examined, and performing three-dimensional reconstruction based on the medical image sequence to obtain a three-dimensional reconstruction model of the object to be examined, wherein the three-dimensional reconstruction model comprises a prostate structure of the object to be examined, the prostate structure comprises a plurality of target identifiers, and the plurality of target identifiers comprise at least one of a prostate puncture marker and a target area marker;
• obtaining a real-time coordinate of an ultrasonic probe and an ultrasound image of the object to be examined, and obtaining at least one section image of the three-dimensional reconstruction model based on the real-time coordinate, the at least one section image being corresponding to at least one scanning plane of the ultrasonic probe in a one-to-one correspondence manner;
• displaying the three-dimensional reconstruction model in a first display area, displaying an ultrasonic probe model of the ultrasonic probe based on the real-time coordinate, and displaying the at least one section image of the three-dimensional reconstruction corresponding to the at least one scanning plane of the ultrasonic probe in at least one detection region of the ultrasonic probe model;
• displaying a fusion image of the at least one section image of the three-dimensional reconstruction corresponding to the at least one scanning plane of the ultrasonic probe and the ultrasound image in a second display area;
• projecting at least one of the plurality of target identifiers onto the fusion image to obtain two-dimensional imaging of the at least one of the plurality of target identifiers when the at least one scanning plane intersects the at least one of the plurality of target identifiers in the three-dimensional reconstruction model;
• and
• correlating the two-dimensional imaging of the at least one of the plurality of target identifiers with at least one corresponding puncture hole of a puncture plate installed on the ultrasonic probe, and displaying a correlation relationship between the two-dimensional imaging of the at least one of the plurality of target identifiers and the at least one corresponding puncture hole in the second display area,
• wherein the step of projecting at least one of the plurality of target identifiers onto the fusion image to obtain two-dimensional imaging of the at least one of the plurality of target identifiers comprises:
• acquiring first coordinate information of the at least one scanning plane and second coordinate information of the at least one of the plurality of target identifiers that intersects the at least one scanning plane;
• and
• projecting the at least one of the plurality of target identifiers onto the fusion image of the at least one scanning plane in two dimensions based on the first coordinate information and the second coordinate information,
• wherein the puncture plate is provided with a plurality of puncture holes, and
• wherein the step of correlating the two-dimensional imaging of the at least one of the plurality of target identifiers with at least one corresponding puncture hole of the puncture plate installed on the ultrasonic probe, and displaying a correlation relationship between the two-dimensional imaging of the at least one of the plurality of target identifiers and the at least one corresponding puncture hole in the second display area comprises:
• calculating to determine the at least one corresponding puncture hole corresponding to the at least one of the plurality of target identifiers which intersects the at least one scanning plane according to the second coordinate information, displaying the plurality of puncture holes in the second display area, and displaying at least one extending line extending from the at least one corresponding puncture hole and passing through the at least one of the plurality of target identifiers corresponding to the at least one corresponding puncture hole in the second display area to show the correlation relationship between the two-dimensional imaging of the at least one of the plurality of target identifiers and the at least one corresponding puncture hole.