Image processing apparatus, system, image processing method, and image processing program

An image processing method executed by a processor, the image processing method comprising:

• acquiring an optical camera image of a subject captured by an optical camera and an ultrasound image of the subject captured by an ultrasonography apparatus in a state where a plurality of markers are provided on the subject, each of the plurality of markers having a first feature identifiable by the optical camera image and a second feature identifiable by the ultrasound image, and being distinguishable from other markers based on the first feature and the second feature;
• detecting a region of interest from the ultrasound image of the subject;
• detecting a first marker image representing the marker from the optical camera image of the subject;
• detecting a second marker image representing the marker from the ultrasound image of the subject;
• associating, based on marker feature information, which is a correspondence relationship between the first feature, the second feature, and a type of the marker, the first marker image and the second marker image such that the first marker image and the second marker image represent the same type of marker based on the first feature of the first marker image and the second feature of the second marker image;
• and
• performing registration between the optical camera image of the subject and the ultrasound image of the subject based on a positional relationship between the associated first marker image and the associated second marker image and displaying an image representing the region of interest superimposed on the optical camera image of the subject,
• wherein the plurality of markers includes a marker embedded in a gel-pad positioned on a surface of the subject,
• the first feature is a color of a surface of the marker in the optical camera image, and
• the second feature is a position, in a depth direction, of the marker in the ultrasound image, the position corresponding to a depth from a surface of the gel-pad to a center of the marker.

A non-transitory computer-readable storage medium storing an image processing program causing a processor to execute a process comprising:

• acquiring an optical camera image of a subject captured by an optical camera and an ultrasound image of the subject captured by an ultrasonography apparatus in a state where a plurality of markers are provided on the subject, each of the plurality of markers having a first feature identifiable by the optical camera image and a second feature identifiable by the ultrasound image, and being distinguishable from other markers based on the first feature and the second feature;
• detecting a region of interest from the ultrasound image of the subject;
• detecting a first marker image representing the marker from the optical camera image of the subject;
• detecting a second marker image representing the marker from the ultrasound image of the subject;
• associating, based on marker feature information, which is a correspondence relationship between the first feature, the second feature, and a type of the marker, the first marker image and the second marker image such that the first marker image and the second marker image represent the same type of marker based on the first feature of the first marker image and the second feature of the second marker image;
• and
• performing registration between the optical camera image of the subject and the ultrasound image of the subject based on a positional relationship between the associated first marker image and the associated second marker image and displaying an image representing the region of interest superimposed on the optical camera image of the subject,
• wherein the plurality of markers includes a marker embedded in a gel-pad positioned on a surface of the subject,
• the first feature is a color of a surface of the marker in the optical camera image, and
• the second feature is a position, in a depth direction, of the marker in the ultrasound image, the position corresponding to a depth from a surface of the gel-pad to a center of the marker.