Multi-modal image alignment method and system

A multi-modal image alignment method, performing by a processing device, the multi-modal image alignment method comprising:

• obtaining a plurality of two-dimensional (2D) images and a plurality of three-dimensional (3D) images associated with a calibration object, wherein the calibration object comprises a center vertex and a plurality of side vertices, the plurality of 2D images are associated with a first 3D coordinate system, and the plurality of 3D images are associated with a second 3D coordinate system;
• obtaining a plurality of first points corresponding to the center vertex and a plurality of second point groups corresponding to the plurality of side vertices from the plurality of 2D images;
• obtaining a plurality of third points corresponding to the center vertex from the plurality of 3D images;
• performing first optimizing computation based on the first 3D coordinate system by using an initial first transformation matrix, the plurality of first points and the plurality of third points to obtain an optimized first transformation matrix;
• processing on the plurality of 3D images by using the optimized first transformation matrix to generate a plurality of firstly-transformed images respectively;
• performing second optimizing computation based on the first 3D coordinate system by using the plurality of firstly-transformed images, the plurality of first points, the plurality of second point groups and a predetermined specification parameter set of the calibration object to obtain an optimized second transformation matrix;
• and
• transforming an image to be processed to the second 3D coordinate system or the first 3D coordinate system by using the optimized first transformation matrix and the optimized second transformation matrix.