Neural network-based pose estimation and registration method and device for heterogeneous images, and medium

A neural network-based pose estimation method for heterogeneous images, comprising:

• step S1, taking a first pre-trained U-Net network and a second pre-trained U-Net network as two feature extractors, taking a heterogeneous template image and a picture to be matched as two original input pictures of the two feature extractors, respectively, and extracting isomorphic features in the two original input pictures to obtain a first feature map and a second feature map which are isomorphic:
• step S2, performing a Fourier transform on the first feature map and the second feature map obtained in step S1, respectively, to obtain respective magnitude spectra:
• step S3, performing a logarithmic polar coordinate transform on the two magnitude spectra obtained in step S2, respectively, to allow the two magnitude spectra to be transformed from a Cartesian coordinate system to a logarithmic polar coordinate system, in such a manner that a rotation transform between the two magnitude spectra in the Cartesian coordinate system is mapped into a translation transform in a v direction in the logarithmic polar coordinate system:
• step S4, performing a phase correlation solution on the two magnitude spectra after coordinate transform in step S3 to obtain a translation transform relation between the two magnitude spectra, and re-transforming according to a mapping relation between the Cartesian coordinate system and the logarithmic polar coordinate system in step S3 to obtain a rotation transform relation between the template image and the picture to be matched:
• step S5, taking a third pre-trained U-Net network and a fourth pre-trained U-Net network as two feature extractors, taking a heterogeneous template image and a picture to be matched as two original input pictures of the two feature extractors, respectively, and extracting isomorphic features in the two original input pictures to obtain a third feature map and a fourth feature map which are isomorphic:
• step S6, performing a Fourier transform on the third feature map and the fourth feature map obtained in step S5, respectively, to obtain respective magnitude spectra;
• step S7, performing a logarithmic polar coordinate transform on the two magnitude spectra obtained in step S6, respectively, to allow the two magnitude spectra to be transformed from a Cartesian coordinate system to a logarithmic polar coordinate system, in such a manner that a scaling transform between the two magnitude spectra in the Cartesian coordinate system is mapped into a translation transform in an x direction in the logarithmic polar coordinate system:
• step S8, performing a phase correlation solution on the two magnitude spectra after coordinate transform in step S7 to obtain a translation transform relation between the two magnitude spectra, and re-transforming according to a mapping relation between the Cartesian coordinate system and the logarithmic polar coordinate system in step S7 to obtain a scaling transform relation between the template image and the picture to be matched:
• step S9, performing corresponding rotation and scaling transforms on the picture to be matched according to the rotation transform relation and scaling transform relation obtained in step S4 and step S8 to obtain a new template image of the picture to be matched:
• step S10, taking a fifth pre-trained U-Net network and a sixth pre-trained U-Net network as two feature extractors, taking a template image and a new picture to be matched as two original input pictures of the two feature extractors, respectively, and extracting isomorphic features in the two original input pictures to obtain a fifth feature map and a sixth feature map that are isomorphic:
• step S11, performing a phase correlation solution on the fifth feature map and the sixth feature map obtained in step S10 to obtain a translation transform relation in the x direction between the template image and the picture to be matched:
• step S12, taking a seventh pre-trained U-Net network and a eighth pre-trained U-Net network as two feature extractors, taking a template image and a new picture to be matched as two original input pictures of the two feature extractors, respectively, extracting isomorphic features in the two original input pictures, and obtaining a seventh feature map and an eighth feature map which are isomorphic and only retain a translation transform relation between the original input pictures;
• and
• S13, performing a phase correlation solution on the seventh feature map and the eighth feature map obtained in step S12 to obtain a translation transform relation in the y direction between the template image and the picture to be matched, and completing a pose estimation of three transform relations of rotation, scaling and translation between the heterogeneous template image and the picture to be matched.