Self-supervised multimodal structured illumination microscopic reconstruction method and system

A self-supervised multimodal structured illumination microscopic reconstruction system, comprising:

• a super-resolution reconstruction module (210), configured to perform super-resolution reconstruction on an image using a standard structured illumination super-resolution reconstruction algorithm;
• and
• a denoising module (220), comprising a denoising neural network;
• wherein the self-supervised multimodal structured illumination microscopic reconstruction system is configured to:
• excite a biological sample with structured illumination to obtain J raw fluorescence image sequences generated by exciting of the biological sample, wherein J is an integer greater than or equal to 1, wherein each raw fluorescence image sequence comprises S fluorescence images, S is an integer greater than or equal to 2, and each fluorescence image has a pixel size of M*N, with M and N being even numbers;
• wherein the self-supervised multimodal structured illumination microscopic reconstruction system is further configured to, for each of the J raw fluorescence image sequences:
• (1) extract, by means of pixel realigning, a first sub-image, a second sub-image, a third sub-image, and a fourth sub-image each with a pixel size of (M/2)*(N/2) from an i-th fluorescence image among the S fluorescence images, wherein a two-dimensional (2D) pixel matrix yi,(2m-1)(2n-1) of the first sub-image, a 2D pixel matrix yi,(2m-1)(2n) of the second sub-image, a 2D pixel matrix yi,(2m)(2n-1) of the third sub-image, and a 2D pixel matrix yi,(2m)(2n) of the fourth sub-image are selected from a 2D pixel matrix yi, MN of the i-th fluorescence image respectively, where m is an integer and m=1, 2, 3, . . . , M/2, and n is an integer and n=1, 2, 3, . . . , N/2;
• (2) perform 2× pixel upsampling on the first sub-image, the second sub-image, the third sub-image, and the fourth sub-image to obtain a first upsampled sub-image, a second upsampled sub-image, a third upsampled sub-image, and a fourth upsampled sub-image each with a pixel size of M*N;
• (3) perform a pixel translation of [0.5, 0.5] on a 2D pixel matrix of the first upsampled sub-image to obtain a first 2D subpixel translated sub-image;
• perform a pixel translation of [−0.5, 0.5] on a 2D pixel matrix of the second upsampled sub-image to obtain a second 2D subpixel translated sub-image;
• perform a pixel translation of [0.5, −0.5] on a 2D pixel matrix of the third upsampled sub-image to obtain a third 2D subpixel translated sub-image;
• and perform a pixel translation of [−0.5, −0.5] on a 2D pixel matrix of the fourth upsampled sub-image to obtain a fourth 2D subpixel translated sub-image;
• (4) group S first 2D subpixel translated sub-images obtained from the S fluorescence images into a first sub-image group, group S second 2D subpixel translated sub-images obtained from the S fluorescence images into a second sub-image group, group S third 2D subpixel translated sub-images obtained from the S fluorescence images into a third sub-image group, and group S fourth 2D subpixel translated sub-images obtained from the S fluorescence images into a fourth sub-image group;
• (5) perform SIM reconstruction on the first sub-image group, the second sub-image group, the third sub-image group, and the fourth sub-image group using a standard SIM reconstruction algorithm, to obtain a first super-resolution sub-image, a second super-resolution sub-image, a third super-resolution sub-image, and a fourth super-resolution sub-image respectively;
• and
• (6) select two different super-resolution sub-images from the first super-resolution sub-image, the second super-resolution sub-image, the third super-resolution sub-image, and the fourth super-resolution sub-image to form a super-resolution image pair for training, and incorporate the super-resolution image pair for training into a training set;
• and
• wherein the self-supervised multimodal structured illumination microscopic reconstruction system is further configured to:
• (7) train a denoising neural network based on the training set;
• and
• (8) perform super-resolution reconstruction on the S fluorescence images in each of the J raw fluorescence image sequences using a standard structured illumination super-resolution reconstruction algorithm, to form a super-resolution image, wherein the super-resolution image is input into the denoising neural network to obtain a final super-resolution reconstructed image.