# US 12,224,057 B2 · Medical image processing method, processing apparatus, and computer-readable storage medium > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,224,057 B2 - Document ID: US12224057B2 - Status: Granted - Kind: B2 - Application number: 17/739,121 - Filed: 2022-05-08 - Published: 2022-08-18 - Granted: 2025-02-11 - Earliest claimed: 2020-07-21 - Assignees: Shenzhen Institutes of Advanced Technology - Applicants: Shenzhen Institutes of Advanced Technology - Inventors: Na Zhang; Hairong Zheng; Xin Liu; Zhanli Hu; Zhiyuan Huang; Dong Liang - Web page: https://patentgrep.com/patent/US12224057B2 ## CPC - G16H 30/40: for processing medical images, e.g. editing - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/084: Backpropagation, e.g. using gradient descent - G06N 3/09: Supervised learning - G06T 5/70: Denoising; Smoothing - G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching} - G06V 10/7753: {Incorporation of unlabelled data, e.g. multiple instance learning [MIL]} - G06V 2201/03: Recognition of patterns in medical or anatomical images - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A medical image processing method and processing apparatus, and a computer readable storage medium. The method includes: obtaining a to-be-processed image; performing a feature extraction on the to-be-processed image to obtain a corresponding feature image; and re-determining a pixel value of each pixel in the to-be-processed image based on first information and second information of a corresponding pixel in the feature image, and processing the to-be-processed image; wherein the first information is information of a pixel adjacent to the corresponding pixel in the features image, and the second information is information of a pixel that is not adjacent to and is similar to the corresponding pixel in the features image. ## Claims ### Claim 1 A medical image processing method, comprising: obtaining a to-be-processed image; performing a feature extraction on the to-be-processed image to obtain a corresponding feature image; and re-determining a pixel value of each pixel in the to-be-processed image based on first information and second information of a corresponding pixel in the feature image, and processing the to-be-processed image; wherein the first information is information of a pixel adjacent to the corresponding pixel in the features image, and the second information is information of a pixel that is not adjacent to and is similar to the corresponding pixel in the features image, wherein the re-determining a pixel value of each pixel in the to-be-processed image based on first information and second information of a corresponding pixel in the feature image, and processing the to-be-processed image comprise: obtaining pixels in a neighborhood with a radius r around the corresponding pixel in the feature image to form a first pixel set, and obtaining pixels similar to the corresponding pixel in the feature image and located outside the neighborhood to form a second pixel set; wherein r is a preset neighborhood radius parameter, being a positive integer; obtaining a first contribution rate of each first pixel in the first pixel set to the corresponding pixel in the features image, and obtaining a second contribution rate of each second pixel in the second pixel set to the corresponding pixel in the features image; wherein a sum of all the first contribution rates and all the second contribution rates is 1; and for each first pixel, multiplying a pixel value and the corresponding first contribution rate of the first pixel to obtain a first product; for each second pixel, multiplying a pixel value and the corresponding second contribution rate of the second pixel to obtain a second product; and adding all the first products with all the second products to obtain a sum, and taking the sum as a pixel value of the corresponding pixel in the features image. ### Claim 2 (depends on 1) The method according to claim 1, wherein the obtaining pixels similar to the corresponding pixel in the feature image and located outside the neighborhood to form a second pixel set comprises: calculating a Euclidean distance between a feature vector of the corresponding pixel in the feature image and a feature vector of each pixel outside the neighborhood of radius r, and obtaining N pixels with a least Euclidean distance to form the second pixel set; wherein in the second pixel set, the less the Euclidean distance, the greater the second contribution rate of the corresponding second pixel; N is a positive integer. ### Claim 3 (depends on 1) The method according to claim 1, further comprising: inputting the to-be-processed image into a preset convolutional neural network, and performing the feature extraction on the to-be-processed image through the preset convolutional neural network, to obtain the corresponding feature image and to redetermine the pixel value of each pixel in the to-be-processed image based on the first information and the second information of the corresponding each pixel in the feature image, thereby processing the to-be-processed image. ### Claim 4 (depends on 3) The method according to claim 3, further comprising: establishing the preset convolutional neural network; wherein the preset convolutional neural network comprises at least one feature extraction layer, at least one denoising layer, at least one batch of normalization layers, and at least one activation layer; each feature extraction layer is connected to one of the at least one activation layer, and each denoising layer is sequentially connected to one of the at least one batch of normalization layers and one of the at least one activation layer; obtaining an unprocessed medical image and a corresponding processed medical image as a training data set; and training the preset convolutional neural network with the unprocessed medical image as input data and the corresponding processed medical image as a ground truth label. ### Claim 5 (depends on 4) The method according to claim 4, wherein the preset convolutional neural network further comprises at least one adaptive residual unit connected in series; each adaptive residual unit comprises three of the at least one denoising layer, three of the at least one activation layer 2, three of the at least one batch of normalization layers, and one adaptive skip connection unit. ### Claim 6 (depends on 4) The method according to claim 4, wherein the unprocessed medical image is a medical image obtained under a scanning condition less than a set time period, and the corresponding processed medical image is a medical image obtained under a scanning condition greater than the set time period. ### Claim 7 (depends on 4) The method according to claim 4, wherein the training the preset convolutional neural network with the unprocessed medical image as input data and the corresponding processed medical image as a ground truth label comprises: training the preset convolutional neural network through a mean square error loss function; and updating a model parameter of the preset convolutional neural network through a gradient back-propagation algorithm, such that the mean square error loss function tends to be minimized. ### Claim 8 A medical image processing apparatus, comprising a memory, a processor, and a computer program stored in the memory and executable by the processor; wherein the processor is capable of executing the computer program to implement: obtaining a to-be-processed image; performing a feature extraction on the to-be-processed image to obtain a corresponding feature image; and re-determining a pixel value of each pixel in the to-be-processed image based on first information and second information of a corresponding pixel in the feature image, and processing the to-be-processed image; wherein the first information is information of a pixel adjacent to the corresponding pixel in the features image, and the second information is information of a pixel that is not adjacent to and is similar to the corresponding pixel in the features image, wherein the re-determining a pixel value of each pixel in the to-be-processed image based on first information and second information of a corresponding pixel in the feature image, and processing the to-be-processed image comprise: obtaining pixels in a neighborhood with a radius r around the corresponding pixel in the feature image to form a first pixel set, and obtaining pixels similar to the corresponding pixel in the feature image and located outside the neighborhood to form a second pixel set; wherein r is a preset neighborhood radius parameter, being a positive integer; obtaining a first contribution rate of each first pixel in the first pixel set to the corresponding pixel in the features image, and obtaining a second contribution rate of each second pixel in the second pixel set to the corresponding pixel in the features image; wherein a sum of all the first contribution rates and all the second contribution rates is 1; and for each first pixel, multiplying a pixel value and the corresponding first contribution rate of the first pixel to obtain a first product; for each second pixel, multiplying a pixel value and the corresponding second contribution rate of the second pixel to obtain a second product; and adding all the first products with all the second products to obtain a sum, and taking the sum as a pixel value of the corresponding pixel in the features image. ### Claim 9 (depends on 8) The medical image processing apparatus according to claim 8, wherein the obtaining pixels similar to the corresponding pixel in the feature image and located outside the neighborhood to form a second pixel set comprises: calculating a Euclidean distance between a feature vector of the corresponding pixel in the feature image and a feature vector of each pixel outside the neighborhood of radius r, and obtaining N pixels with a least Euclidean distance to form the second pixel set; wherein in the second pixel set, the less the Euclidean distance, the greater the second contribution rate of the corresponding second pixel; N is a positive integer. ### Claim 10 (depends on 8) The medical image processing apparatus according to claim 8, wherein the processor is further configured to implement: inputting the to-be-processed image into a preset convolutional neural network, and performing the feature extraction on the to-be-processed image through the preset convolutional neural network, to obtain the corresponding feature image and to redetermine the pixel value of each pixel in the to-be-processed image based on the first information and the second information of the corresponding each pixel in the feature image, thereby processing the to-be-processed image. ### Claim 11 (depends on 10) The medical image processing apparatus according to claim 10, wherein the processor is further configured to implement: establishing the preset convolutional neural network; wherein the preset convolutional neural network comprises at least one feature extraction layer, at least one denoising layer, at least one batch of normalization layers, and at least one activation layer; each feature extraction layer is connected to one of the at least one activation layer, and each denoising layer is sequentially connected to one of the at least one batch of normalization layers and one of the at least one activation layer; obtaining an unprocessed medical image and a corresponding processed medical image as a training data set; and training the preset convolutional neural network with the unprocessed medical image as input data and the corresponding processed medical image as a ground truth label. ### Claim 12 (depends on 11) The medical image processing apparatus according to claim 11, wherein the preset convolutional neural network further comprises at least one adaptive residual unit connected in series; each adaptive residual unit comprises three of the at least one denoising layer, three of the at least one activation layer 2, three of the at least one batch of normalization layers, and one adaptive skip connection unit. ### Claim 13 (depends on 11) The medical image processing apparatus according to claim 11, wherein the unprocessed medical image is a medical image obtained under a scanning condition less than a set time period, and the corresponding processed medical image is a medical image obtained under a scanning condition greater than the set time period. ### Claim 14 (depends on 11) The medical image processing apparatus according to claim 11, wherein the training the preset convolutional neural network with the unprocessed medical image as input data and the corresponding processed medical image as a ground truth label comprises: training the preset convolutional neural network through a mean square error loss function; and updating a model parameter of the preset convolutional neural network through a gradient back-propagation algorithm, such that the mean square error loss function tends to be minimized. ### Claim 15 A non-transitory computer-readable storage medium, storing a computer program; wherein the computer program is executable by a processor to implement: obtaining a to-be-processed image; performing a feature extraction on the to-be-processed image to obtain a corresponding feature image; and re-determining a pixel value of each pixel in the to-be-processed image based on first information and second information of a corresponding pixel in the feature image, and processing the to-be-processed image; wherein the first information is information of a pixel adjacent to the corresponding pixel in the features image, and the second information is information of a pixel that is not adjacent to and is similar to the corresponding pixel in the features image, wherein the re-determining a pixel value of each pixel in the to-be-processed image based on first information and second information of a corresponding pixel in the feature image, and processing the to-be-processed image comprise: obtaining pixels in a neighborhood with a radius r around the corresponding pixel in the feature image to form a first pixel set, and obtaining pixels similar to the corresponding pixel in the feature image and located outside the neighborhood to form a second pixel set; wherein r is a preset neighborhood radius parameter, being a positive integer; obtaining a first contribution rate of each first pixel in the first pixel set to the corresponding pixel in the features image, and obtaining a second contribution rate of each second pixel in the second pixel set to the corresponding pixel in the features image; wherein a sum of all the first contribution rates and all the second contribution rates is 1; and for each first pixel, multiplying a pixel value and the corresponding first contribution rate of the first pixel to obtain a first product; for each second pixel, multiplying a pixel value and the corresponding second contribution rate of the second pixel to obtain a second product; and adding all the first products with all the second products to obtain a sum, and taking the sum as a pixel value of the corresponding pixel in the features image. ### Claim 16 (depends on 15) The non-transitory computer-readable storage medium according to claim 15, wherein the obtaining pixels similar to the corresponding pixel in the feature image and located outside the neighborhood to form a second pixel set comprises: calculating a Euclidean distance between a feature vector of the corresponding pixel in the feature image and a feature vector of each pixel outside the neighborhood of radius r, and obtaining N pixels with a least Euclidean distance to form the second pixel set; wherein in the second pixel set, the less the Euclidean distance, the greater the second contribution rate of the corresponding second pixel; N is a positive integer. ### Claim 17 (depends on 15) The non-transitory computer-readable storage medium according to claim 15, wherein the processor is further configured to implement: inputting the to-be-processed image into a preset convolutional neural network, and performing the feature extraction on the to-be-processed image through the preset convolutional neural network, to obtain the corresponding feature image and to redetermine the pixel value of each pixel in the to-be-processed image based on the first information and the second information of the corresponding each pixel in the feature image, thereby processing the to-be-processed image.