# US 11,076,824 B1 · Method and system for diagnosis of COVID-19 using artificial intelligence > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,076,824 B1 - Document ID: US11076824B1 - Status: Granted - Kind: B1 - Application number: 17/067,181 - Filed: 2020-10-09 - Granted: 2021-08-03 - Earliest claimed: 2020-08-07 - Assignees: Shenzhen Keya Medical Technology Corporation - Applicants: Shenzhen Keya Medical Technology Corporation - Inventors: Xin Wang; Youbing Yin; Bin Kong; Yi Lu; Junjie Bai; Zhenghan Fang; Qi Song - Web page: https://patentgrep.com/patent/US11076824B1 ## CPC - G06T 7/0012: {Biomedical image inspection} - A61B 5/08: Measuring devices for evaluating the respiratory organs - A61B 5/7267: {involving training the classification device} - A61B 5/7275: {Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor} - A61B 6/032: {Transmission computed tomography [CT]} - A61B 6/50: specially adapted for specific body parts; specially adapted for specific clinical applications - A61B 6/5217: {extracting a diagnostic or physiological parameter from medical diagnostic data} - G06N 3/04: Architecture, e.g. interconnection topology - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/0475: Generative networks - G06N 3/08: Learning methods - G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning - G06N 3/09: Supervised learning - G06N 3/094: Adversarial learning - G16H 10/60: for patient-specific data, e.g. for electronic patient records - G16H 30/40: for processing medical images, e.g. editing - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients - A61B 5/004: {adapted for image acquisition of a particular organ or body part (A61B5/0082 takes precedence; arrangements for optical scanning A61B5/0062)} - A61B 6/025: {Tomosynthesis} - A61B 6/469: {for selecting a region of interest [ROI]} - G06T 2207/10076: 4D tomography; Time-sequential 3D tomography - G06T 2207/10081: Computed x-ray tomography [CT] - G06T 2207/20081: Training; Learning - G06T 2207/20084: Artificial neural networks [ANN] - G06T 2207/30061: Lung - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract Embodiments of the disclosure provide methods and systems for detecting COVID-19 from a lung image of a patient. The exemplary system may include a communication interface configured to receive the lung image acquired of the patient's lung by an image acquisition device. The system may further include at least one processor, configured to determine a region of interest comprising the lung from the lung image and apply a COVID-19 detection network to the region of interest to determine a condition of the lung. The COVID-19 detection network is a multi-class classification learning network that labels the region of interest as one of COVID-19, non-COVID-19 pneumonia, non-pneumonia abnormal, or normal. The at least one processor is further configured to provide a diagnostic output based on the determined condition of the lung. ## Claims ### Claim 1 A system for detecting COVID-19 from a lung image of a patient, comprising: a communication interface configured to receive the lung image acquired of a lung of the patient by an image acquisition device, wherein the lung image is a 3D CT image; and at least one processor, configured to: determine a 3D region of interest comprising the lung from the 3D CT image; apply a COVID-19 detection network to the 3D region of interest to determine a condition of the patient's lung, wherein the COVID-19 detection network is a multi-class classification learning network that labels the region of interest as one of COVID-19, non-COVID-19 pneumonia, non-pneumonia abnormal, or normal, wherein the COVID-19 detection network is a 3D deep learning network; and provide a diagnostic output based on the determined condition of the lung, wherein to apply the 3D deep learning network to the 3D region of interest, the at least one processor is further configured to: obtain feature maps corresponding to the 3D region of interest and obtain the diagnostic output from the feature maps by applying a combination network comprising a convolutional neural network (CNN) and another neural network. ### Claim 2 (depends on 1) The system of claim 1, wherein to obtain feature maps corresponding to the 3D region of interest, the at least one processor is further configured to: obtain feature maps corresponding to a plurality of 2D slices of the 3D region of interest by applying combination convolutional neural network-recurrent neural networks (CNN-RNNs) to the respective 2D slices in parallel. ### Claim 3 (depends on 2) The system of claim 2, wherein applying the combination CNN-RNN network to each 2D slice comprises applying a CNN and a recurrent neural network (RNN) sequentially to the 2D slice. ### Claim 4 (depends on 3) The system of claim 3, wherein the RNNs in the combination CNN-RNN networks are configured to exchange information bidirectionally to capture global features across the plurality of 2D slices. ### Claim 5 (depends on 2) The system of claim 2, wherein the at least one processor is further configured to: combine the feature maps corresponding to the plurality of 2D slices to determine a composite feature map; and apply a fully connected layer to the composite feature map. ### Claim 6 (depends on 1) The system of claim 1, wherein the CNN network comprises a 3D CNN and the another neural network comprises a fully connected layer, and to apply the 3D deep learning network to the 3D region of interest, the at least one processor is further configured to: obtain the feature maps by applying the 3D CNN to the 3D region of interest; and apply the fully connected layer to the feature maps. ### Claim 7 (depends on 1) The system of claim 1, wherein the at least one processor is further configured to generate a probability score for the determined condition. ### Claim 8 (depends on 1) The system of claim 1, wherein the at least one processor is further configured to generate a heatmap of the region of interest indicative of lung characteristics that lead to the determination of the condition by the COVID-19 detection network. ### Claim 9 (depends on 1) The system of claim 1, wherein the COVID-19 detection network is trained using sample lung images and known conditions associated with the sample lung images. ### Claim 10 (depends on 1) The system of claim 1, wherein the communication interface is further configured to receive information of the patient including at least one of a demography, a disease history, and a laboratory testing result of the patient, wherein the received information is stored as meta data associated with the 3D CT image, wherein the at least one processor is further configured to apply the COVID-19 detection network additionally to the received information of the patient in order to determine the condition of the lung. ### Claim 11 A method for detecting COVID-19 from a lung image of a patient, comprising: receiving, at a communication interface, the lung image acquired of a lung of the patient by an image acquisition device, wherein the lung image is a 3D CT image; determining, by at least one processor, a 3D region of interest comprising the lung from the 3D CT lung image; applying, by the at least one processor, a COVID-19 detection network to the 3D region of interest to determine a condition of the lung, wherein the COVID-19 detection network is a multi-class classification learning network that labels the region of interest as one of COVID-19, non-COVID-19 pneumonia, non-pneumonia abnormal, or normal, wherein the COVID-19 detection network is a 3D deep learning network; and providing, by the at least one processor, a diagnostic output based on the determined condition of the lung, wherein applying the 3D deep learning network to the 3D region of interest further comprises: obtaining feature maps corresponding to the 3D region of interest and obtaining the diagnostic output from the feature maps by applying a combination network comprising a convolutional neural network (CNN) and another neural network. ### Claim 12 (depends on 11) The method of claim 11, wherein the obtaining feature maps corresponding to the 3D region of interest further comprises: obtaining feature maps corresponding to a plurality of 2D slices of the 3D region of interest by applying combination convolutional neural network-recurrent neural networks (CNN-RNNs) to the respective 2D slices in parallel. ### Claim 13 (depends on 12) The method of claim 12, wherein applying the combination CNN-RNN network to each 2D slice further comprises: applying a CNN and a recurrent neural network (RNN) sequentially to the 2D slice. ### Claim 14 (depends on 13) The method of claim 13, further comprising: exchanging information bidirectionally to capture global features across the plurality of 2D slices by the RNNs in the combination CNN-RNN networks. ### Claim 15 (depends on 11) The method of claim 11, further comprising: combining the feature maps corresponding to the plurality of 2D slices to determine a composite feature map; and applying a fully connected layer to the composite feature map. ### Claim 16 (depends on 11) The method of claim 11, wherein the CNN network comprises a 3D CNN and the another neural network comprises a fully connected layer, and the applying the COVID-19 detection network to the 3D region of interest further comprises: obtaining the feature maps by applying the 3D CNN to the 3D region of interest; and applying the fully connected layer to the feature maps. ### Claim 17 (depends on 11) The method of claim 11, further comprising: generating a probability score for the determined condition; and generating a heatmap of the region of interest indicative of lung characteristics that lead to the determination of the condition by the COVID-19 detection network. ### Claim 18 A non-transitory computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by at least one processor, performs a method for detecting COVID-19 from a lung image of a patient, the method comprising: receiving the lung image acquired of a lung of the patient by an image acquisition device wherein the lung image is a 3D CT image; determining a 3D region of interest comprising the lung from the 3D CT image; applying a COVID-19 detection network to the 3D region of interest to determine a condition of the lung, wherein the COVID-19 detection network is a multi-class classification learning network that labels the region of interest as one of COVID-19, non-COVID-19 pneumonia, non-pneumonia abnormal, or normal, wherein the COVID-19 detection network is a 3D deep learning network; and providing a diagnostic output based on the determined condition of the lung, wherein applying the 3D deep learning network to the 3D region of interest further comprises: obtaining feature maps corresponding to the 3D region of interest and obtaining the diagnostic output from the feature maps by applying a combination network comprising a convolutional neural network (CNN) and another neural network.