# US 2021/0343398 A1 · Method and systems for predicting medical conditions and forecasting rate of infection of medical conditions via artificial intellidence models using graph stream processors

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0343398 A1
- Document ID: US20210343398A1
- Status: Published application
- Kind: A1
- Application number: 17/238,289
- Filed: 2021-04-23
- Published: 2021-11-04
- Earliest claimed: 2020-05-02
- Assignees: Blaize, Inc.
- Applicants: Blaize, Inc.
- Inventors: Deepak Chandra Bijalwan; Dinakar C. Munagala
- Web page: https://patentgrep.com/patent/US20210343398A1

## CPC

- G06T 7/0012: {Biomedical image inspection}
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/2163: {Partitioning the feature space}
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/0495: Quantised networks; Sparse networks; Compressed networks
- G06N 3/08: Learning methods
- G06N 3/09: Supervised learning
- G06T 5/70: Denoising; Smoothing
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/776: Validation; Performance evaluation
- G06V 10/82: using neural networks
- G16H 30/40: for processing medical images, e.g. editing
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G16H 50/80: for detecting, monitoring or modelling epidemics or pandemics, e.g. flu
- G06T 2207/10081: Computed x-ray tomography [CT]
- G06T 2207/10116: X-ray image
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30061: Lung
- G06V 2201/03: Recognition of patterns in medical or anatomical images
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

Systems and methods are disclosed for predicting one or more medical conditions utilizing digital images and employing artificial intelligent algorithms. The system offers accurate predictions utilizing quantized pre-trained deep learning model. The pre-trained deep learning model is trained on data samples and later refined as the system processes more digital images or new medical conditions are incorporated. One pre-trained deep learning model is used to predict the probability of one or more medical conditions and identify locations in the digital image effected by the one or more medical conditions. Further, one pre-trained deep learning model utilizing additional data and plurality of digital images, forecasts rate of infection and spread of the medical condition over time.

## Claims

### Claim 1

A computer implemented image processing method for predicting one or more medical conditions, the method comprising:
receiving a digital image of at least portion of a user;
pre-processing the digital image by removing noise, aligning, and correcting luminance of the digital image;
generating an initial prediction, wherein the initial prediction comprises a probability of the one or more medical conditions by processing the pre-processed image through_pre-trained weights of the pre-trained deep learning model; and
predicting an output based on the initial prediction and further processing through the pre-trained deep learning model, where in the predicting includes,
modifying the probability of the one or more medical conditions by labelling the preprocessed image from the one or more medical conditions and computing the probability of the one or more medical conditions; and
identifying at least a location in the digital image effected by the one or more medical conditions, wherein the identifying includes generating a plurality of anchor boxes and estimating at least one final anchor box.

### Claim 2 (depends on 1)

The method of claim 1, where in the predicting further comprises receiving a plurality of digital images over a period of time and forecasting a rate of spread of the one or more medical conditions and spread of the one or more medical conditions over time based on additional medical data.

### Claim 3 (depends on 1)

The method of claim 1, where in generating the pre-trained deep learning model comprises generating a floating-point model utilizing a plurality of data samples and compacting the floating-point model.

### Claim 4 (depends on 1)

The method of claim 1, where in generating the pre-trained deep learning model comprises generating a floating-point model utilizing a plurality of data samples and quantizing the floating-point model to generate a mixed precision quantized model.

### Claim 5 (depends on 3)

The method of claim 3, further comprising verifying by an expert user, by comparing the predicted output with the digital image and including the digital image in the plurality of data samples when the predicted output is incorrect.

### Claim 6 (depends on 1)

The method of claim 1, wherein the digital image type comprises any one of X-Ray, CT-Scan, and non-medical image.

### Claim 7 (depends on 6)

The method of claim 6, further utilizing the X-Ray image for generating the initial prediction and discarding other types of digital images.

### Claim 8 (depends on 1)

The method of claim 1, wherein the one or more medical conditions comprises Covid-19.

### Claim 9 (depends on 3)

The method of claim 3, wherein the plurality of data samples comprises positive data samples and negative data samples, wherein the positive data samples comprises a plurality of data samples of a first type, and negative samples comprises a plurality of data samples in other type, wherein the plurality of data samples of the first type are used to generate the pre-trained deep learning model and negative samples are discarded.

### Claim 10 (depends on 9)

The method of claim 9, further comprising labelling the positive samples with the one or more medical conditions, aligning images of the positive data samples and data balancing the plurality of data samples.

### Claim 11 (depends on 10)

The method of claim 10, further comprising dividing the balanced data samples into training data samples for generating the pre-trained deep learning model, test data samples for testing the pre-trained deep learning model and validation data samples for validation of the pre-trained deep learning model.

### Claim 12 (depends on 11)

The method of claim 11, further comprising the validating the pre-trained deep learning model by processing the validation data samples through the pre-trained deep learning model and comparing the predicted output with the positive samples.

### Claim 13 (depends on 11)

The method of claim 11, further comprising, utilizing test data samples to test for system randomness.

### Claim 14 (depends on 1)

The method of claim 1, further comprises generating the pre-trained deep learning model, where in the generating includes:
pre-processing the plurality of training data samples to reduce noise, adjust illumination and image alignment to generate pre-processed training data samples;
extracting textural features from the pre-processed training data samples;
generating a map of the textual features; and
processing through a decision network to train the pre-trained deep learning model to the predict the output.

### Claim 15 (depends on 3)

The method of claim 3, further comprising receiving a plurality of training data samples of second medical condition, updating the floating-point model utilizing the plurality of training data samples of the second medical condition, and distilling the floating point model to update the pre-trained deep learning model.

### Claim 16 (depends on 3)

The method of claim 3, further comprising receiving a plurality of new data samples, updating the floating-point model utilizing the plurality of new data samples, and distilling the floating point model to update the pre-trained deep learning model.

### Claim 17

A graph stream processing system, comprising:
an inference module operative to execute instructions on a plurality of graph streaming processors (GSP) to process a digital image associated with a patient, generating an initial prediction utilizing pre-trained weights in a mixed precision deep learning model; and
a post-processing module operative to execute instructions on the GSP to compute a prediction comprising utilizing the mixed precision deep learning model:
probability of a medical condition of the patient based on the initial prediction, wherein the computing includes labelling the digital image from the medical condition,
identifying a location of the medical condition in the digital image wherein the identifying comprises generating a plurality of anchor boxes and estimating at least one final anchor that indicates the medical condition, and
forecasting a rate of spread of the medical condition and spread of the medical condition over time based on a plurality of digital images received over time, the initial prediction and additional medical data.

### Claim 18 (depends on 17)

The system of claim 17, where the mixed precision pre-trained deep learning model is generated by computing a floating-point model utilizing a plurality of data samples, compacting the floating-point model and quantizing the compacted floating-point model.

### Claim 19 (depends on 18)

The system of claim 18, further comprising verifying by an expert user, by comparing the prediction with the digital image and including the digital image with the plurality of data samples when the predicted output is incorrect.

### Claim 20

A data processing method comprising:
receiving a digital description of at least a symptom of one or more medical conditions in a first format;
performing, using a plurality of GSP, an (AI based) initial prediction of the one or more medical conditions by processing the digital description through a pre-trained deep learning model to generate a probability of the one or more medical conditions comprising of Covid-19;
determining using the plurality of GSP,
adjusting the probability of the one or more medical conditions,
identifying a location in the description of the one or more medical conditions, and
forecasting a rate of spread of the one or more medical conditions and a spread of the one or more medical conditions over time based on a plurality of digital descriptions and additional medical data.
