# US 2025/0095642 A1 · Natural language based clinical recommendation system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0095642 A1
- Document ID: US20250095642A1
- Status: Published application
- Kind: A1
- Application number: 18/468,593
- Filed: 2023-09-15
- Published: 2025-03-20
- Earliest claimed: 2023-09-15
- Assignees: GE Precision Healthcare LLC
- Applicants: GE Precision Healthcare LLC
- Inventors: Soumya Ghose; Sanand Sasidharan; Sanghee Cho; Akshit Achara; Rakesh Mullick; Anuradha Kanamarlapudi; Fiona Ginty; Annamraju Ravi Bhardwaj; Sundararajan Mani; Brion Sarachan
- Web page: https://patentgrep.com/patent/US20250095642A1

## CPC

- G10L 15/197: Probabilistic grammars, e.g. word n-grams
- G06N 3/006: based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/045: Combinations of networks
- G06N 3/0455: Auto-encoder networks; Encoder-decoder networks
- G06N 3/047: Probabilistic or stochastic networks
- G06N 3/09: Supervised learning
- G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks
- G10L 15/063: {Training}
- G10L 15/16: using artificial neural networks
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- G16H 10/20: for electronic clinical trials or questionnaires
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G16H 40/20: for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients
- G16H 70/20: relating to practices or guidelines
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

The current disclosure provides methods for an automated clinical recommendation system that generates clinical recommendations for patients of a health care system based on natural language queries submitted by care providers of the health care system. The natural language queries may be typed into a user interface (UI) of the clinical recommendation system, or submitted by voice via a microphone. The clinical recommendation system provides a clinically explainable disease state of a patient based on patient data included in a query, and recommends a next course of action (e.g., a treatment) based on clinical guidelines and population statistics, in a manner that reduces a current burden of clinicians in consulting digital clinical manuals via a series of time-consuming and cumbersome interactions with a graphical user interface (GUI) of the digital clinical manuals.

## Claims

### Claim 1

A method for a clinical recommendation system, the method comprising:
generating an automated response to a natural language query received from a care provider in real time using a first, large language model (LLM) of the clinical recommendation system, the automated response including a recommended treatment of a patient, the LLM trained on a first set of training data generated by applying a second, recommendation model to anonymized patient data, the recommendation model based on clinical guidelines;
verifying the automated response outputted by the LLM using a third, prediction model; and
in response to the automated response being verified, storing the automated response in a patient decision workflow of the patient, the patient decision workflow including a record of clinical recommendations made with respect to the patient using the clinical recommendation system, and displaying the automated response on a display device of the clinical recommendation system.

### Claim 2 (depends on 1)

The method of claim 1, wherein the recommendation model includes one of a decision tree and a knowledge graph.

### Claim 3 (depends on 1)

The method of claim 1, wherein the prediction model includes one of a Bayesian network and a convolutional neural network (CNN).

### Claim 4 (depends on 1)

The method of claim 1, wherein the prediction model is trained on a second set of training data generated by applying the recommendation model to the anonymized patient data.

### Claim 5 (depends on 1)

The method of claim 1, wherein verifying the automated response outputted by the LLM using the prediction model further comprises:
extracting the patient data from the natural language query;
extracting the recommended treatment from the automated response outputted by the LLM;
determining a second treatment with a highest probability of being selected based on the extracted patient data, using the prediction model;
in response to the recommended treatment matching the second treatment, verifying the automated response; and
in response to the recommended treatment not matching the second treatment, reporting an inconsistency between the recommended treatment and the clinical guidelines, the clinical guidelines including the National Comprehensive Cancer Network (NCCN) Clinical Practice Guidelines in Oncology (NCCN Guidelines®) or equivalent local guidelines.

### Claim 6 (depends on 1)

The method of claim 1, wherein the recommended treatment is based on a previous recommended treatment stored in a patient decision workflow of a previous patient.

### Claim 7 (depends on 1)

The method of claim 1, wherein verifying the automated response outputted by the LLM further comprises comparing a recommended treatment of the automated response with a previous recommended treatment of a previous patient stored in a patient decision workflow of the previous patient.

### Claim 8 (depends on 1)

The method of claim 1, further comprising:
receiving patient outcome data corresponding to the patient from an electronic medical record (EMR); and
using the patient outcome data to further train the prediction model.

### Claim 9 (depends on 1)

The method of claim 1, wherein training the LLM on the first set of training data further comprises:
training the LLM on a general-domain NLP dataset;
fine-tuning the LLM by training the LLM on natural language training pairs using supervised learning, the natural language training pairs including a prompt and a ground truth response to the prompt; and
optimizing the LLM by training the LLM on various corpora of unlabeled data, using reinforcement learning.

### Claim 10 (depends on 9)

The method of claim 9, wherein the natural language training pairs are generated by at least one of human experts using the recommendation model as a guide, and one or more chat-bots that traverse the recommendation model.

### Claim 11 (depends on 10)

The method of claim 10, wherein fine-tuning the LLM by training the LLM on natural language training pairs using supervised learning further comprises training the LLM on a first set of training pairs generated manually by the human experts, and then training the LLM on a second set of training pairs generated by the one or more chat-bots.

### Claim 12 (depends on 10)

The method of claim 10, wherein optimizing the LLM by training the LLM on various corpora of unlabeled data using reinforcement learning further comprises:
inputting a set of generated natural language training prompts into the fine-tuned LLM a plurality of times to generate a plurality of alternative outputs;
receiving a ranking of the plurality of alternative outputs by the human experts;
training a reward model on the generated natural language training prompts and the alternative outputs using the rankings as ground truth data; and
training the LLM using reinforcement learning based on the reward model and a reinforcement learning algorithm.

### Claim 13 (depends on 1)

The method of claim 1, wherein the natural language query is received from the care provider by voice via a microphone, and/or the response is delivered to the care provider via an audio signal delivered over a speaker.

### Claim 14

A clinical recommendation system, comprising a processor and a non-transitory memory storing instructions that when executed, cause the processor to:
receive a natural language query from a care provider via a user interface of the clinical recommendation system, the natural language query including patient data of a patient;
generate a response to the natural language query using a large language model (LLM), the response including a recommended treatment based on the patient data;
store the response in a patient decision workflow of the patient; and
deliver the response to the care provider via a display device and/or a speaker of the clinical recommendation system; wherein
the LLM is trained using supervised learning on human- or chat-bot-generated natural language training pairs generated in accordance with a recommendation model based on clinical guidelines, and optimized by training the LLM on various corpora of unlabeled data using reinforcement learning based on a reward model trained on human-ranked training data.

### Claim 15 (depends on 14)

The clinical recommendation system of claim 14, wherein the recommendation model is one of a decision tree and a knowledge graph including decision points where different treatments are recommended based on the patient data.

### Claim 16 (depends on 14)

The clinical recommendation system of claim 14, wherein further instructions are stored in the non-transitory memory that, when executed, cause the processor to:
verify the response using a prediction model trained to take the patient data as input, and output a treatment that has a highest probability of being selected based on population statistics and/or historical patient data, using the recommendation model as a guide.

### Claim 17 (depends on 16)

The clinical recommendation system of claim 16, wherein verifying the response using the prediction model comprises:
extracting the patient data from the natural language query;
extracting a first treatment from the response outputted by the LLM;
determining a second treatment with the highest probability of being selected based on the extracted patient data, using the prediction model;
in response to the first treatment matching the second treatment, verifying the response; and
in response to the first treatment matching the second treatment, reporting an inconsistency between the recommended treatment and the clinical guidelines.

### Claim 18

A method for providing clinical guidance to care providers via a clinical recommendation system, the method comprising:
receiving a query from a care provider, the query including data of a patient;
translating the query into a natural language prompt;
submitting the natural language prompt to a large language model (LLM), the LLM trained to output a response with a recommended treatment for the patient based on the patient data and one or more sets of digital clinical guidelines;
extracting the patient data from the natural language prompt;
predicting a most probable treatment for the patient based on the extracted patient data, using a prediction model trained on population statistics and anonymized historical patient data and/or prior recommended treatments;
extracting the recommended treatment included from the response generated by the LLM;
comparing the recommended treatment with the predicted most probable treatment;
in response to the recommended treatment matching the predicted most probable treatment, storing the recommended treatment in a patient decision workflow of the patient and delivering the response to the care provider via a display device and/or a speaker of the clinical recommendation system; and
in response to the recommended treatment not matching the predicted most probable treatment, logging an inconsistency of the recommended treatment with a current practice.

### Claim 19 (depends on 18)

The method of claim 18, wherein the training of the LLM includes:
fine-tuning the LLM by training the LLM on natural language training pairs using supervised learning, each natural language training pair including a prompt and a ground truth response to the prompt generated by at least one of a human expert and a chat-bot using a recommendation model trained on the one or more sets of digital clinical guidelines as a guide; and
optimizing the LLM by training the LLM on various corpora of unlabeled data, using reinforcement learning, based on a reward model trained on human-ranked responses to a selected set of generated prompts.

### Claim 20 (depends on 19)

The method of claim 19, wherein the recommendation model includes a decision tree based on a set of digital clinical guidelines, the recommendation model taking patient data as input, traversing the decision tree in accordance with the patient data, and outputting a recommended treatment at a decision point of the decision tree.