# US 2021/0343427 A1 · Systems and Methods for an Artificial Intelligence System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0343427 A1
- Document ID: US20210343427A1
- Status: Published application
- Kind: A1
- Application number: 17/305,830
- Filed: 2021-07-15
- Published: 2021-11-04
- Earliest claimed: 2020-02-06
- Assignees: HealthStream, Inc.
- Applicants: HealthStream, Inc.
- Inventors: Robert Frist, JR.
- Web page: https://patentgrep.com/patent/US20210343427A1

## CPC

- G06F 40/279: Recognition of textual entities
- G16H 70/40: relating to drugs, e.g. their side effects or intended usage
- G06F 40/205: Parsing
- G16H 40/63: for local operation
- G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems
- G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring
- H04W 4/14: Short messaging services, e.g. short message services [SMS] or unstructured supplementary service data [USSD]
- G16H 70/20: relating to practices or guidelines
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

A method is disclosed, which may include generating, in a natural language processing (NLP) system, a plurality of entity data objects. The method may include generating, in the NLP system, a plurality of activity data objects. The method may include generating, on at least one server, an evaluation data object. The evaluation data object may include a problem data object, an observation data object, or an action data object. The method may include configuring each problem data object, observation data object, or action data object of the evaluation data object with a scoring rubric. Other methods, systems, and computer-readable media are also disclosed.

## Claims

### Claim 1

A computer-implemented method, comprising:
receiving, at a server, a plurality of user responses from a user interface of a computing device, wherein the plurality of responses includes an action response,
sending the plurality of responses to a natural language processing (NLP) system;
receiving, from the NLP system, a computer-readable parsing based on a portion of the plurality of responses, wherein the computer-readable parsing includes a first activity key, and wherein the first activity key includes a first text string indicating at least one of
administering
notifying,
assisting, or
monitoring;
providing an evaluation data object, wherein the evaluation data object includes an action data object, wherein the action data object includes a second activity key and an action scoring rubric;
mapping the first activity key to the second activity key by determining whether the first activity key includes the second activity key;
calculating a first sub-score based on the action scoring rubric and whether the first activity key includes the second activity key, wherein the first sub-score indicates a clinical judgment evaluation of a user.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein the action response comprises a user action that responds to a diagnosis of a medical condition.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1, wherein the plurality of responses further includes a problem response that includes a diagnosis of a medical condition provided by the user.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the computer-readable parsing further includes a first entity key, and wherein the first entity key includes a second text string indicating the medical condition.

### Claim 5 (depends on 4)

The computer-implemented method of claim 4, wherein the evaluation data object further includes a problem data object, wherein the problem data object includes a second entity key and a problem scoring rubric.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5, further comprising:
mapping the first entity key to the second entity key by determining whether the first entity key includes the second entity key;
calculating a second sub-score based on the problem scoring rubric and whether the first entity key includes the second entity key; and
calculating, by combining the first sub-score and the second sub-score, a score for the user, wherein the score indicates the clinical judgment evaluation of the user.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein the computer-readable parsing further includes metadata, wherein the metadata includes data identifying a medical training scenario associated with the plurality of responses.

### Claim 8

A computer-implemented method, comprising:
receiving, at a server, a plurality of user responses from a user interface of a computing device, wherein the plurality of responses includes an action response,
sending the plurality of responses to a natural language processing (NLP) system;
receiving, from the NLP system, a computer-readable parsing based on a portion of the plurality of responses, wherein
the computer-readable parsing includes a first entity key and a first activity key,
the first entity key includes a first text string indicating at least one of
a medication,
a medical procedure,
a medical diagnostic test, or
a patient assessment,
the first activity key includes a second text string indicating at least one of
ambulating,
assessing,
assisting, or
ordering;
providing an evaluation data object, wherein the evaluation data object includes an action data object, wherein the action data object includes a second entity key, a second activity key, and an action scoring rubric;
mapping the first entity key to the second entity key by determining whether the first entity key includes the second entity key;
mapping the first activity key to the second activity key by determining whether the first activity key includes the second activity key;
calculating a first sub-score based on the action scoring rubric and whether the first entity key includes the second entity key;
calculating a second sub-score based on the action scoring rubric and whether the first activity key includes the second activity key,
calculating, by combining the first sub-score and the second sub-score, a score for a user, wherein the score indicates a clinical judgment evaluation of the user.

### Claim 9 (depends on 8)

The computer-implemented method of claim 8, wherein the action response comprises at least one of:
a medication that responds to a diagnosis of a medical condition;
a medical device that responds to the diagnosis of the medical condition; or
a medical supply that responds to the diagnosis of the medical condition.

### Claim 10 (depends on 8)

The computer-implemented method of claim 8, wherein the plurality of responses further includes a problem response that includes a diagnosis of a medical condition provided by the user.

### Claim 11 (depends on 10)

The computer-implemented method of claim 10, wherein the computer-readable parsing further includes a third entity key, and wherein the third entity key includes a text string indicating the medical condition.

### Claim 12 (depends on 11)

The computer-implemented method of claim 11, wherein the evaluation data object further includes a problem data object, wherein the problem data object includes a fourth entity key and a problem scoring rubric.

### Claim 13 (depends on 12)

The computer-implemented method of claim 12, further comprising:
mapping the third entity key to the fourth entity key by determining whether the third entity key includes the fourth entity key;
calculating a third sub-score based on the problem scoring rubric and whether the third entity key includes the fourth entity key; and
calculating, by combining the first sub-score, the second sub-score, and the third sub-score, the score for the user, wherein the score indicates the clinical judgment evaluation of the user.

### Claim 14 (depends on 8)

The computer-implemented method of claim 8, wherein the computer-readable parsing further includes metadata, wherein the metadata includes data identifying a medical training scenario associated with the plurality of responses.

### Claim 15

A computer-implemented method, comprising:
presenting, via a user interface of a computing device, a user with medical scenario information;
receiving, at a server, a plurality of user responses from the user, wherein the plurality of responses includes an action response,
sending the plurality of responses to a natural language processing (NLP) system;
receiving, from the NLP system, a computer-readable parsing based on a portion of the plurality of responses, wherein
the computer-readable parsing includes a first entity key and a first activity key,
the first entity key includes a first text string indicating at least one of
a medication,
a medical procedure,
a medical diagnostic test, or
a patient assessment,
the first activity key includes a second text string indicating at least one of
ambulating,
assessing,
assisting, or
ordering;
providing an evaluation data object, wherein the evaluation data object includes an action data object, wherein the action data object includes a second entity key, a second activity key, and an action scoring rubric;
mapping the first entity key to the second entity key by determining whether the first entity key includes the second entity key;
mapping the first activity key to the second activity key by determining whether the first activity key includes the second activity key;
calculating a first sub-score based on the action scoring rubric and whether the first entity key includes the second entity key;
calculating a second sub-score based on the action scoring rubric and whether the first activity key includes the second activity key,
calculating, by combining the first sub-score and the second sub-score, a score for a user, wherein the score indicates a clinical judgment evaluation of the user.

### Claim 16 (depends on 15)

The computer-implemented method of claim 15, wherein presenting the user with the medical scenario information comprises presenting, to the user, at least one of:
one or more medical diagnostic test values; or
one or more medical assessment findings.

### Claim 17 (depends on 15)

The computer-implemented method of claim 15, wherein the user interface comprises a text chat user interface, and wherein the text chat user interface is configured to receive text data from the user.

### Claim 18 (depends on 17)

The computer-implemented method of claim 17, further comprising presenting, via the text chat user interface, a server message, wherein the server message includes at least one of:
a prompting question to the user; or
a prompting statement to the user.

### Claim 19 (depends on 18)

The computer-implemented method of claim 18, wherein the server message includes at least one of:
text data,
image data,
video data, or
audio data.

### Claim 20 (depends on 15)

The computer-implemented method of claim 15, wherein the plurality of responses further includes a problem response that includes a diagnosis of a medical condition provided by the user.
