# US 2023/0055094 A1 · Capturing Detailed Structure from Patient-Doctor Conversations for Use in Clinical Documentation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0055094 A1
- Document ID: US20230055094A1
- Status: Published application
- Kind: A1
- Application number: 17/978,790
- Filed: 2022-11-01
- Published: 2023-02-23
- Earliest claimed: 2019-11-12
- Assignees: Google Inc.
- Applicants: Google LLC
- Inventors: Christopher Co; Gang Li; Philip Chung; Justin Paul; Daniel Shing Shun Tse; Katherine Chou; Diana Jaunzeikare; Alvin Rajkomar
- Web page: https://patentgrep.com/patent/US20230055094A1

## CPC

- G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof
- G06N 20/00: Machine learning
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G16H 70/20: relating to practices or guidelines
- H04M 11/10: with dictation recording and playback systems
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- H04M: TELEPHONIC COMMUNICATION (circuits for controlling other apparatus via a telephone cable and not involving telephone switching apparatus G08)

## Abstract

A method and system is provided for assisting a user to assign a label to words or spans of text in a transcript of a conversation between a patient and a medical professional and form groupings of such labelled words or spans of text in the transcript. The transcript is displayed on an interface of a workstation. A tool is provided for highlighting spans of text in the transcript consisting of one or more words. Another tool is provided for assigning a label to the highlighted spans of text. This tool includes a feature enabling searching through a set of predefined labels available for assignment to the highlighted span of text. The predefined labels encode medical entities and attributes of the medical entities. The interface further includes a tool for creating groupings of related highlighted spans of texts. The tools can consist of mouse action or keystrokes or a combination thereof.

## Claims

### Claim 1

A computer-implemented method comprising:
receiving a training dataset comprising a plurality of labeled transcripts of respective medical practitioner-patient conversations, each labeled transcript having been generated by a user interaction with the transcript, the user interaction having comprised of a user relating two different highlighted spans of text by utilizing an input device of a computing device, and the computing device having been configured to interpret the user interaction to be an indication that the two user-related different highlighted spans of text are medically related to a same health condition of the patient;
training, based on the training dataset, a machine learning model to automatically annotate an input transcript of a recording of a medical conversation; and
outputting the trained machine learning model.

### Claim 2 (depends on 1)

The method of claim 1, wherein the transcript of the recording is indexed to time segment information.

### Claim 3 (depends on 1)

The method of claim 1, wherein the automatic annotation of the input transcript comprises storing the annotation of the input transcript as metadata associated with the input transcript.

### Claim 4 (depends on 1)

The method of claim 1, wherein each highlighted span of text in the transcript is associated with a label encoding a medical entity and one or more attributes of the medical entity appearing in the highlighted span, wherein medical entities indicate categories of medical topics, and wherein medical attributes indicate descriptive properties or characteristics of an associated medical entity.

### Claim 5 (depends on 4)

The method of claim 4, wherein the medical entity comprises one or more of medications, procedures, symptoms, vitals, lab results, chief complaint, social history, medical conditions, surgery, imaging, provider, vaccine, reproductive history, examination, and medical equipment.

### Claim 6 (depends on 5)

The method of claim 5, wherein the medical entity is a symptom medical entity comprising an attribute of at least severity, frequency, onset, or location of a medical condition.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
generating the training dataset by providing a plurality of unlabeled transcripts to a pre-labeling system and receiving, from the pre-labeling system a pre-annotated transcript containing suggested labels for spans of text in each of the plurality of unlabeled transcripts.

### Claim 8 (depends on 7)

The method of claim 7, wherein the pre-labeling system includes a named entity recognition model trained on at least one of medical textbooks, a lexicon of clinical terms, clinical documentation in electronic health records, and annotated transcripts of doctor-patient conversations.

### Claim 9 (depends on 1)

The method of claim 1, further comprising:
providing the annotated version of the input transcript to another computing device for another user interaction with the annotated version, wherein the other user interaction comprises modifications including one or more additional annotations of the annotated version;
receiving, from the other computing device, the modified version of the annotated transcript; and
fine-tuning the trained machine learning model based on the modified version.

### Claim 10

A computer-implemented method comprising:
receiving an input transcript of a recording of a medical practitioner-patient conversation;
applying a trained machine learning model to automatically annotate the input transcript, the machine learning model having been trained on a training dataset comprising a plurality of labeled transcripts of respective medical conversations, each labeled transcript having been generated by a user interaction with the transcript, the user interaction having comprised of a user relating two different highlighted spans of text by utilizing an input device of a computing device, and the computing device having been configured to interpret the user interaction to be an indication that the two user-related different highlighted spans of text are medically related to a same health condition of the patient; and
providing the annotated version of the input transcript.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
generating the input transcript of the recording.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
generating a display of the input transcript of the recording.

### Claim 13 (depends on 10)

The method of claim 10, wherein the input transcript is indexed to time segment information.

### Claim 14 (depends on 10)

The method of claim 10, wherein the automatic annotation of the input transcript comprises storing the annotation of the input transcript as metadata associated with the input transcript.

### Claim 15 (depends on 10)

The method of claim 10, wherein the providing of the annotated version comprises associating each highlighted span of text in the input transcript with a label encoding a medical entity and one or more attributes of the medical entity appearing in the highlighted span, wherein medical entities indicate categories of medical topics, and wherein medical attributes indicate descriptive properties or characteristics of an associated medical entity.

### Claim 16 (depends on 15)

The method of claim 15, wherein the medical entity comprises one or more of medications, procedures, symptoms, vitals, lab results, chief complaint, social history, medical conditions, surgery, imaging, provider, vaccine, reproductive history, examination, and medical equipment.

### Claim 17 (depends on 15)

The method of claim 15, wherein the medical entity is a symptom medical entity comprising an attribute of at least severity, frequency, onset, or location of a medical condition.

### Claim 18

A system comprising:
a computing device; and
data storage, wherein the data storage has stored thereon computer-executable instructions that, when executed by the one or more processors, cause the computing device to carry out functions comprising:
receiving an input transcript of a recording of a medical practitioner-patient conversation;
applying a trained machine learning model to automatically annotate the input transcript, the machine learning model having been trained on a training dataset comprising a plurality of labeled transcripts of respective medical conversations, each labeled transcript having been generated by a user interaction with the transcript, the user interaction having comprised of a user relating two different highlighted spans of text by utilizing an input device of a computing device, and the computing device having been configured to interpret the user interaction to be an indication that the two user-related different highlighted spans of text are medically related to a same health condition of the patient; and
providing the annotated version of the input transcript.

### Claim 19 (depends on 18)

The system of claim 18, further comprising:
generating the input transcript of the recording.

### Claim 20 (depends on 18)

The system of claim 18, wherein the providing of the annotated version comprises associating each highlighted span of text in the input transcript with a label encoding a medical entity and one or more attributes of the medical entity appearing in the highlighted span, wherein medical entities indicate categories of medical topics, and wherein medical attributes indicate descriptive properties or characteristics of an associated medical entity.
