# US 8,788,289 B2 · Methods and apparatus for linking extracted clinical facts to text

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 8,788,289 B2
- Document ID: US8788289B2
- Status: Granted
- Kind: B2
- Application number: 13/030,981
- Filed: 2011-02-18
- Published: 2012-08-23
- Granted: 2014-07-22
- Earliest claimed: 2011-02-18
- Assignees: Nuance Communications, Inc.
- Applicants: James R. Flanagan; Davide Zaccagnini; Frank Montyne; David Decraene; David Hellman; Matthew R. Shelton; Mariana Casella dos Santos; Karen Anne Doyle; Johan Raedemaeker; Joeri Van der Vloet; Isam Habboush; Anthony J. Elcocks; Anush Hartunian
- Inventors: James R. Flanagan; Davide Zaccagnini; Frank Montyne; David Decraene; David Hellman; Matthew R. Shelton; Mariana Casella dos Santos; Karen Anne Doyle; Johan Raedemaeker; Joeri Van der Vloet; Isam Habboush; Anthony J. Elcocks; Anush Hartunian
- Web page: https://patentgrep.com/patent/US8788289B2

## CPC

- G16H 10/20: for electronic clinical trials or questionnaires
- G06Q 10/10: Office automation; Time management
- G10L 15/1815: {Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning}
- G10L 15/26: Speech to text systems (G10L15/08 takes precedence)
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING

## Abstract

A plurality of clinical facts may be extracted from a free-form narration of a patient encounter provided by a clinician. The plurality of clinical facts may include a first fact and a second fact. The first fact may be extracted from a first portion of the free-form narration, and the second fact may be extracted from a second portion of the free-form narration. A first indicator that indicates a first linkage between the first fact and the first portion of the free-form narration may be provided to a user. A second indicator, different from the first indicator, that indicates a second linkage between the second fact and the second portion of the free-form narration may also be provided to the user.

## Claims

### Claim 1

A method comprising:
extracting, using at least one processor, a plurality of clinical facts from a free-form narration of a patient encounter provided by a clinician, wherein the plurality of clinical facts comprises a first fact and a second fact, wherein the first fact is extracted from a first portion of the free-form narration, and wherein the second fact is extracted from a second portion of the free-form narration, the extracting comprising
analyzing the free-form narration to identify one or more sets of features of at least the first and second portions of the free-form narration,
correlating the one or more sets of features to a plurality of abstract semantic concepts, and
generating computer-readable data that expresses the plurality of abstract semantic concepts as the plurality of clinical facts extracted from the free-form narration; and
providing to a user a first indicator that indicates a first linkage between the first fact and the first portion of the free-form narration, wherein providing the first indicator comprises:
in response to the user selecting the first fact via a user interface, providing to the user an indicator that distinguishes the first portion of the free-form narration from which the first fact was extracted from other portions of the free-form narration from which the first fact was not extracted; and
providing to the user a second indicator, different from the first indicator, that indicates a second linkage between the second fact and the second portion of the free-form narration.

### Claim 2 (depends on 1)

The method of claim 1, wherein the first indicator comprises a visual indicator.

### Claim 3 (depends on 2)

The method of claim 2, wherein the first indicator indicates the first linkage using a different textual and/or graphic style than the second indicator uses to indicate the second linkage.

### Claim 4 (depends on 3)

The method of claim 3, wherein the first indicator indicates the first linkage using a first color, and the second indicator indicates the second linkage using a second color, wherein the first color is different from the second color.

### Claim 5 (depends on 2)

The method of claim 2, wherein providing the first indicator comprises:
in response to a selection by the user of one of the first fact and a textual representation of the first portion of the free-form narration, visually identifying the other of the first fact and the textual representation of the first portion of the free-form narration.

### Claim 6 (depends on 2)

The method of claim 2, wherein providing the first indicator comprises:
in response to a selection by the user of one of the first fact and a textual representation of the first portion of the free-form narration, displaying a graphical connector between the first fact and the textual representation of the first portion of the free-form narration.

### Claim 7 (depends on 2)

The method of claim 2, wherein providing the first indicator comprises:
in response to a selection by the user of the first fact, scrolling a display of a textual representation of the free-form narration to a textual representation of the first portion of the free-form narration.

### Claim 8 (depends on 1)

The method of claim 1, wherein the first indicator comprises an audio indicator.

### Claim 9 (depends on 8)

The method of claim 8, wherein providing the first indicator comprises:
in response to a selection by the user of the first fact, playing an audio version of the first portion of the free-form narration.

### Claim 10 (depends on 9)

The method of claim 9, wherein the audio version of the first portion of the free-form narration comprises a portion of an audio recording of the clinician dictating the free-form narration.

### Claim 11 (depends on 9)

The method of claim 9, wherein the audio version of the first portion of the free-form narration is generated from a textual representation of the free-form narration using automatic speech synthesis.

### Claim 12 (depends on 1)

The method of claim 1, further comprising automatically editing one of the first fact and a data representation of the first portion of the free-form narration in response to the user editing the other of the first fact and the data representation of the first portion of the free-form narration.

### Claim 13 (depends on 1)

The method of claim 1, further comprising:
in response to the user selecting a third portion of the free-form narration, extracting a third fact from the third portion of the free-form narration.

### Claim 14 (depends on 1)

The method of claim 1, further comprising formatting a textual representation of the free-form narration to produce a formatted text, wherein the plurality of clinical facts are extracted from the formatted text.

### Claim 15 (depends on 14)

The method of claim 14, wherein the formatting is performed in a way that facilitates the extracting of the plurality of clinical facts.

### Claim 16 (depends on 1)

The method of claim 1, further comprising:
including the first fact in a patient record; and
indicating a linkage between the first fact in the patient record and the first portion of the free-form narration.

### Claim 17 (depends on 16)

The method of claim 16, wherein the indicating comprises:
simultaneously displaying at least a portion of the patient record and a textual representation of at least the first portion of the free-form narration; and
visually indicating the linkage between the first fact in the patient record and a textual representation of the first portion of the free-form narration.

### Claim 18 (depends on 1)

The method of claim 1, wherein the user is the clinician.

### Claim 19 (depends on 1)

The method of claim 1, wherein the user is a person other than the clinician.

### Claim 20

Apparatus comprising:
at least one processor; and
a memory storing processor-executable instructions that, when executed by the at least one processor, perform a method comprising:
extracting a plurality of clinical facts from a free-form narration of a patient encounter provided by a clinician, wherein the plurality of clinical facts comprises a first fact and a second fact, wherein the first fact is extracted from a first portion of the free-form narration, and wherein the second fact is extracted from a second portion of the free-form narration, the extracting comprising
analyzing the free-form narration to identify one or more sets of features of at least the first and second portions of the free-form narration,
correlating the one or more sets of features to a plurality of abstract semantic concepts, and
generating computer-readable data that expresses the plurality of abstract semantic concepts as the plurality of clinical facts extracted from the free-form narration; and
providing to a user a first indicator that indicates a first linkage between the first fact and the first portion of the free-form narration, wherein providing the first indicator comprises:
in response to the user selecting the first fact via a user interface, providing to the user an indicator that distinguishes the first portion of the free-form narration from which the first fact was extracted from other portions of the free-form narration from which the first fact was not extracted; and
providing to the user a second indicator, different from the first indicator, that indicates a second linkage between the second fact and the second portion of the free-form narration.

### Claim 21 (depends on 20)

The apparatus of claim 20, wherein the first indicator comprises a visual indicator.

### Claim 22 (depends on 21)

The apparatus of claim 21, wherein the first indicator indicates the first linkage using a different textual and/or graphic style than the second indicator uses to indicate the second linkage.

### Claim 23 (depends on 22)

The apparatus of claim 22, wherein the first indicator indicates the first linkage using a first color, and the second indicator indicates the second linkage using a second color, wherein the first color is different from the second color.

### Claim 24 (depends on 21)

The apparatus of claim 21, wherein providing the first indicator comprises:
in response to a selection by the user of one of the first fact and a textual representation of the first portion of the free-form narration, visually identifying the other of the first fact and the textual representation of the first portion of the free-form narration.

### Claim 25 (depends on 21)

The apparatus of claim 21, wherein providing the first indicator comprises:
in response to a selection by the user of one of the first fact and a textual representation of the first portion of the free-form narration, displaying a graphical connector between the first fact and the textual representation of the first portion of the free-form narration.

### Claim 26 (depends on 21)

The apparatus of claim 21, wherein providing the first indicator comprises:
in response to a selection by the user of the first fact, scrolling a display of a textual representation of the free-form narration to a textual representation of the first portion of the free-form narration.

### Claim 27 (depends on 20)

The apparatus of claim 20, wherein the first indicator comprises an audio indicator.

### Claim 28 (depends on 27)

The apparatus of claim 27, wherein providing the first indicator comprises:
in response to a selection by the user of the first fact, playing an audio version of the first portion of the free-form narration.

### Claim 29 (depends on 28)

The apparatus of claim 28, wherein the audio version of the first portion of the free-form narration comprises a portion of an audio recording of the clinician dictating the free-form narration.

### Claim 30 (depends on 28)

The apparatus of claim 28, wherein the audio version of the first portion of the free-form narration is generated from a textual representation of the free-form narration using automatic speech synthesis.

### Claim 31 (depends on 20)

The apparatus of claim 20, wherein the method further comprises automatically editing one of the first fact and a data representation of the first portion of the free-form narration in response to the user editing the other of the first fact and the data representation of the first portion of the free-form narration.

### Claim 32 (depends on 20)

The apparatus of claim 20, wherein the method further comprises:
in response to the user selecting a third portion of the free-form narration, extracting a third fact from the third portion of the free-form narration.

### Claim 33 (depends on 20)

The apparatus of claim 20, wherein the method further comprises formatting a textual representation of the free-form narration to produce a formatted text, wherein the plurality of clinical facts are extracted from the formatted text.

### Claim 34 (depends on 33)

The apparatus of claim 33, wherein the formatting is performed in a way that facilitates the extracting of the plurality of clinical facts.

### Claim 35 (depends on 20)

The apparatus of claim 20, wherein the method further comprises:
including the first fact in a patient record; and
indicating a linkage between the first fact in the patient record and the first portion of the free-form narration.

### Claim 36 (depends on 35)

The apparatus of claim 35, wherein the indicating comprises:
simultaneously displaying at least a portion of the patient record and a textual representation of at least the first portion of the free-form narration; and
visually indicating the linkage between the first fact in the patient record and a textual representation of the first portion of the free-form narration.

### Claim 37 (depends on 20)

The apparatus of claim 20, wherein the user is the clinician.

### Claim 38 (depends on 20)

The apparatus of claim 20, wherein the user is a person other than the clinician.

### Claim 39

At least one non-transitory computer-readable storage medium encoded with a plurality of computer-executable instructions that, when executed, perform a method comprising:
extracting a plurality of clinical facts from a free-form narration of a patient encounter provided by a clinician, wherein the plurality of clinical facts comprises a first fact and a second fact, wherein the first fact is extracted from a first portion of the free-form narration, and wherein the second fact is extracted from a second portion of the free-form narration, the extracting comprising
analyzing the free-form narration to identify one or more sets of features of at least the first and second portions of the free-form narration,
correlating the one or more sets of features to a plurality of abstract semantic concepts, and
generating computer-readable data that expresses the plurality of abstract semantic concepts as the plurality of clinical facts extracted from the free-form narration; and
providing to a user a first indicator that indicates a first linkage between the first fact and the first portion of the free-form narration, wherein providing the first indicator comprises:
in response to the user selecting the first fact via a user interface, providing to the user an indicator that distinguishes the first portion of the free-form narration from which the first fact was extracted from other portions of the free-form narration from which the first fact was not extracted; and
providing to the user a second indicator, different from the first indicator, that indicates a second linkage between the second fact and the second portion of the free-form narration.

### Claim 40 (depends on 39)

The at least one non-transitory computer-readable storage medium of claim 39, wherein the first indicator comprises a visual indicator.

### Claim 41 (depends on 40)

The at least one non-transitory computer-readable storage medium of claim 40, wherein the first indicator indicates the first linkage using a different textual and/or graphic style than the second indicator uses to indicate the second linkage.

### Claim 42 (depends on 41)

The at least one non-transitory computer-readable storage medium of claim 41, wherein the first indicator indicates the first linkage using a first color, and the second indicator indicates the second linkage using a second color, wherein the first color is different from the second color.

### Claim 43 (depends on 40)

The at least one non-transitory computer-readable storage medium of claim 40, wherein providing the first indicator comprises:
in response to a selection by the user of one of the first fact and a textual representation of the first portion of the free-form narration, visually identifying the other of the first fact and the textual representation of the first portion of the free-form narration.

### Claim 44 (depends on 40)

The at least one non-transitory computer-readable storage medium of claim 40, wherein providing the first indicator comprises:
in response to a selection by the user of one of the first fact and a textual representation of the first portion of the free-form narration, displaying a graphical connector between the first fact and the textual representation of the first portion of the free-form narration.

### Claim 45 (depends on 40)

The at least one non-transitory computer-readable storage medium of claim 40, wherein providing the first indicator comprises:
in response to a selection by the user of the first fact, scrolling a display of a textual representation of the free-form narration to a textual representation of the first portion of the free-form narration.

### Claim 46 (depends on 39)

The at least one non-transitory computer-readable storage medium of claim 39, wherein the first indicator comprises an audio indicator.

### Claim 47 (depends on 46)

The at least one non-transitory computer-readable storage medium of claim 46, wherein providing the first indicator comprises:
in response to a selection by the user of the first fact, playing an audio version of the first portion of the free-form narration.

### Claim 48 (depends on 47)

The at least one non-transitory computer-readable storage medium of claim 47, wherein the audio version of the first portion of the free-form narration comprises a portion of an audio recording of the clinician dictating the free-form narration.

### Claim 49 (depends on 47)

The at least one non-transitory computer-readable storage medium of claim 47, wherein the audio version of the first portion of the free-form narration is generated from a textual representation of the free-form narration using automatic speech synthesis.

### Claim 50 (depends on 39)

The at least one non-transitory computer-readable storage medium of claim 39, wherein the method further comprises automatically editing one of the first fact and a data representation of the first portion of the free-form narration in response to the user editing the other of the first fact and the data representation of the first portion of the free-form narration.

### Claim 51 (depends on 39)

The at least one non-transitory computer-readable storage medium of claim 39, wherein the method further comprises:
in response to the user selecting a third portion of the free-form narration, extracting a third fact from the third portion of the free-form narration.

### Claim 52 (depends on 39)

The at least one non-transitory computer-readable storage medium of claim 39, wherein the method further comprises formatting a textual representation of the free-form narration to produce a formatted text, wherein the plurality of clinical facts are extracted from the formatted text.

### Claim 53 (depends on 52)

The at least one non-transitory computer-readable storage medium of claim 52, wherein the formatting is performed in a way that facilitates the extracting of the plurality of clinical facts.

### Claim 54 (depends on 39)

The at least one non-transitory computer-readable storage medium of claim 39, wherein the method further comprises:
including the first fact in a patient record; and
indicating a linkage between the first fact in the patient record and the first portion of the free-form narration.

### Claim 55 (depends on 54)

The at least one non-transitory computer-readable storage medium of claim 54, wherein the indicating comprises:
simultaneously displaying at least a portion of the patient record and a textual representation of at least the first portion of the free-form narration; and
visually indicating the linkage between the first fact in the patient record and a textual representation of the first portion of the free-form narration.

### Claim 56 (depends on 39)

The at least one non-transitory computer-readable storage medium of claim 39, wherein the user is the clinician.

### Claim 57 (depends on 39)

The at least one non-transitory computer-readable storage medium of claim 39, wherein the user is a person other than the clinician.