# US 11,442,614 B2 · Method and system for generating transcripts of patient-healthcare provider conversations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,442,614 B2 - Document ID: US11442614B2 - Status: Granted - Kind: B2 - Application number: 17/215,512 - Filed: 2021-03-29 - Published: 2021-07-15 - Granted: 2022-09-13 - Earliest claimed: 2017-10-23 - Assignees: Google Inc. - Applicants: Google LLC - Inventors: Melissa Strader; William Ito; Christopher Co; Katherine Chou; Alvin Rajkomar; Rebecca Rolfe - Web page: https://patentgrep.com/patent/US11442614B2 ## CPC - G06F 3/04855: Interaction with scrollbars - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 40/289: Phrasal analysis, e.g. finite state techniques or chunking - G06F 40/30: Semantic analysis - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G16H 10/20: for electronic clinical trials or questionnaires - G16H 10/60: for patient-specific data, e.g. for electronic patient records - G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof - G16H 80/00: ICT specially adapted for facilitating communication between medical practitioners or patients, e.g. for collaborative diagnosis, therapy or health monitoring - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract A method and workstation for generating a transcript of a conversation between a patient and a healthcare practitioner is disclosed. A workstation is provided with a tool for rendering of an audio recording of the conversation and generating a display of a transcript of the audio recording using a speech-to-text engine, thereby enabling inspection of the accuracy of conversion of speech to text. A tool is provided for scrolling through the transcript and rendering the portion of the audio according to the position of the scrolling. There is a highlighting in the transcript of words or phrases spoken by the patient relating to symptoms, medications or other medically relevant concepts. Additionally, there is provided a set of transcript supplement tools enabling editing of specific portions of the transcript based on the content of the corresponding portion of audio recording. ## Claims ### Claim 1 A method for generating a transcript of a conversation between a patient and a healthcare practitioner, comprising: providing a rendering of an audio recording of the conversation and generating a display of the transcript of the audio recording using a speech-to-text engine in substantial real time with the rendering of the audio recording; enabling scrolling through the transcript and rendering a portion of the audio recording according to a position of the scrolling; automatically recognizing, in the transcript or in the audio recording, words or phrases spoken by the patient relating to one or more of symptoms, medications or other medically relevant concepts; and displaying a note simultaneously with the display of the transcript and populating the note with the words or phrases in substantial real time with the rendering of the audio recording. ### Claim 2 (depends on 1) The method of claim 1, further comprising: providing a set of note supplement tools enabling editing of specific portions of the note based on a content of a corresponding portion of the transcript or the audio recording. ### Claim 3 (depends on 2) The method of claim 2, wherein the set of note supplement tools include at least one of: a) a display of smart suggestions for words or phrases and a tool for editing, approving, rejecting or providing feedback on the smart suggestions; b) a display of suggested corrected medical terminology; and c) a display of an indication of confidence level in suggested words or phrases. ### Claim 4 (depends on 3) The method of claim 3, wherein the set of note supplement tools include each of the displays a), b) and c) recited in claim 3. ### Claim 5 (depends on 3) The method of claim 3, wherein the display of the indication of the confidence level comprises display upon a determination that the confidence level exceeds a threshold confidence level. ### Claim 6 (depends on 1) The method of claim 1, further comprising: minimizing display of the transcript and viewing the note only, and wherein the note is generated in substantial real time with the rendering of the audio recording. ### Claim 7 (depends on 1) The method of claim 1, wherein the words or phrase are placed into appropriate categories or classifications in the note. ### Claim 8 (depends on 1) The method of claim 1, further comprising: providing, in the note, supplementary information for symptoms including labels for phrases required for billing. ### Claim 9 (depends on 1) The method of claim 1, further comprising: a set of transcript supplement tools to enable editing of specific portions of the transcript based on a content of a corresponding portion of the audio recording. ### Claim 10 (depends on 1) The method of claim 1, further comprising: linking words or phrases in the note to relevant parts of the transcript from which the words or phrases in the note originated. ### Claim 11 (depends on 1) The method of claim 1, wherein the automatically recognizing of the words or phrases is performed by using a trained machine learning model. ### Claim 12 A server for displaying a transcript of a conversation between a patient and a healthcare practitioner, comprising: one or more processors; and memory storing computer-executable instructions that, when executed by the one or more processors, cause the server to: receive an audio recording of the conversation; apply a machine learning model to generate the transcript of the audio recording using a speech-to-text engine in substantial real time with a rendering of the audio recording, wherein the machine learning model is trained to automatically recognize, in the audio recording or in the transcript, words or phrases spoken by the patient relating to one or more of symptoms, medications or other medically relevant concepts; and transmit the transcript by an application programming interface to a workstation, and wherein the instructions cause the workstation to: generate a display of the transcript of the audio recording; provide a user interface element enabling scrolling through the transcript and rendering a portion of the audio according to a position the scrolling; automatically recognize, in the transcript, words or phrases spoken by the patient relating to one or more of symptoms, medications or other medically relevant concepts; and display a note simultaneously with the display of the transcript and populating the note with the words or phrases in substantial real time with the rendering of the audio recording. ### Claim 13 (depends on 12) The server of claim 12, wherein the instructions cause the server to: apply the machine learning model to generate the note in substantial real time with the rendering of the audio recording; and transmit the note by the application programming interface to the workstation. ### Claim 14 (depends on 12) The server of claim 12, wherein the instructions further cause the workstation to: provide a set of note supplement tools enabling editing of specific portions of the note based on a content of a corresponding portion of the transcript or the audio recording. ### Claim 15 (depends on 14) The server of claim 14, wherein the set of note supplement tools include at least one of: a) a display of smart suggestions for words or phrases and a tool for editing, approving, rejecting or providing feedback on the smart suggestions; b) a display of suggested corrected medical terminology; and c) a display of an indication of confidence level in suggested words or phrases. ### Claim 16 (depends on 15) The server of claim 15, wherein the display of the indication of the confidence level comprises display upon a determination that the confidence level exceeds a threshold confidence level. ### Claim 17 (depends on 12) The server of claim 12, wherein the instructions further cause the workstation to: provide, in the note, supplementary information for symptoms including labels for phrases required for billing. ### Claim 18 (depends on 12) The server of claim 12, wherein the instructions further cause the workstation to: link words or phrases in the note to relevant parts of the transcript from which the words or phrases in the note originated. ### Claim 19 (depends on 12) The server of claim 12, wherein the instructions further cause the workstation to: minimize display of the transcript and enable viewing of the note only. ### Claim 20 An article manufacture comprising one or more computer readable media having computer-readable instructions stored thereon that, when executed by one or more processors of a computing device, cause the computing device to: provide a rendering of an audio recording of a conversation between a patient and a healthcare practitioner, and generate a display of a transcript of the audio recording using a speech-to-text engine in substantial real time with the rendering of the audio recording; enable scrolling through the transcript and render a portion of the audio recording according to a position of the scrolling; automatically recognize, in the transcript or in the audio recording, words or phrases spoken by the patient relating to one or more of symptoms, medications or other medically relevant concepts; and display a note simultaneously with the display of the transcript and populate the note with the words or phrases in substantial real time with the rendering of the audio recording.