# US 11,896,324 B2 · System for transcribing and performing analysis on patient data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,896,324 B2 - Document ID: US11896324B2 - Status: Granted - Kind: B2 - Application number: 17/746,722 - Filed: 2022-05-17 - Published: 2023-07-06 - Granted: 2024-02-13 - Earliest claimed: 2022-01-04 - Assignees: IX Innovation LLC - Applicants: IX Innovation LLC - Inventors: Jeffrey Roh; Justin Esterberg; John Cronin; Seth Cronin; Michael John Baker - Web page: https://patentgrep.com/patent/US11896324B2 ## CPC - A61B 34/25: {User interfaces for surgical systems} - G06V 10/70: using pattern recognition or machine learning (optical pattern recognition or electronic computations therefor G06V10/88) - A61B 90/90: Identification means for patients or instruments, e.g. tags - A61B 90/98: using electromagnetic means, e.g. transponders - G06T 7/0012: {Biomedical image inspection} - G06V 10/40: Extraction of image or video features - G10L 13/02: Methods for producing synthetic speech; Speech synthesisers - G10L 15/063: {Training} - G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue - G10L 15/26: Speech to text systems (G10L15/08 takes precedence) - G10L 15/30: Distributed recognition, e.g. in client-server systems, for mobile phones or network applications - G16B 20/20: Allele or variant detection, e.g. single nucleotide polymorphism [SNP] detection - G16B 40/20: Supervised data analysis - G16H 10/60: for patient-specific data, e.g. for electronic patient records - G16H 50/20: for computer-aided diagnosis, e.g. based on medical expert systems - G16H 50/50: for simulation or modelling of medical disorders - A61B 2034/256: {having a database of accessory information, e.g. including context sensitive help or scientific articles} - G06T 2207/30004: Biomedical image processing - G06V 2201/03: Recognition of patterns in medical or anatomical images - G06V 2201/10: Recognition assisted with metadata - G10L 13/00: Speech synthesis; Text to speech systems - G16H 50/70: for mining of medical data, e.g. analysing previous cases of other patients - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING - G16B: BIOINFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR GENETIC OR PROTEIN-RELATED DATA PROCESSING IN COMPUTATIONAL MOLECULAR BIOLOGY - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract Methods, apparatuses, and systems for transcribing and performing analysis on patient data are disclosed. Data is collected from one or more medical professionals as well as sensors and imaging devices positioned on or oriented towards a patient. An analysis is performed on the patient data and the data is presented to a medical professional via a verbal interface in a conversational manner, allowing the medical professional to provide additional data such as observations or instructions which may be used for further analysis or to perform actions related to the patient's care. ## Claims ### Claim 1 A computer-implemented method comprising: transcribing physician input received from a physician using a voice-interactive transcription system that is configured to be voice-controlled by the physician, wherein the voice-interactive transcription system has a plurality of transcription modules that are selectable by the physician for a surgical procedure; during the surgical procedure being performed on a patient's body, analyzing the patient's body using a machine learning model associated with a selected one of the transcription modules to identify one or more surgical events based on sensor data generated from intraoperative patient vital measurements of the patient's body and the transcribed physician input, wherein the machine learning model is trained based on input from the physician, historical patient data, and historical treatment data; and outputting verbally the analysis of the patient's body to the physician using computer-generated natural language. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, comprising: generating the sensor data from the measurements of the patient's body using one or more sensors. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, comprising: retrieving patient data of the patient from a patient database, the patient data comprising at least a medical history of the patient; and determining a diagnosis based on the sensor data and the patient data. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, wherein the patient data comprises sequenced DNA of the patient, and the diagnosis comprises a hereditary medical condition. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, comprising: monitoring live surgical procedures; and training the machine learning model based on the live surgical procedures while the live surgical procedures are being performed. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein the transcribed physician input comprises medical notes, the method comprising: determining a diagnosis based on the medical notes. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the sensor data comprises medical images of the patient's body received from an imaging device, the method comprising: extracting an anomalous feature from the medical images; and determining a diagnosis using machine learning performed on the anomalous feature. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, comprising: for each of one or more medical conditions: assigning a statistical probability that the patient has each medical condition using the machine learning model; and selecting a medical condition having a greatest statistical probability. ### Claim 9 (depends on 8) The computer-implemented method of claim 8, comprising: selecting each of the one or more medical conditions having a statistical probability greater than a threshold statistical probability. ### Claim 10 (depends on 1) The computer-implemented method of claim 1, wherein training the machine learning model comprises: generating a predicted diagnosis and a predicted treatment for a previous patient based on the historical patient data; and comparing the predicted diagnosis and the predicted treatment to an actual diagnosis and an actual treatment provided by a previous surgeon in the historical patient data, the comparing for training a regression model of the system. ### Claim 11 A system comprising: one or more computer processors; and a non-transitory computer-readable storage medium storing computer instructions, which when executed by the one or more computer processors cause the system to: transcribe physician input received from a physician using a voice-interactive transcription system that is configured to be voice-controlled by the physician, wherein the voice-interactive transcription system has a plurality of transcription modules that are selectable by the physician for a surgical procedure; during the surgical procedure being performed on a patient's body, analyze the patient's body using a machine learning model associated with a selected one of the transcription modules to identify one or more surgical events based on sensor data generated from intraoperative patient vital measurements of the patient's body and the transcribed physician input, the machine learning model trained based on input from the physician, historical patient data, and historical treatment data; and output verbally an analysis of the patient's body to the physician using computer-generated natural language. ### Claim 12 (depends on 11) The system of claim 11, wherein the computer instructions cause the system to: generate the sensor data from the measurements of the patient's body using one or more sensors. ### Claim 13 (depends on 11) The system of claim 11, wherein the computer instructions cause the system to: retrieve patient data of the patient from a patient database, the patient data comprising at least a medical history of the patient; and determine a diagnosis based on the sensor data and the patient data. ### Claim 14 (depends on 13) The system of claim 13, wherein the patient data comprises sequenced DNA of the patient, and the diagnosis comprises a hereditary medical condition. ### Claim 15 (depends on 11) The system of claim 11, wherein the computer instructions cause the system to: monitor live surgical procedures; and train the machine learning model based on the live surgical procedures while the live surgical procedures are being performed. ### Claim 16 (depends on 11) The system of claim 11, wherein the transcribed physician input comprises medical notes, and the computer instructions cause the system to: determine a diagnosis based on the medical notes. ### Claim 17 (depends on 11) The system of claim 11, wherein the sensor data comprises medical images of the patient's body received from an imaging device, and the computer instructions cause the system to: extract an anomalous feature from the medical images; and determine a diagnosis using machine learning performed on the anomalous feature. ### Claim 18 (depends on 11) The system of claim 11, wherein the computer instructions cause the system to: for each of one or more medical conditions: assign a statistical probability that the patient has each medical condition using the machine learning model; and select a medical condition having a greatest statistical probability. ### Claim 19 (depends on 18) The system of claim 18, wherein the computer instructions cause the system to: select each of the one or more medical conditions having a statistical probability greater than a threshold statistical probability. ### Claim 20 A non-transitory computer-readable storage medium storing computer instructions, which when executed by one or more computer processors cause the one or more computer processors to: transcribe physician input received from a physician using a voice-interactive transcription system that is configured to be voice-controlled by the physician, wherein the voice-interactive transcription system has a plurality of transcription modules that are selectable by the physician for a surgical procedure; during the surgical procedure being performed on a patient's body, analyze the patient's body using a machine learning model associated with a selected one of the transcription modules to identify one or more surgical events based on sensor data generated from intraoperative patient vital measurements of the patient's body and the transcribed physician input, the machine learning model trained based on input from the physician, historical patient data, and historical treatment data; and output verbally an analysis of the patient's body to the physician using computer-generated natural language.