Automated clinical documentation system and method for synchronizing machine vision and audio encounter information

An automated clinical documentation (ACD) system comprising:

• one or more machine vision devices configured to record machine vision encounter information of a clinical encounter that includes a patient meeting with a participant in a clinical environment monitored by the ACD system;
• one or more audio sensors configured to record audio encounter information of the clinical encounter, the one or more audio sensors and the one or more machine vision devices of the ACD system being deployed in the clinical environment, wherein the one or more audio sensors are not synchronized with the one or more machine vision devices such that the machine vision encounter information and the audio encounter information lack synchronization at the time of recording, and wherein the machine vision encounter information and the audio encounter information are temporally aligned to produce a temporally-aligned encounter recording that synchronizes the machine vision encounter information with the audio encounter information;
• at least one processor configured to generate a transcript of the clinical encounter based on the temporally-aligned encounter recording, wherein at least a portion of the transcript is generated by performing visual speech recognition on the synchronized machine vision encounter information, and wherein the at least one processor is further configured to proactively scan the transcript based on a medical conditions symptoms datasource to identify indications of a specific medical condition, wherein the medical conditions symptoms datasource defines symptoms for various medical conditions, and wherein the ACD system sends a message to a participant of the clinical encounter during the clinical encounter responsive to determining that a number of the indications of the specific medical condition exceeds a threshold, the message recommending that the patient undergo testing for the specific medical condition;
• and
• a display configured to display a visual representation of the transcript, the visual representation of the transcript identifying portions of the transcript that indicate the specific medical condition.

An automated clinical documentation (ACD) system comprising:

• at least one processor;
• and
• a memory storing programming instructions for execution by the at least one processor, the programming instructions, upon execution by the at least one processor, causing the ACD system to perform the following operations:
• recording, by one or more machine vision devices of the ACD system, machine vision encounter information of a clinical encounter that includes a patient meeting with a participant in a clinical environment monitored by the ACD system, the one or more machine vision devices of the ACD system being deployed in the clinical environment;
• recording, by one or more audio sensors of the ACD system, audio encounter information of the clinical encounter, the one or more audio sensors of the ACD system being deployed in the clinical environment, wherein the one or more audio sensors are not synchronized with the one or more machine vision devices such that the machine vision encounter information and the audio encounter information lack synchronization at the time of recording;
• temporally-aligning the machine vision encounter information and the audio encounter information to produce a temporally-aligned encounter recording that synchronizes the machine vision encounter information with the audio encounter information;
• generating, by the ACD system, a transcript of the clinical encounter based on the temporally-aligned encounter recording, wherein at least a portion of the transcript is generated by performing visual speech recognition on the synchronized machine vision encounter information;
• proactively scanning, by the ACD system, the transcript based on a medical conditions symptoms datasource to identify indications of a specific medical condition, wherein the medical conditions symptoms datasource defines symptoms for various medical conditions;
• sending, by the ACD system during the clinical encounter, a message to a participant of the clinical encounter responsive to determining that a number of the indications of the specific medical condition exceeds a threshold, the message recommending that the patient undergo testing for the specific medical condition;
• and
• displaying, via a display of the ACD system, a visual representation of the transcript, the visual representation of the transcript identifying portions of the transcript that indicate the specific medical condition.