Operating room black-box device, system, method and computer readable medium for event and error prediction

A computer implemented method executed on a electronic recording apparatus configured for automatically processing audio-visual recordings of one or more surgical procedures to automatically generate data structures representing durations of time where predicted events are estimated to augment rendering of a decision support graphical user interface associating digital recordings of one or more surgical procedures with the predicted events based on predictions generated by a machine learning data model architecture, the method comprising:

• recording real-time medical or surgical data streams using a recording device including at least one of cameras, sensors, audio devices, and patient monitoring hardware recording time synchronized real-time medical or surgical data streams relating to a real-time medical procedure within an operating or clinical site;
• maintaining a trained machine learning data model architecture trained to support iterative machine learning;
• identifying, using the trained machine learning data model architecture, one or more potential time-stamped clinical events within the real-time medical or surgical data streams;
• generating a time-synchronized metadata track representing timestamp markers corresponding to specific timestamps or timestamp ranges of the durations of time each timestamp marker of the timestamp markers corresponding to a specific time-stamped potential clinical event and including a data field representing the associated confidence level;
• training the machine learning data model architecture using reviewer validation inputs representing one or more labels corresponding to at least one of the time-stamped potential clinical events, the one or more labels each indicating whether the corresponding time-stamped potential clinical event was correct or false;
• and
• controlling rendering of the decision support graphical user interface by overlaying a visual rendering of the time-synchronized metadata track upon a rendering of a timeline chart, one or more interactive visual interface elements, each interactive visual interface element corresponding to the corresponding specific time-stamped potential clinical event.

A non-transitory computer readable medium, storing machine interpretable instruction sets, which when executed by a processor, cause the processor to perform a computer implemented method for automatically processing audio-visual recordings of one or more surgical procedures to automatically generate data structures representing durations of time where predicted events are estimated to augment rendering of a decision support graphical user interface associating digital recordings of one or more surgical procedures with the predicted events based on predictions generated by a machine learning data model architecture, the method executed on a electronic recording apparatus, the method comprising:

• collecting real-time medical or surgical data streams using a recording device including at least one of cameras, sensors, audio devices, and patient monitoring hardware recording time synchronized real-time medical or surgical data streams relating to a real-time medical procedure within an operating or clinical site;
• maintaining a trained machine learning data model architecture trained to support iterative machine learning;
• identifying, using the trained machine learning data model architecture, one or more potential time-stamped clinical events within the real-time medical or surgical data streams;
• generating a time-synchronized metadata track representing timestamp markers corresponding to specific timestamps or timestamp ranges of the durations of time, each timestamp marker of the timestamp markers corresponding to a specific time-stamped potential clinical event;
• training the machine learning data model architecture using reviewer validation inputs representing one or more labels corresponding to at least one of the time-stamped potential clinical events, the one or more labels each indicating whether the corresponding time-stamped potential clinical event was correct or false;
• and
• controlling rendering of the decision support graphical user interface by overlaying a visual rendering of the time-synchronized metadata track upon a rendering of a timeline chart, one or more interactive visual interface elements, each interactive visual interface element corresponding the corresponding specific time-stamped potential clinical event.