Anchor points for multi-modal data streams verification and contextualization

A method of automatically generating a characterization of a surgical procedure, the method comprising:

• acquiring surgical procedure data of the surgical procedure including at least a first intraoperative data stream and a second intraoperative data stream, wherein the first intraoperative data stream is different than the second intraoperative data stream, and wherein the first intraoperative data stream is captured simultaneously with the second intraoperative data stream;
• determining a first context in the first intraoperative data stream at a time in the first intraoperative data stream;
• based on the determined first context, determining a corresponding second context in the second intraoperative data stream at and/or proximate the same time in the second intraoperative data stream;
• inputting at least a portion of the first intraoperative data stream, at least a portion of the second intraoperative data stream, the first context, and the second context as inputs into an artificial intelligence (AI) application;
• and
• utilizing the AI application to convert the inputs into one or more natural language descriptions characterizing the surgical procedure.

A system for automatically generating a characterization of a surgical procedure, the method comprising:

• a sensor array including multiple sensors configured to simultaneously capture surgical procedure data of the surgical procedure including at least a first intraoperative data stream and a second intraoperative data stream, wherein the first intraoperative data stream is different than the second intraoperative data stream;
• and
• a surgical characterization processing device programmed with non-transitory computer readable instructions that, when executed by the surgical characterization processing device, cause the surgical characterization processing device to—acquire the surgical procedure data captured by the sensor array;
• determine a first context in the first intraoperative data stream at a time in the first intraoperative data stream;
• based on the determined first context, determine a corresponding second context in the second intraoperative data stream at and/or proximate the same time in the second intraoperative data stream;
• input at least a portion of the first intraoperative data stream, at least a portion of the second intraoperative data stream, the first context, and the second context as inputs into an artificial intelligence (AI) application;
• and
• utilize the AI application to convert the inputs into one or more natural language descriptions characterizing the surgical procedure.