Methods and systems for using multiple data structures to process surgical data

A computer-implemented method comprising:

• accessing, by a processor, a surgical dataset associated with a surgical procedure, the surgical dataset comprising a surgical video of the surgical procedure being performed;
• creating, by the processor, a surgical map of the surgical procedure by analyzing the surgical video using a trained machine learning algorithm, wherein the surgical map comprises at least:
• a first data structure that comprises a first plurality of nodes and a first plurality of edges, wherein a first node from the first data structure represents a procedural state in the surgical procedure;
• a second data structure that comprises a second plurality of nodes and a second plurality of edges, wherein a second node from the second data structure represents a surgical tool used in the surgical procedure;
• adding, by the processor, an interconnecting edge between the first node and the second node based on a surgical event detected in the surgical video, wherein the interconnecting edge is associated with a relational metadata representing a use of the surgical tool of the second node in the procedural state of the first node;
• generating, by the processor, electronic data to be associated with a portion of the surgical video, the electronic data describing an operational note based at least on the procedural state of the first node, a data associated with the second node, and the relational metadata associated with the interconnecting edge between the first node and the second node, the portion comprising the use of the surgical tool of the second node in the procedural state of the first node;
• and
• outputting, by the processor, the electronic data during playback of the portion of the surgical video.

A system comprising:

• a multi-dimensional artificial intelligence system;
• a surgical operating system;
• and one or more processors coupled with the multi-dimensional artificial intelligence system, and the surgical operating system, the one or more processors are configured to perform a method comprising:
• accessing a surgical dataset associated with a surgical procedure, the surgical dataset comprising a surgical video of the surgical procedure being performed;
• creating a surgical map of the surgical procedure by analyzing the surgical video using the multi-dimensional artificial intelligence system, wherein the surgical map comprises at least:
• a first data structure that comprises a first plurality of nodes and a first plurality of edges, wherein a first node from the first data structure represents a first procedural feature of the surgical procedure;
• and
• a second data structure that comprises a second plurality of nodes and a second plurality of edges, wherein a second node from the second data structure represents second procedural feature of the surgical procedure;
• adding an interconnecting edge between the first node and the second node based on a surgical event detected in the surgical video, wherein the interconnecting edge is associated with a relational metadata representing a use of the first procedural feature of the first node in connection with the second procedural feature of the second node;
• generating an electronic data to be associated with a portion of the surgical video, wherein the electronic data describes an operational note based at least on the first procedural feature, the second procedural feature, and the relational metadata associated with the interconnecting edge between the first node and the second node, the portion comprising the use of the first procedural feature in connection with the second procedural feature;
• and
• outputting the electronic data during playback of the portion of the surgical video.

A computer program product comprising a non-transitory computer storage medium having computer executable instructions stored thereon, the computer executable instructions when executed by one or more processors cause the one or more processors to perform a method comprising:

• creating a surgical map of a surgical procedure by analyzing a surgical video of the surgical procedure using a trained machine learning algorithm, wherein the surgical map comprises at least:
• a first data structure that comprises a first plurality of nodes, wherein a first node from the first data structure represents a first procedural feature of the surgical procedure;
• and
• a second data structure that comprises a second plurality of nodes, wherein a second node from the second data structure represents second procedural feature of the surgical procedure;
• adding an interconnecting edge between the first node and the second node based on a surgical event detected in the surgical video, wherein the interconnecting edge is associated with a relational metadata representing a use of the first procedural feature of the first node in connection with the second procedural feature of the second node;
• augmenting the surgical video by including an electronic data to a portion of the surgical video, the electronic data describing an operational note based at least on the first procedural feature, the second procedural feature, and the relational metadata associated with the interconnecting edge between the first node and the second node.