Using multiple modalities of surgical data for comprehensive data analytics of a surgical procedure

Abstract

This patent disclosure provides various embodiments of combining multiple modalities of non-text surgical data in forms of videos, images, and audios in a meaningful manner so that the combined data can be used to perform comprehensive data analytics for a surgical procedure. In some embodiments, the disclosed system can begin by receiving two or more modalities of surgical data during the surgical procedure. The system then time-synchronizes the two or more modalities of surgical data to generate two or more modalities of time-synchronized surgical data. Next, the system converts each modality of the time-synchronized surgical data into a corresponding array of values of a common format. The system then combines the two or more arrays of values to generate a combined set of values. The system subsequently performs comprehensive data analytics on the combined set of values to generate a surgical decision for the surgical procedure.

Dependencies are inferred from the claim text.