Stereo video in augmented reality

A system comprising one or more processors, and a non-transitory computer-readable medium including one or more sequences of instructions that, when executed by the one or more processors, cause the system to perform operations comprising:

• generating within a unified three-dimensional (3D) coordinate space, a virtual 3D medical model positioned according to a model pose;
• receiving video data from a plurality of video sources;
• rendering a first Augmented Reality (AR) display that includes concurrent display of the virtual 3D medical model and visualization of at least a portion of video data from a first video source;
• rendering a second AR display that includes concurrent display of the virtual 3D medical model and visualization of at least a portion of video data from a second video source;
• detecting a first physical gesture indicative of identifying a selection to initiate a type of virtual interaction from a plurality of types of virtual interactions based on the detected first physical gesture;
• concurrently modifying the first AR display and the second AR display, via the AR headset device, by (a) initiating display of a selectable first virtual object in the first AR display that corresponds with the selected type of virtual interaction and by (b) initiating display of a selectable second virtual object in the second AR display that corresponds with the selected type of virtual interaction;
• wherein rendering the first AR display further comprises:
• rendering a first virtual screen portraying the visualization of the video from the first video source concurrently with display of the first virtual object;
• and
• wherein rendering the second AR display further comprises:
• rendering a second virtual screen portraying the visualization of the video from the second video source concurrently with display of the second virtual object, the first and the second virtual objects each comprising an instance of a same type of virtual object.