Remote surgical mentoring

A surgical robotic system, comprising:

• a plurality of depth cameras, including a depth camera integrated with a portable electronic device operated by a local user in an operating room (OR), and a stationary depth camera arranged in the OR;
• a surgical robot and controller;
• and
• a processor, configured to perform the following:
• receiving, during a surgical robotic procedure of a patient in the OR, 1) surgical robotic information that indicates a position of the surgical robot in the OR and 2) sensor data captured by the depth camera and by the stationary depth camera;
• generating, using the surgical robotic information and sensor data, a virtual OR that includes a virtual patient and a virtual surgical robot;
• generating three-dimensional (3D) point cloud data based on the sensor data, the 3D point cloud data representing at least a portion of the OR that includes the patient and the surgical robot
• transmitting the virtual OR and the 3D point cloud data to a virtual reality headset worn by a remote user, causing the virtual reality headset to render a virtual reality environment to a display of the virtual reality headset worn by the remote user;
• and
• transmitting, to the portable electronic device of the local user and from the remote user, input indicating one or more recommended surgical ports of entry on the patient and a position of a virtual remote user within the virtual OR, wherein the position of the virtual remote user shares a common coordinate system that is common to 1) positions of the virtual patient and the virtual surgical robot within the virtual OR and 2) a position of the portable electronic device within the OR,
• wherein the portable device of the local user is to render, in augmented reality, the received one or more recommended surgical ports of entry over the patient and the a virtual representation of the remote user at the position within the virtual OR.