Real time image guided portable robotic intervention system

A method, comprising:

• receiving, from an imaging device coupled to a modular robotic arm, a plurality of real-time images of an object, wherein the imaging device is coupled to a first end of the modular robotic arm with a coupling mechanism;
• receiving, from the modular robotic arm coupled to a processing unit, pose information of a robotic manipulator, the modular robotic arm comprising a robotic manipulator and an instrument guide, wherein the instrument guide is coupled to a second end of the robotic manipulator;
• receiving, from the modular robotic arm, pose information of the instrument guide;
• determining the location and orientation of the instrument guide relative to the imaging device;
• displaying, on a user interface coupled to the processing unit, the plurality of real-time images of the object, wherein the plurality of real-time images includes a display of pixels from the imaging device for calculating an updated pose of the robotic arm;
• generating the updated pose of the robotic arm and a desired trajectory of the instrument guide based on a first selected pixel from the display of pixels;
• and
• augmenting the real-time images with a first graphical overlay, the first graphical overlay representing an updated desired trajectory of the instrument guide based on a second selected pixel from the display of pixels, wherein the updated desired trajectory indicates the path of an instrument inserted into the object using the instrument guide.

A non-transitory computer readable storage media comprising instructions, the instructions executable by a processor to perform a method, the method comprising:

• receiving, from an imaging device coupled to a modular robotic arm, a plurality of real-time images of an object, wherein the imaging device is coupled to a first end of the modular robotic arm with a coupling mechanism and wherein the plurality of real-time images include a display of pixels from the imaging device for calculating an updated pose of the robotic arm;
• receiving, from the modular robotic arm coupled to a processing unit, pose information of a robotic manipulator, the modular robotic arm comprising a robotic manipulator and an instrument guide, wherein the instrument guide is coupled to a second end of the robotic manipulator;
• receiving, from the modular robotic arm, pose information of the instrument guide;
• determining the location and orientation of the instrument guide relative to the imaging device;
• displaying, on a user interface coupled to the processing unit, the plurality of real-time images of the object;
• generating the updated pose of the robotic arm and a desired trajectory of the instrument guide based on a first selected pixel from the display of pixels;
• and
• augmenting the real-time images with a first graphical overlay, the first graphical overlay representing an updated desired trajectory of the instrument guide based on a second selected pixel from the display of pixels, wherein the updated desired trajectory indicates the path of an instrument inserted into the object using the instrument guide.