# US 11,998,287 B1 · Platform for facilitating remote robotic medical procedures > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,998,287 B1 - Document ID: US11998287B1 - Status: Granted - Kind: B1 - Application number: 16/823,234 - Filed: 2020-03-18 - Granted: 2024-06-04 - Earliest claimed: 2019-03-18 - Assignees: Dopl Technologies Inc. - Applicants: Dopl Technologies Inc. - Inventors: Ryan Christopher James - Web page: https://patentgrep.com/patent/US11998287B1 ## CPC - A61B 34/30: Surgical robots - A61B 18/1492: {having a flexible, catheter-like structure, e.g. for heart ablation (A61B18/1477 takes precedence)} - B25J 9/0009: {Constructional details, e.g. manipulator supports, bases} - B25J 9/1689: {Teleoperation} - B25J 9/1694: {characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion} - B25J 9/1697: {Vision controlled systems} - B25J 19/02: Sensing devices - G06F 16/24568: {Data stream processing; Continuous queries} - G06F 16/254: {Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses} - H04L 67/12: specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks - A61B 2017/00022: {Sensing or detecting at the treatment site} - A61B 2017/00199: {with a console, e.g. a control panel with a display} - A61B 2017/00212: {using remote controls} - A61B 2017/00221: {with wireless transmission of data, e.g. by infrared radiation or radiowaves} - A61B 2018/00345: {Vascular system} - A61B 2018/00577: {Ablation} - A61B 2034/301: {for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes} - G05B 2219/40153: Teleassistance, operator assists, controls autonomous robot - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50}) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) ## Abstract A hardware platform includes adapters to interface with medical provisioning equipment involved in a remote robotic medical procedure at a patient site and a software system component, potentially cloud-based, to process real-time incoming data streams and a database to store the data streams for later use or archival. Each data stream can itself contain different types of sensory data. The software system component provides overall control and overrideable supervision over the remote robotic operations includes a state manager that synergistically processes the incoming data streams possibly of varying transmission rates, times of receipt or transmission, resolution, and quality. The state manager forms a near- or real-time virtual representation of the operative field. The software system component includes a data broker that can parse the RTVR into subsets data tailored to the needs of the medical provisioning and monitoring equipment, which may also be deployed and co-located at the patient site. ## Claims ### Claim 1 A platform for facilitating remote robotic procedures, comprising: a plurality of incoming data streams comprising sensory data obtained from provisioning equipment involved in a remote robotic procedure at a target site, the plurality of incoming data streams comprising different types of sensory data, wherein the sensory data comprises sensory data pieces associated with markers; and at least one computer comprising a processor coupled to a memory storing computer-executable code, the processor configured to: receive incoming data streams into a state manager, wherein at least some of the data streams are received at disparate rates from each other and at least some of the sensory data in the received data streams is received out of temporal sequence, the state manager adapted to synchronize the incoming data streams at least partially based on the markers associated with the sensory data pieces and derive from the different types of sensory data a near-real time virtual representation of an operative field at the target site; provide the near-real time virtual representation to a data broker that is adapted to parse the near-real time virtual representation into one or more subsets adapted to needs of a requesting component; a heuristics module adapted to generate using artificial intelligence at least one autonomous control function comprising determining a correct time for a user command, detecting the user command prior to the correct time, and generating the autonomous control function to override the user command based on the detection; and an at least one outgoing data stream further comprising the at least one autonomous control function directed to the provisioning equipment. ### Claim 2 (depends on 1) A platform in accordance with claim 1, the state manager further comprising: an aggregation module adapted to aggregate the different types of sensory data into the near-real time virtual representation. ### Claim 3 (depends on 1) A platform in accordance with claim 1, the state manager further comprising: an aggregation module adapted to aggregate one or more of position, rotation, orientation data from the different types of sensory data into at least one of an representation of a remote target site. ### Claim 4 (depends on 1) A platform in accordance with claim 1, further comprising: the provisioning equipment comprising one of a sensor and a device adapted to detect and quantify one or more of voltage, pressure, sound, chemical presence, light reflectance, light absorption, electromagnetic energy, radiofrequency energy, and infrared energy. ### Claim 5 (depends on 4) A platform in accordance with claim 4, the state manager further comprising: an aggregation module adapted to aggregate the sensory data from the one of a sensor and a device into the near-real time virtual representation to represent position and rotation of the one of a sensor and a device within the target site. ### Claim 6 (depends on 4) A platform in accordance with claim 4, the state manager further comprising: the provisioning equipment further comprising a plurality of the one of a sensor and a device; and an aggregation module adapted to aggregate the sensory data from the plurality of the one of a sensor and a device into the near-real time virtual representation to represent location and orientation of the plurality of the one of a sensor and a device within the target site. ### Claim 7 (depends on 6) A platform in accordance with claim 6, wherein the one of a sensor and a device is adapted to visualize and provide a representation of an operative field at the target site into at least one of the outgoing data streams. ### Claim 8 (depends on 1) A platform in accordance with claim 1, the state manager further comprising: a heuristics module adapted to determine a predicted future state of the procedure and to integrate the predicted future state into the near-real time virtual representation. ### Claim 9 (depends on 1) A platform in accordance with claim 1, each incoming data stream further comprising: an analog data feed transmitted by the provisioning equipment; and a digital data feed derived from the provisioning equipment. ### Claim 10 (depends on 9) A platform in accordance with claim 9, further comprising: an adapter operatively coupled to the provisioning equipment and comprising at least one of: a digital-to-analog converter; a digital signal filter; an analog signal filter; a signal amplifier; and a signal attenuator. ### Claim 11 (depends on 1) A platform in accordance with claim 1, further comprising: the provisioning equipment adapted to generate coordinates further comprised in the at least one of the incoming data streams describing one or more of position and orientation of the provisioning equipment within the target site. ### Claim 12 (depends on 1) A platform in accordance with claim 1, wherein the at least one incoming data stream comprises one or more messages. ### Claim 13 (depends on 1) A platform in accordance with claim 1, the requesting component comprising: a display device involved in the remote robotic procedure at a remote site. ### Claim 14 (depends on 1) A platform in accordance with claim 1, the requesting component comprising: remotely controllable provisioning equipment involved in the remote robotic procedure at a remote site. ### Claim 15 (depends on 1) A platform in accordance with claim 1, the requesting component comprising: an analysis module adapted to algorithmically evaluate the one or more subsets of the near-real time virtual representation to arrive at an analytical result. ### Claim 16 (depends on 1) A platform in accordance with claim 1, wherein the at least one computer is comprised within an online cloud computing center. ### Claim 17 (depends on 1) A platform in accordance with claim 1, wherein the markers are one of temporal markers and serialization markers. ### Claim 18 (depends on 1) A platform in accordance with claim 1, wherein the sensory data does not comprise sound data. ### Claim 19 (depends on 1) A platform in accordance with claim 1, wherein at least some of the incoming data streams comprise human-readable codes and the state manager is further adapted to derive the near-real time virtual representation of the operative field at the target site using the human-readable codes.