# US 2023/0054586 A1 · Robotic surgical inventory management > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2023/0054586 A1 - Document ID: US20230054586A1 - Status: Published application - Kind: A1 - Application number: 17/408,404 - Filed: 2021-08-21 - Published: 2023-02-23 - Earliest claimed: 2021-08-21 - Assignees: IX Innovation LLC - Applicants: IX Innovation LLC - Inventors: Jeffrey Roh; Justin Esterberg; John Cronin; Seth Cronin; Michael D'Andrea - Web page: https://patentgrep.com/patent/US20230054586A1 ## CPC - A61B 90/08: {Accessories or related features not otherwise provided for} - A61B 90/98: using electromagnetic means, e.g. transponders - A61B 34/25: {User interfaces for surgical systems} - A61B 34/30: Surgical robots - A61B 34/70: {Manipulators specially adapted for use in surgery} - A61B 50/33: Trays - A61B 90/96: using barcodes - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 5/01: Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound - G06N 7/01: Probabilistic graphical models, e.g. probabilistic networks - G06N 20/00: Machine learning - G06N 20/10: using kernel methods, e.g. support vector machines [SVM] - G06N 20/20: Ensemble learning - G16H 10/60: for patient-specific data, e.g. for electronic patient records - G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture - G16H 40/20: for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms - G16H 40/63: for local operation - A61B 5/01: Measuring temperature of body parts {; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue} (clinical contact thermometers G01K13/20) - A61B 5/021: Measuring pressure in heart or blood vessels - A61B 5/024: Measuring pulse rate or heart rate - A61B 5/0816: {Measuring devices for examining respiratory frequency (measuring frequency of electric signals G01R23/00)} - A61B 5/24: Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof - A61B 5/318: Heart-related electrical modalities, e.g. electrocardiography [ECG] - A61B 5/369: Electroencephalography [EEG] - A61B 5/389: Electromyography [EMG] - A61B 90/20: Surgical microscopes characterised by non-optical aspects - A61B 2034/2055: {Optical tracking systems} - A61B 2034/252: {indicating steps of a surgical procedure} - A61B 2034/254: {being adapted depending on the stage of the surgical procedure} - A61B 2034/256: {having a database of accessory information, e.g. including context sensitive help or scientific articles} - A61B 2050/375: {for sponges} - A61B 2090/0804: {Counting number of instruments used; Instrument detectors} - A61B 2090/306: {using optical fibres} - A61B 2090/309: {using white LEDs} - A61B 2090/3614: {using optical fibre} - A61B 2090/378: {using ultrasound} - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA ## Abstract A supply tray for a surgical procedure is selected based on the surgical procedure and patient data retrieved from an electronic health records database. Multiple steps of the surgical procedure are retrieved from the electronic health records database. A message is sent to a first manipulator to move a supply from the supply tray to a staging area for performing a step. A first indication is received from a first sensor that the supply is needed at a present time. A position where the supply is needed in an operating area proximate to the staging area is determined using a second sensor. A second message is sent to a second manipulator to move the supply from the staging area to the position. A second indication is received from a third sensor that the step is complete. A third message is sent to a third manipulator to remove the supply. ## Claims ### Claim at position 1 1-34. (canceled) ### Claim 35 A method comprising: identifying, by a supply monitoring surgical robot via end-effector supply imagers, a supply tray for a surgical procedure from a supply database, the identifying based on the surgical procedure and patient data retrieved from an electronic health records database; controlling, by the supply monitoring surgical robot a first manipulator to move a supply from the supply tray to a staging area, the supply for performing a step of a plurality of steps of the surgical procedure; detecting, via a first movable end-effector sensor on the supply monitoring surgical robot, that the supply is needed at a present time for the step, wherein detecting the supply is needed is based on one or more real-time patient measurements, collected by one or more vital monitoring sensors, reaching a threshold; in response to detecting the supply is needed, identifying, via a second movable end-effector sensor on the supply monitoring surgical robot, a position where the supply is needed in an operating area proximate to the staging area, wherein the supply monitoring surgical robot is configured to move the first and/or second movable end-effector sensors to analyze areas of interest; controlling, by the supply monitoring surgical robot, a second manipulator to move the supply from the staging area to the position in the operating area; detecting, via a third movable end-effector sensor on the supply monitoring surgical robot, that the step is complete; and in response to detecting the step is complete, controlling, by the supply monitoring surgical robot, a third manipulator to remove the supply from the position in the operating area. ### Claim 36 (depends on 35) The method of claim 35, further comprising: identifying, by the supply monitoring surgical robot, a supply fault based on a mismatch between a first number of supplies in the supply tray and a second number of supplies in the staging area, the operating area, and a receiving area; and displaying, by the supply monitoring surgical robot, a notification based on the supply fault. ### Claim 37 (depends on 35) The method of claim 35, wherein the first manipulator is located outside a sterile field and the second manipulator is located inside the sterile field. ### Claim 38 (depends on 35) The method of claim 35, wherein the first movable end-effector sensor is a microphone and the microphone detects an instruction spoken by a medical practitioner. ### Claim 39 (depends on 35) The method of claim 35, wherein the first movable end-effector sensor is a camera and the camera captures an image, the method further comprising performing image recognition on the image to determine that the image indicates the supply. ### Claim 40 (depends on 39) The method of claim 39, wherein the image recognition is performed using a machine learning model trained on a plurality of images to generate a determination that the image indicates the supply. ### Claim 41 (depends on 39) The method of claim 39, further comprising matching the image to the plurality of steps to predict that a next supply will be needed at a next time for a next step of the plurality of steps. ### Claim 42 A system comprising: one or more computer processors; and a non-transitory computer readable storage medium storing computer instructions, which when executed by the one or more computer processors, causes the one or more computer processors to: identify, by a supply monitoring surgical robot via end-effector supply imagers, a supply tray for a surgical procedure from a supply database, the identifying based on the surgical procedure and patient data retrieved from an electronic health records database; control, by the supply monitoring surgical robot a first manipulator to move a supply from the supply tray to a staging area, the supply for performing a step of a plurality of steps of the surgical procedure; detect, via a first movable end-effector sensor on the supply monitoring surgical robot, that the supply is needed at a present time for the step, wherein detecting the supply is needed is based on one or more real-time patient measurements, collected by one or more vital monitoring sensors, reaching a threshold; in response to detecting the supply is needed, identify, via a second movable end-effector sensor on the supply monitoring surgical robot, a position where the supply is needed in an operating area proximate to the staging area, wherein the supply monitoring surgical robot is configured to move the first and/or second movable end-effector sensors to analyze areas of interest; control, by the supply monitoring surgical robot, a second manipulator to move the supply from the staging area to the position in the operating area; detect, via a third movable end-effector sensor on the supply monitoring surgical robot, that the step is complete; and in response to detecting the step is complete, control, by the supply monitoring surgical robot, a third manipulator to remove the supply from the position in the operating area. ### Claim 43 (depends on 42) The system of claim 42, wherein the computer instructions, which when executed by the one or more computer processors further cause the one or more computer processors to: identify, by the supply monitoring surgical robot, a supply fault based on a mismatch between a first number of supplies in the supply tray and a second number of supplies in the staging area, the operating area, and a receiving area; and display, by the supply monitoring surgical robot, a notification based on the supply fault. ### Claim 44 (depends on 42) The system of claim 42, wherein the first manipulator is located outside a sterile field and the second manipulator is located inside the sterile field. ### Claim 45 (depends on 42) The system of claim 42, wherein the first movable end-effector sensor is a microphone and the microphone detects an instruction spoken by a medical practitioner. ### Claim 46 (depends on 42) The system of claim 42, wherein the first movable end-effector sensor is a camera and the camera captures an image, wherein the computer instructions, which when executed by the one or more computer processors further cause the one or more computer processors to perform image recognition on the image to determine that the image indicates the supply. ### Claim 47 (depends on 46) The system of claim 46, wherein the image recognition is performed using a machine learning model trained on a plurality of images to generate a determination that the image indicates the supply. ### Claim 48 (depends on 46) The system of claim 46, wherein the computer instructions, which when executed by the one or more computer processors further cause the one or more computer processors to match the image to the plurality of steps to predict that a next supply will be needed at a next time for a next step of the plurality of steps. ### Claim 49 A non-transitory computer readable storage medium storing computer instructions, which when executed by one or more computer processors, causes the one or more computer processors to: identify, by a supply monitoring surgical robot via end-effector supply imagers, a supply tray for a surgical procedure from a supply database, the identifying based on the surgical procedure and patient data retrieved from an electronic health records database; control, by the supply monitoring surgical robot a first manipulator to move a supply from the supply tray to a staging area, the supply for performing a step of a plurality of steps of the surgical procedure; detect, via a first movable end-effector sensor on the supply monitoring surgical robot, that the supply is needed at a present time for the step, wherein detecting the supply is needed is based on one or more real-time patient measurements, collected by one or more vital monitoring sensors, reaching a threshold; in response to detecting the supply is needed, identify, via a second movable end-effector sensor on the supply monitoring surgical robot, a position where the supply is needed in an operating area proximate to the staging area, wherein the supply monitoring surgical robot is configured to move the first and/or second movable end-effector sensors to analyze areas of interest; control, by the supply monitoring surgical robot, a second manipulator to move the supply from the staging area to the position in the operating area; detect, via a third movable end-effector sensor on the supply monitoring surgical robot, that the step is complete; and in response to detecting the step is complete, control, by the supply monitoring surgical robot, a third manipulator to remove the supply from the position in the operating area. ### Claim 50 (depends on 49) The non-transitory computer readable storage medium of claim 49, wherein the computer instructions, which when executed by the one or more computer processors further cause the one or more computer processors to: identify, by the supply monitoring surgical robot, a supply fault based on a mismatch between a first number of supplies in the supply tray and a second number of supplies in the staging area, the operating area, and a receiving area; and display, by the supply monitoring surgical robot, a notification based on the supply fault. ### Claim 51 (depends on 49) The non-transitory computer readable storage medium of claim 49, wherein the first manipulator is located outside a sterile field and the second manipulator is located inside the sterile field. ### Claim 52 (depends on 49) The non-transitory computer readable storage medium of claim 49, wherein the first movable end-effector sensor is a microphone and the microphone detects an instruction spoken by a medical practitioner. ### Claim 53 (depends on 49) The non-transitory computer readable storage medium of claim 49, wherein the first movable end-effector sensor is a camera and the camera captures an image, wherein the computer instructions, which when executed by the one or more computer processors further cause the one or more computer processors to perform image recognition on the image to determine that the image indicates the supply. ### Claim 54 (depends on 53) The non-transitory computer readable storage medium of claim 53, wherein the image recognition is performed using a machine learning model trained on a plurality of images to generate a determination that the image indicates the supply.