# US 11,896,315 B2 · Virtual reality system with customizable operation room > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,896,315 B2 - Document ID: US11896315B2 - Status: Granted - Kind: B2 - Application number: 17/839,153 - Filed: 2022-06-13 - Published: 2022-09-22 - Granted: 2024-02-13 - Earliest claimed: 2019-10-29 - Assignees: Verb Surgical Inc. - Applicants: Verb Surgical Inc. - Inventors: Bernhard Fuerst; Eric Johnson; Pablo Garcia Kilroy - Web page: https://patentgrep.com/patent/US11896315B2 ## CPC - A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations - G16H 50/50: for simulation or modelling of medical disorders - A61B 34/25: {User interfaces for surgical systems} - A61B 34/37: Leader-follower robots (A61B34/35 takes precedence) - A61B 34/74: {Manipulators with manual electric input means} - A61B 90/361: {Image-producing devices, e.g. surgical cameras} - G06T 15/20: Perspective computation - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture - G16H 40/63: for local operation - A61B 34/30: Surgical robots - A61B 2034/102: {Modelling of surgical devices, implants or prosthesis} - A61B 2034/104: {Modelling the effect of the tool, e.g. the effect of an implanted prosthesis or for predicting the effect of ablation or burring} - A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones} - A61B 2034/107: {Visualisation of planned trajectories or target regions} - A61B 2090/502: {Headgear, e.g. helmet, spectacles} - G06T 2210/41: Medical - G06T 2219/2016: Rotation, translation, scaling - G06T 2219/2021: Shape modification - G16H 30/40: for processing medical images, e.g. editing - A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A virtual operating room (OR) is generated that includes virtual surgical equipment based on existing OR models and existing equipment models. Sensor feedback is received that defines a physical OR having physical equipment, the physical equipment including a surgical robotic system within the physical OR. The virtual OR and the virtual surgical equipment is updated based on the sensor feedback. A layout of the virtual surgical equipment is optimized in the virtual OR. The virtual OR and the virtual surgical equipment are rendered on a display. ## Claims ### Claim 1 A method performed by one or more processors, comprising: processing one or more images from a camera to determine a layout of a physical surgical equipment within a physical operating room (OR); generating a virtual surgical equipment within a virtual OR with a layout that is matched to the layout of the physical surgical equipment within the physical OR; determining a compatibility determination for the layout of the virtual surgical equipment within the virtual OR, wherein the compatibility determination indicates whether the virtual surgical equipment may be reliably moved in and out of the virtual OR; responsive to the compatibility determination indicating that the virtual surgical equipment cannot be reliably moved in and out of the virtual OR, changing a geometry of the virtual OR to allow the virtual surgical equipment to be reliably moved in and out of the virtual OR; and presenting the virtual OR to a head-mounted display (HMD) as a three-dimensional (3D) virtual reality (VR) simulation. ### Claim 2 (depends on 1) The method of claim 1, wherein the compatibility determination is determined based on a clearance between the virtual surgical equipment in the virtual OR. ### Claim 3 (depends on 1) The method of claim 1, wherein the compatibility determination is determined based on a clearance between the virtual surgical equipment and a virtual person in the virtual OR. ### Claim 4 (depends on 1) The method of claim 1, wherein the compatibility determination is determined based on comparing a size of the virtual surgical equipment with an entrance of the virtual OR. ### Claim 5 (depends on 1) The method of claim 1, wherein the compatibility determination is determined based on comparing a size of the virtual surgical equipment with a pathway of the virtual OR. ### Claim 6 (depends on 1) The method of claim 1, further comprising adjusting a location or an orientation of the virtual surgical equipment in response to input from a user from within the 3D VR simulation. ### Claim 7 (depends on 6) The method of claim 6, wherein moving the virtual surgical equipment in response to the input from the user includes adjusting an angle or height of a surgical platform. ### Claim 8 (depends on 6) The method of claim 6, wherein moving the virtual surgical equipment in response to the input from the user includes adjusting a location or orientation of a robotic arm. ### Claim 9 (depends on 1) The method of claim 1, wherein processing the one or more images from the camera to determine the layout of the physical surgical equipment within the physical OR includes using computer vision to detect the physical surgical equipment and its position in the physical OR. ### Claim 10 A system, comprising: a head mounted display (HMD) worn by a user to move through a three-dimensional (3D) virtual reality (VR) simulation; and a processor, configured to: process one or more images from a camera to determine a layout of a physical surgical equipment within a physical operating room (OR); generate a virtual surgical equipment within a virtual OR with a layout that is matched to the layout of the physical surgical equipment within the physical OR; determine a compatibility determination for the layout of the virtual surgical equipment within the virtual OR, wherein the compatibility determination indicates whether the virtual surgical equipment may be reliably moved in and out of the virtual OR; responsive to the compatibility determination indicating that the virtual surgical equipment cannot be reliably moved in and out of the virtual OR, changing a geometry of the virtual OR to allow the virtual surgical equipment to be reliably moved in and out of the virtual OR; and present the virtual OR in the VR simulation to the HMD. ### Claim 11 (depends on 10) The system of claim 10, wherein the compatibility determination is determined based on a clearance between the virtual surgical equipment in the virtual OR. ### Claim 12 (depends on 10) The system of claim 10, wherein the compatibility determination is determined based on a clearance between the virtual surgical equipment and a virtual person in the virtual OR. ### Claim 13 (depends on 10) The system of claim 10, wherein the compatibility determination is determined based on comparing a size of the virtual surgical equipment with an entrance of the virtual OR. ### Claim 14 (depends on 10) The system of claim 10, wherein the compatibility determination is determined based on comparing a size of the virtual surgical equipment with a pathway of the virtual OR. ### Claim 15 (depends on 10) The system of claim 10, wherein the processor is further to adjust a location or an orientation of the virtual surgical equipment in response to input from the user from within the 3D VR simulation. ### Claim 16 (depends on 15) The system of claim 15, wherein moving the virtual surgical equipment in response to the input from the user includes adjusting an angle or height of a surgical platform. ### Claim 17 (depends on 15) The system of claim 15, wherein moving the virtual surgical equipment in response to the input from the user includes adjusting a location or orientation of a robotic arm. ### Claim 18 (depends on 10) The system of claim 10, wherein processing the one or more images from the camera to determine the layout of the physical surgical equipment within the physical OR includes using computer vision to detect the physical surgical equipment and its position in the physical OR.