# US 12,594,123 B2 · Virtual reality system with customizable operation room

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,594,123 B2
- Document ID: US12594123B2
- Status: Granted
- Kind: B2
- Application number: 18/412,804
- Filed: 2024-01-15
- Published: 2024-07-18
- Granted: 2026-04-07
- Earliest claimed: 2019-10-29
- Assignees: Auris Health, Inc.
- Applicants: Verb Surgical Inc.
- Inventors: Bernhard Fuerst; Eric Johnson; Pablo Garcia Kilroy
- Web page: https://patentgrep.com/patent/US12594123B2

## CPC

- A61B 34/10: Computer-aided planning, simulation or modelling of surgical operations
- G16H 50/50: for simulation or modelling of medical disorders
- A61B 34/25: {User interfaces for surgical systems}
- A61B 34/37: Leader-follower robots (A61B34/35 takes precedence)
- A61B 34/74: {Manipulators with manual electric input means}
- A61B 90/361: {Image-producing devices, e.g. surgical cameras}
- G06T 15/20: Perspective computation
- G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts
- G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
- G16H 40/63: for local operation
- A61B 34/30: Surgical robots
- A61B 2034/102: {Modelling of surgical devices, implants or prosthesis}
- A61B 2034/104: {Modelling the effect of the tool, e.g. the effect of an implanted prosthesis or for predicting the effect of ablation or burring}
- A61B 2034/105: {Modelling of the patient, e.g. for ligaments or bones}
- A61B 2034/107: {Visualisation of planned trajectories or target regions}
- A61B 2090/502: {Headgear, e.g. helmet, spectacles}
- G06T 2210/41: Medical
- G06T 2219/2016: Rotation, translation, scaling
- G06T 2219/2021: Shape modification
- G16H 30/40: for processing medical images, e.g. editing
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

A virtual operating room (OR) is generated that includes virtual surgical equipment based on existing OR models and existing equipment models. Sensor feedback is received that defines a physical OR having physical equipment, the physical equipment including a surgical robotic system within the physical OR. The virtual OR and the virtual surgical equipment is updated based on the sensor feedback. A layout of the virtual surgical equipment is optimized in the virtual OR. The virtual OR and the virtual surgical equipment are rendered on a display.

## Claims

### Claim 1

A method comprising:
generating a virtual operating room (OR) including virtual surgical equipment and a virtual patient that is generated from one or more images of a physical patient;
identifying a surgical procedure to be performed upon the virtual patient within the virtual OR;
using the identified surgical procedure to determine a volume of a workspace inside the virtual patient and a position of a virtual tool in the workspace;
using the position of the virtual tool within the workspace to determine a layout of the virtual surgical equipment that allows a distal end of the virtual tool to reach or move throughout the volume of the workspace; and
displaying the virtual OR including the virtual surgical equipment.

### Claim 2 (depends on 1)

The method of claim 1, wherein the virtual surgical equipment comprises a virtual robotic arm that comprises the virtual tool, wherein using the position of the virtual tool within the workspace to determine the layout of the virtual surgical equipment comprises producing a position of the virtual robotic arm within the virtual OR as output of an optimization process responsive to input based on the position of the virtual tool.

### Claim 3 (depends on 2)

The method of claim 2, wherein the virtual robotic arm is mounted on a virtual table within the virtual OR, the virtual patient being disposed on the virtual table, wherein the virtual surgical equipment comprises other virtual surgical equipment that are separate from the virtual table, wherein the method further comprises producing positions of the other virtual surgical equipment within the virtual OR as output of the optimization process responsive to input based on the position of the virtual robotic arm.

### Claim 4 (depends on 3)

The method of claim 3, wherein the other virtual surgical equipment comprises at least one of a virtual control tower, a virtual table for holding the virtual patient, and a virtual user console.

### Claim 5 (depends on 1)

The method of claim 1 further comprising:
receiving input that changes at least one of the virtual patient, the surgical procedure, and the virtual OR; and
updating the layout of the virtual surgical equipment based on the received input.

### Claim 6 (depends on 1)

The method of claim 1, wherein the virtual OR is displayed through a head-mounted display.

### Claim 7 (depends on 1)

The method of claim 1 further comprising:
determining a compatibility score for the layout of the virtual surgical equipment within the virtual OR; and
displaying the compatibility score on a display along with the virtual OR.

### Claim 8 (depends on 7)

The method of claim 7, wherein the compatibility score of the virtual surgical equipment is based on a comparison between the virtual surgical equipment and an entrance or pathway of the virtual OR.

### Claim 9 (depends on 1)

The method of claim 1, wherein the virtual surgical equipment is displayed within the virtual OR in the determined layout for planning of a surgical robotic workflow.

### Claim 10 (depends on 1)

The method of claim 1, wherein the layout comprises a route of a virtual cable of the virtual surgical equipment, wherein the route of the virtual cable is displayed as a non-traversable zone within the virtual OR.

### Claim 11

A system comprising:
a display; and
at least one processor configured to:
generate a virtual operating room (OR) including virtual surgical equipment and a virtual patient that is generated from one or more images of a physical patient,
identify a surgical procedure to be performed upon the virtual patient within the virtual OR;
using the identified surgical procedure to determine a volume of a workspace inside the virtual patient and a position of a virtual trocar inserted into a body wall of the virtual patient,
use the position of the virtual trocar inserted into the body wall of the virtual patient to determine a layout of the virtual surgical equipment such that a distal end of a virtual tool that is introduced through the virtual trocar while inserted into the body wall maintains reach or movement throughout the volume of the workspace, and
display the virtual OR including the virtual surgical equipment on the display.

### Claim 12 (depends on 11)

The system of claim 11, wherein the virtual surgical equipment comprises a virtual robotic arm that comprises the virtual tool, wherein the at least one processor is configured use the position of the virtual trocar to determine the layout of the virtual surgical equipment by producing a position of the virtual robotic arm within the virtual OR as output of an optimization process responsive to input based on the position of the virtual tool.

### Claim 13 (depends on 12)

The system of claim 12, wherein the virtual surgical equipment comprises other virtual surgical equipment, wherein the at least one processor is further configured to determine positions of the other virtual surgical equipment within the virtual OR as output of the optimization process responsive to input based on the position of the virtual robotic arm.

### Claim 14 (depends on 13)

The system of claim 13, wherein the other virtual surgical equipment comprises at least one of a virtual control tower, a virtual table for holding the virtual patient, and a virtual user console.

### Claim 15 (depends on 11)

The system of claim 11, wherein the at least one processor is further configured to:
receive input that changes at least one of the virtual patient, the surgical procedure, and the virtual OR; and
update the layout of the virtual surgical equipment based on the received input.

### Claim 16 (depends on 11)

The system of claim 11, wherein the at least one processor is to determine the position of the virtual tool by:
determining the position of the virtual trocar that passes through a port in the body wall of the virtual patient based on the surgical procedure; and
determining the reach of the virtual tool within the workspace while introduced through the virtual trocar.

### Claim 17 (depends on 11)

The system of claim 11, wherein the layout comprises a route of a virtual cable of the virtual surgical equipment, wherein the route of the virtual cable is displayed as a non-traversable zone within the virtual OR.

### Claim 18 (depends on 11)

The system of claim 11, wherein the display is a head-mounted display worn by a user to move through a three-dimensional (3D) virtual reality (VR) simulation that includes the virtual OR.

### Claim 19

A non-transitory machine-readable medium having stored therein instructions that, when executed by at least one processor causes the at least one processor to:
generate a virtual operating room (OR) including virtual surgical equipment and a virtual patient that is generated from one or more images of a physical patient;
identifying a surgical procedure to be performed upon the virtual patient within the virtual OR;
using the identified surgical procedure to determine a volume of a workspace within a larger volume inside the virtual patient and a position of a virtual tool in the workspace;
use the position of the virtual tool within the workspace to determine a layout of the virtual surgical equipment that allows a distal end of the virtual tool to reach or move throughout the volume of the workspace; and
display the virtual OR including the virtual surgical equipment.

### Claim 20 (depends on 19)

The non-transitory machine-readable medium of claim 19, wherein the instructions to use the position of the virtual tool within the workspace to determine the layout of the virtual surgical equipment comprises instructions to:
determine a position of a virtual robotic arm within the virtual OR based on the position of the virtual tool; and
determine the positions of the virtual surgical equipment within the virtual OR based on the position of the virtual robotic arm, wherein the virtual surgical equipment comprises at least one of a virtual control tower, a virtual table for holding the virtual patient, and a virtual user console.