# US 12,223,092 B2 · Modifying data from a surgical robotic system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,223,092 B2
- Document ID: US12223092B2
- Status: Granted
- Kind: B2
- Application number: 17/626,312
- Filed: 2020-07-10
- Published: 2022-09-08
- Granted: 2025-02-11
- Earliest claimed: 2019-07-11
- Assignees: Cmr Surgical Limited
- Applicants: Cmr Surgical Limited
- Inventors: Steven Bishop; Stephen Bell; Ricardo Michael Henderson Da Silva
- Web page: https://patentgrep.com/patent/US12223092B2

## CPC

- G06F 21/6254: {by anonymising data, e.g. decorrelating personal data from the owner's identification}
- A61B 34/35: for telesurgery
- A61B 34/37: Leader-follower robots (A61B34/35 takes precedence)
- G06N 20/00: Machine learning
- G16H 10/60: for patient-specific data, e.g. for electronic patient records
- G16H 15/00: ICT specially adapted for medical reports, e.g. generation or transmission thereof
- G16H 20/40: relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
- G16H 30/20: for handling medical images, e.g. DICOM, HL7 or PACS
- G16H 30/40: for processing medical images, e.g. editing
- G16H 40/67: for remote operation
- H04L 9/0894: {Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage}
- H04L 63/0407: {wherein the identity of one or more communicating identities is hidden (cryptographic mechanisms or cryptographic arrangements for anonymous credentials or for identity based cryptographic systems H04L9/00)}
- H04L 63/0421: {Anonymous communication, i.e. the party's identifiers are hidden from the other party or parties, e.g. using an anonymizer}
- H04L 63/0428: {wherein the data content is protected, e.g. by encrypting or encapsulating the payload}
- H04L 67/125: involving control of end-device applications over a network
- H04N 19/136: Incoming video signal characteristics or properties
- A61B 2034/301: {for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes}
- H04L 2209/42: Anonymization, e.g. involving pseudonyms
- H04L 2209/88: Medical equipments
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A method of reducing a data volume of a data stream in a surgical robotic system, the surgical robotic system comprising a robot having a base and an arm extending from the base to an attachment for an instrument, the arm comprising a plurality of joints whereby the configuration of the arm can be altered, the method comprising: receiving a data stream captured by the surgical robotic system, the data stream comprising data relating to a surgical procedure and having a first data volume; identifying a feature in the received data stream indicative of an event in the surgical procedure; and modifying, in dependence on the identified feature, the received data stream to generate a modified data stream having a second data volume that is smaller than the first data volume.

## Claims

### Claim 1

A method of controlling a data modifier to reduce overall a data volume of a data stream captured by a surgical robotic system, the surgical robotic system comprising a robot having a base and an arm extending from the base to an attachment for an instrument, the arm comprising a plurality of joints whereby the configuration of the arm can be altered, the method comprising:
receiving a data stream captured by the surgical robotic system, the data stream comprising data relating to a surgical procedure and having a first data volume;
identifying a feature in the received data stream indicative of an event in the surgical procedure;
in dependence on the identified feature, determining selected data within the received data stream for which data fidelity is to be retained;
in dependence on the determined selected data, controlling a data modifier to move between:
an active state in which the data modifier is arranged to modify the received data stream to generate a modified data stream, and
an inactive state in which the data modifier is arranged not to modify the received data stream, thereby leaving the received data stream unmodified;
such that the data modifier selectively modifies in dependence on the identified feature, the received data stream to generate a modified data stream having a second data volume that is smaller than the first data volume whilst retaining the data fidelity of the determined data; and
outputting the modified data stream.

### Claim 2 (depends on 1)

A method according to claim 1, comprising generating the modified data stream by modifying one or more of:
a first data portion of the received data stream before the identified feature; and
a second data portion of the received data stream after the identified feature.

### Claim 3 (depends on 2)

A method according to claim 2, in which one or both of the first data portion and the second data portion relate to predetermined time periods.

### Claim 4 (depends on 1)

A method according to claim 1, comprising identifying a further feature in the received data stream indicative of a further event in the surgical procedure, and generating the modified data stream by modifying a further data portion relating to a further time period of the received data stream:
between the identified feature and the identified further feature; or
before the first of the identified feature and the identified further feature and after the second of the identified feature and the identified further feature.

### Claim 5 (depends on 1)

A method according to claim 1, in which identifying the feature comprises determining whether a rule of a set of rules is satisfied and identifying the feature where the rule is satisfied.

### Claim 6 (depends on 5)

A method according to claim 5, in which the rule is satisfied where a parameter of a group of parameters of the surgical robotic system and/or the surgical procedure matches a predefined criterion.

### Claim 7 (depends on 6)

A method according to claim 6, in which the received data stream comprises the parameter or in which the parameter is determined from analysis of the received data stream.

### Claim 8 (depends on 1)

A method according to claim 1, in which the received data stream comprises one or more data channel from a group of data channels comprising:
video data received from an endoscope coupled to the surgical robotic system;
audio data recorded in respect of the surgical procedure;
telematics data corresponding to the surgical robotic system;
state data comprising the state of at least a portion of the surgical robotic system; and
further data transmitted by additional devices on the local network;
and in which the method comprises identifying the feature in dependence on one or more data channel from the group of data channels.

### Claim 9 (depends on 8)

A method according to claim 8, in which the method comprises identifying the feature in dependence on a first data channel of the group of data channels, and generating the modified data stream comprises modifying one or more of:
the first data channel and
a second data channel of the group of data channels.

### Claim 10 (depends on 8)

A method according to claim 8, in which the method comprises generating the modified data stream by modifying one or more of:
a video definition of at least a portion of the video data;
a number of video channels of at least a portion of the video data;
a frame rate of at least a portion of the video data;
a colour gamut of at least a portion of the video data;
a representation of the state of the robotic system;
an audio definition of at least a portion of the audio data;
a number of data channels in the group of data channels.

### Claim 11 (depends on 8)

A method according to claim 8, in which the method comprises modifying the received data stream such that the modified data stream comprises one or more of:
a portion of video data having a first resolution and a portion of video data having a second, lower, resolution;
a portion of video data having a first number of channels and a portion of video data having a second, smaller, number of channels;
a portion of video data having a first frame rate and a portion of video data having a second, lower, frame rate;
a portion of video data having a first colour gamut and a portion of video data having a second, smaller, colour gamut;
data relating to changes in state of the robotic system in place of data relating to the state of the robotic system at a plurality of points in time between which the state does not change;
a wire frame model of a known shape in the video data in place of full video or graphics render data relating to that known shape;
a description of a structure in the video data in place of full video data relating to that structure; and
a description of an absolute location and/or a relative location, and a description of a speed and/or a trajectory of movement, in place of full video data or full telematics data.

### Claim 12 (depends on 11)

A method according to claim 11, in which the description of the structure comprises a description of one or more of an anatomical structure, an instrument, and an end effector.

### Claim 13 (depends on 1)

A method according to claim 1, in which the identified feature is indicative of a value of or change in one or more of:
an attachment state of an instrument;
an operational state of an instrument attached to the arm;
an operational state or operational mode of the robotic system;
a configuration of the arm and/or of an instrument attached to the arm; and
a control state of the control console.

### Claim 14 (depends on 1)

A method according to claim 1, in which the identified feature is indicative of one or more of:
a feature detected in the video data;
sensor data; and
data reported by an externally connected system.

### Claim 15 (depends on 1)

A method according to claim 1, in which the method comprises generating the modified data stream by compressing at least a portion of one or more data channel from the group of data channels.

### Claim 16 (depends on 1)

A method according to claim 1, in which the method comprises generating the modified data stream as the surgical procedure is being performed.

### Claim 17 (depends on 1)

A method according to claim 1, in which the method comprises:
generating the modified data stream in real time or substantially real time;
sending the modified data stream to a remote processor thereby enabling the remote processor to perform real time or substantially real time analysis of the modified data stream; and
receiving from the remote processor in real time or substantially real time the result of the analysis for assisting an operator of the surgical robotic system.

### Claim 18 (depends on 1)

A computer readable-storage medium having stored thereon computer readable instructions that, when executed at a computer system, cause the computer system to perform the method according to claim 1.

### Claim 19

A data volume modification system for a surgical robotic system for controlling a data modifier to reduce overall a data volume of a data stream in the surgical robotic system, the surgical robotic system comprising a robot having a base and an arm extending from the base to an attachment for an instrument, the arm comprising a plurality of joints whereby the configuration of the arm can be altered, the system comprising:
a receiver configured to receive a data stream captured by the surgical robotic system, the data stream comprising data relating to a surgical procedure and having a first data volume;
a feature detector configured to identify a feature in the received data stream indicative of an event in the surgical procedure;
a data modifier configured to selectively modify the received data stream to generate a modified data stream; and
an output configured to output the modified data stream;
wherein the system is configured to, in dependence on the identified feature, determine selected data within the received data stream for which data fidelity is to be retained and, in dependence on the determined selected data, to control the data modifier to move between:
an active state in which the data modifier is arranged to modify the received data stream to generate the modified data stream, where the modified data stream has a second data volume that is smaller than the first data volume whilst retaining the data fidelity of the determined data, and
an inactive state in which the data modifier is arranged not to modify the received data stream, thereby leaving the received data stream unmodified.