# US 2023/0067515 A1 · Systems and methods for grasp adjustment based on grasp properties

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0067515 A1
- Document ID: US20230067515A1
- Status: Published application
- Kind: A1
- Application number: 17/982,343
- Filed: 2022-11-07
- Published: 2023-03-02
- Earliest claimed: 2018-04-03
- Assignees: Intuitive Surgical Operations, Inc.
- Applicants: Intuitive Surgical Operations, Inc.
- Inventors: Ashley Lynne WELLMAN; David W. WEIR; Matthew A. WIXEY
- Web page: https://patentgrep.com/patent/US20230067515A1

## CPC

- A61B 17/282: {Jaws}
- A61B 17/29: Forceps for use in minimally invasive surgery
- A61B 17/22031: {Gripping instruments, e.g. forceps, for removing or smashing calculi (A61B17/221 takes precedence)}
- A61B 34/30: Surgical robots
- A61B 34/35: for telesurgery
- A61B 34/76: {Manipulators having means for providing feel, e.g. force or tactile feedback}
- A61B 34/25: {User interfaces for surgical systems}
- A61B 34/77: {Manipulators with motion or force scaling}
- A61B 90/03: {Automatic limiting or abutting means, e.g. for safety}
- A61B 90/361: {Image-producing devices, e.g. surgical cameras}
- A61B 90/37: {Surgical systems with images on a monitor during operation}
- A61B 2017/2926: {Details of heads or jaws}
- A61B 2034/2065: {Tracking using image or pattern recognition}
- A61B 2034/305: {Details of wrist mechanisms at distal ends of robotic arms}
- A61B 2090/031: {torque limiting}
- A61B 2090/064: {for measuring force, pressure or mechanical tension}
- A61B 2090/066: {for measuring torque}
- A61B 2090/3937: {Visible markers}
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)

## Abstract

Techniques for grasp adjustment include a computer-assisted device comprising a repositionable structure configured to support an end effector and one or more processors. The one or more processors are configured to receive one or more images of the end effector; determine, based on the one or more images, at least one of a first length between a proximal end of jaws of the end effector and a proximal end of a grasping zone, a second length corresponding to a length of the grasping zone, a third length between a distal end of the grasping zone and the distal end of the at least one jaw; or an angle between the jaws of the end effector; and adjust a force or a torque magnitude limit used to limit actuation of the end effector based on at least one of the first length, the second length, the third length, or the angle.

## Claims

### Claim 1

A computer-assisted device comprising:
a repositionable structure configured to support an end effector; and
one or more processors;
wherein the one or more processors are configured to:
receive one or more images of the end effector;
determine, based on the one or more images, at least one of:
a first length between a proximal end of jaws of the end effector and a proximal end of a grasping zone, wherein the grasping zone is where at least one jaw and a grasped material are in contact;
a second length corresponding to a length of the grasping zone;
a third length between a distal end of the grasping zone and the distal end of the at least one jaw; or
an angle between the jaws of the end effector; and
adjust a force magnitude limit or a torque magnitude limit used to limit actuation of the end effector based on at least one of the first length, the second length, the third length, or the angle.

### Claim 2 (depends on 1)

The computer-assisted device of claim 1, wherein the one or more processors are further configured to perform one or more edge detection operations at one or more predetermined directions or a combination thereof on the one or more images.

### Claim 3 (depends on 1)

The computer-assisted device of claim 1, wherein the one or more processors are further configured to perform one or more blob detection operations of one or more predetermined sizes or a combination thereof on the one or more images.

### Claim 4 (depends on 1)

The computer-assisted device of claim 1, wherein the one or more processors are further configured to perform one or more morphological operations using predetermined structural elements of one or more sizes and shapes or a combination thereof on the one or more images, on edge detected one or more images, or on blob detected one or more images.

### Claim 5 (depends on 1)

The computer-assisted device of claim 1, wherein the one or more processors are further configured to segment the one or more images.

### Claim 6 (depends on 5)

The computer-assisted device of claim 5, wherein the one or more processors are further configured to extract features from the segmented one or more images.

### Claim 7 (depends on 6)

The computer-assisted device of claim 6, wherein the one or more processors are further configured to generate one or more recognized objects from the extracted features, wherein the one or more recognized objects are generated based on a structural recognition method or a statistical recognition method.

### Claim 8 (depends on 7)

The computer-assisted device of claim 7, wherein the one or more processors are further configured to:
determine a mean value and an acceptable range of variation for each extracted feature;
assign a probability distribution function to the acceptable range of variation for each extracted feature, wherein a probability of the extracted feature is highest when the extracted feature is at its mean value and the probability gets smaller as it gets further away from the mean value, wherein the probability becomes zero at ends of the acceptable range;
determine a probability of one or more extracted features in a first one of the recognized objects; and
determine a score of the first one of the recognized objects using one of multiplying the probabilities of the extracted features, a weighted sum of the probabilities of the extracted features, or a combination thereof.

### Claim 9 (depends on 8)

The computer-assisted device of claim 8, wherein the one or more processors are further configured to:
assign a grammar of how the extracted features are related to each other;
wherein:
a probability is determined for a relation of the extracted features; and
the score is determined using one of multiplying the probabilities of the extracted features and the probability of the relation of the extracted features, a weighted sum of the probabilities of the extracted features and the probability of the relation of the extracted features, or a combination thereof.

### Claim 10 (depends on 1)

The computer-assisted device of claim 1, wherein the one or more processors are further configured to:
receive a lookup table of predetermined objects;
perform normalized correlation of elements of the lookup table and the one or more images; and
determine one or more recognized objects.

### Claim 11 (depends on 10)

The computer-assisted device of claim 10, wherein:
the one or more processors are further configured to determine a score corresponding to each of the one or more recognized objects;
the one or more recognized objects are determined based on comparing the scores; and
the score is a measure of confidence for a recognition of a predetermined object in the one or more images.

### Claim 12

A method comprising, receiving, by one or more processors, one or more images of an end effector;
determining, by the one or more processors based on the one or more images, at least one of:
a first length between a proximal end of jaws of the end effector and a proximal end of a grasping zone, wherein the grasping zone is where at least one jaw and a grasped material are in contact;
a second length corresponding to a length of the grasping zone;
a third length between a distal end of the grasping zone and the distal end of the at least one jaw; or
an angle between the jaws of the end effector; and
adjusting, by the one or more processors, a force magnitude limit or a torque magnitude limit used to limit actuation of the end effector based on at least one of the first length, the second length, the third length, or the angle.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
segmenting, by the one or more processors, the one or more images by performing one or more of edge detection, blob detection, or morphological filtering; and
extracting, by the one or more processors, features from the segmented one or more images.

### Claim 14 (depends on 12)

The method of claim 12, further comprising:
receiving, by the one or more processors, a lookup table of predetermined objects;
performing, by the one or more processors, normalized correlation of elements of the lookup table and the one or more images; and
determining, by the one or more processors, one or more recognized objects.

### Claim 15 (depends on 14)

The method of claim 14, further comprising:
determining, by the one or more processors, a mean value and an acceptable range of variation for each extracted feature;
assigning, by the one or more processors, a probability distribution function to the acceptable range of variation for each extracted feature, wherein a probability of the extracted feature is highest when the extracted feature is at its mean value and the probability gets smaller as it gets further away from the mean value, wherein the probability becomes zero at ends of the acceptable range;
determining, by the one or more processors, a probability of one or more extracted features in a first one of the recognized objects; and
determining, by the one or more processors, a score of the first one of the recognized objects using one of multiplying the probabilities of the extracted features, a weighted sum of the probabilities of the extracted features, or a combination thereof.

### Claim 16 (depends on 15)

The method of claim 15, further comprising:
assigning, by the one or more processors, a grammar of how the extracted features are related to each other;
wherein:
a probability is determined for a relation of the extracted features; and
the score is determined using one of multiplying the probabilities of the extracted features and the probability of the relation of the extracted features, a weighted sum of the probabilities of the extracted features and the probability of the relation of the extracted features, or a combination thereof.

### Claim 17

A non-transitory machine-readable medium having stored thereon machine-readable instructions executable to cause a machine to perform a method comprising:
receiving one or more images of an end effector;
determining, based on the one or more images, at least one of:
a first length between a proximal end of jaws of the end effector and a proximal end of a grasping zone, wherein the grasping zone is where at least one jaw and a grasped material are in contact;
a second length corresponding to a length of the grasping zone;
a third length between a distal end of the grasping zone and the distal end of the at least one jaw; or
an angle between the jaws of the end effector; and
adjusting a force magnitude limit or a torque magnitude limit used to limit actuation of the end effector based on at least one of the first length, the second length, the third length, or the angle.

### Claim 18 (depends on 17)

The non-transitory machine-readable medium of claim 17, wherein the method further comprises:
segmenting the one or more images by performing one or more of edge detection, blob detection, or morphological filtering; and
extracting features from the segmented one or more images.

### Claim 19 (depends on 18)

The non-transitory machine-readable medium of claim 18, wherein the method further comprises generating one or more recognized objects from the extracted features, wherein the one or more recognized objects are generated based on a structural recognition method or a statistical recognition method.

### Claim 20 (depends on 17)

The non-transitory machine-readable medium of claim 17, wherein the method further comprises:
receiving a lookup table of predetermined objects;
performing normalized correlation of elements of the lookup table and the one or more images; and
determining one or more recognized objects.
