# US 2023/0264365 A1 · Information processing apparatus, information processing method, and computer program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0264365 A1
- Document ID: US20230264365A1
- Status: Published application
- Kind: A1
- Application number: 18/006,912
- Filed: 2021-06-22
- Published: 2023-08-24
- Earliest claimed: 2020-08-17
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: KEISUKE MAEDA
- Web page: https://patentgrep.com/patent/US20230264365A1

## CPC

- B25J 9/1612: {characterised by the hand, wrist, grip control}
- B25J 13/08: by means of sensing devices, e.g. viewing or touching devices
- B25J 9/1661: {characterised by task planning, object-oriented languages}
- B25J 15/10: with three or more finger members {(B25J15/0009 takes precedence)}
- G05B 2219/39539: Plan hand shape
- G05B 2219/39543: Recognize object and plan hand shapes in grasping movements
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)

## Abstract

Provided is an information processing apparatus that processes information for implementing the best gripping operation suitable for a characteristic of a gripper. The information processing apparatus includes: a grip information analysis unit that analyzes an object and generates generalized grip shape information; and a generalized grip shape information accumulation unit that accumulates the generalized grip shape information. The information processing apparatus further includes a grip objective planning unit that plans a grip objective of the object on the basis of the generalized grip shape information of the object and a characteristic of a gripper to be used for a grip. The grip objective planning unit plans the grip objective regarding how to grip which portion of the object.

## Claims

### Claim 1

An information processing apparatus comprising:
a grip information analysis unit that analyzes an object and generates generalized grip shape information; and
a generalized grip shape information accumulation unit that accumulates the generalized grip shape information.

### Claim 2 (depends on 1)

The information processing apparatus according to claim 1, wherein
the grip information analysis unit analyzes information of the object including at least one of a position, a posture, a shape, an estimated material, strength, a weight, or a centroid position of the object, and generates the generalized grip shape information for each object type.

### Claim 3 (depends on 1)

The information processing apparatus according to claim 1, wherein
the grip information analysis unit divides a surface shape of the object into a plurality of flat surfaces or curved surfaces, and generates the generalized grip shape information including at least one of flatness, friction, strength, normal line information, or an opening distance calculated for each of the divided surfaces.

### Claim 4 (depends on 2)

The information processing apparatus according to claim 2, wherein
the grip information analysis unit estimates the centroid position of the object on an assumption of a uniform material from the shape, a volume, or an appearance of the object.

### Claim 5 (depends on 2)

The information processing apparatus according to claim 2, wherein
the grip information analysis unit obtains centroids respectively for estimated materials of the object including a plurality of materials and estimates the centroid position by taking a weighted average of the centroids.

### Claim 6 (depends on 3)

The information processing apparatus according to claim 3, wherein
the grip information analysis unit extracts the surface shape of the object, divides the surface shape of the object into the plurality of flat surfaces or curved surfaces, and, for each of the divided surfaces, estimates the flatness and a normal direction of the surface on a basis of a variance of elements constituting the surface, estimates an opening distance in the normal direction of the divided surface on a basis of the normal direction, and estimates the friction and the strength of the object on a basis of the flatness, reflected light, or an estimated material to generate the generalized grip shape information.

### Claim 7 (depends on 1)

The information processing apparatus according to claim 1, further comprising
a grip objective planning unit that plans a grip objective of the object on a basis of the generalized grip shape information of the object and a characteristic of a gripper to be used for a grip.

### Claim 8 (depends on 7)

The information processing apparatus according to claim 7, wherein
the grip objective planning unit plans the grip objective regarding how to grip which portion of the object.

### Claim 9 (depends on 7)

The information processing apparatus according to claim 7, wherein
the grip objective planning unit plans the grip objective including at least one of a contact surface, a contact direction, or a contact force of each of fingers of the gripper that comes into contact with the object, the gripper gripping the object.

### Claim 10 (depends on 7)

The information processing apparatus according to claim 7, wherein
the grip objective planning unit selects a divided surface to be a grip surface candidate from among a plurality of divided surfaces obtained by dividing a surface shape of the object, which is an object to be gripped and determines a combination of the grip surface and a finger of the gripper from among a plurality of the grip surface candidates to determine the grip objective.

### Claim 11 (depends on 10)

The information processing apparatus according to claim 10, wherein
the grip objective planning unit evaluates a suitability of a grip of each of the divided surfaces of the object, which is the object to be gripped, on a basis of a characteristic of the finger of the gripper to be used for gripping the object, and removes a divided surface that is not suitable for the grip from the candidates.

### Claim 12 (depends on 11)

The information processing apparatus according to claim 11, wherein
in a case where the gripper has a plurality of the fingers having different characteristics, the grip objective planning unit evaluates the suitability of the grip of each of the divided surfaces of the object on a basis of the characteristics of the respective fingers.

### Claim 13 (depends on 11)

The information processing apparatus according to claim 11, wherein
the grip objective planning unit evaluates the suitability of the grip of each of the divided surfaces of the object using at least one of pieces of following (a) to (d) as an evaluation item:
(a) a thickness of the finger of the gripper and an opening distance of the divided surface in a normal direction;
(b) a friction coefficient of the finger and an estimated friction coefficient of the divided surface, a gripping force, and slipperiness of the finger with respect to the divided surface based on an estimated weight of the object;
(c) an elastic modulus of the finger and flatness of the divided surface; and
(d) an angle formed by a vector from an estimated centroid position of the object, which is the object to be gripped, to the divided surface and the normal direction of the divided surface.

### Claim 14 (depends on 10)

The information processing apparatus according to claim 10, wherein
the grip objective planning unit determines a best combination of the grip surface and the finger of the gripper on a basis of a vector of a gripping force (including a frictional force) generated by each of the fingers gripping the object.

### Claim 15 (depends on 7)

The information processing apparatus according to claim 7, wherein
in a case where any of a plurality of the grippers is selected to grip the object,
the grip objective planning unit plans the grip objective for each of the plurality of grippers, and
selects a gripper to be used for gripping the object on a basis of a result of comparing the grip objectives of the respective grippers.

### Claim 16 (depends on 15)

The information processing apparatus according to claim 15, wherein
the gripper to be used for gripping the object is selected on a basis of a grip index value including at least one of grip stability when the grip objective of each of the grippers is executed, an expected success rate, or an energy consumption amount.

### Claim 17 (depends on 7)

The information processing apparatus according to claim 7, wherein
the grip objective planning unit takes the generalized grip shape information regarding the object, which is an object to be gripped, from the generalized grip shape information accumulation unit and plans the grip objective.

### Claim 18 (depends on 1)

The information processing apparatus according to claim 1, further comprising:
an interface configured to specify the object which is an object to be gripped;
an operation planning unit that plans operations of a manipulator and a gripper on a basis of the grip objective planned by the grip objective planning unit;
a controller that generates a control value for driving an actuator on a basis of an operation plan planned by the operation planning unit; and
the actuator that is driven according to the control value generated by the controller to operate the manipulator and the gripper.

### Claim 19

An information processing method comprising:
a grip information analysis step of analyzing an object and generating generalized grip shape information; and
a generalized grip shape information accumulation step of accumulating the generalized grip shape information in a generalized grip shape information accumulation unit.

### Claim 20

A computer program recorded in a computer-readable format to cause a computer to function as:
a grip information analysis unit that analyzes an object and generates generalized grip shape information; and
a generalized grip shape information accumulation unit that accumulates the generalized grip shape information.
