# US 2020/0394818 A1 · Information processing device, information processing method, and information processing non-transitory computer readable medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394818 A1
- Document ID: US20200394818A1
- Status: Published application
- Kind: A1
- Application number: 16/975,389
- Filed: 2019-02-18
- Published: 2020-12-17
- Earliest claimed: 2018-03-14
- Assignees: OMRON Corporation
- Applicants: OMRON Corporation
- Inventors: Yoshinori KONISHI
- Web page: https://patentgrep.com/patent/US20200394818A1

## CPC

- G06T 7/75: {involving models}
- B25J 9/1605: {Simulation of manipulator lay-out, design, modelling of manipulator}
- B25J 9/1697: {Vision controlled systems}
- B25J 13/08: by means of sensing devices, e.g. viewing or touching devices
- G06T 7/74: {involving reference images or patches}
- G06V 10/751: {Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching}
- G06V 20/64: Three-dimensional [3D] objects
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

The information processing device includes: a first input unit for generating a three-dimensional model of an object to be recognized; a template generation unit that generates, on the basis of the three-dimensional model information, a two-dimensional template which shows the shape of the upper edge of the object to be recognized; a coordinate relationship identification unit that generates coordinate relationship information indicating the relationship between three-dimensional coordinates of the three-dimensional model and two-dimensional coordinates when the three-dimensional model is imaged; a second input unit that receives the input of the captured image; a matching unit that matches the captured image and the two-dimensional template; and a recognition unit that recognizes the three-dimensional position and posture of the object to be recognized by referencing the coordinate relationship information, which is detected according to the results of the two-dimensional matching by the matching unit.

## Claims

### Claim 1

An information processing device, comprising:
a first input unit, configured to receive an input of a three-dimensional model information for generating a three-dimensional model of an object to be recognized;
a template generation unit, configured to generate a two-dimensional template which shows a shape of an upper edge of the object to be recognized on the basis of the three-dimensional model information;
a coordinate relationship identification unit, configured to generate a coordinate relationship information indicating a relationship between three-dimensional coordinates of the three-dimensional model and two-dimensional coordinates when the three-dimensional model is imaged;
a second input unit, configured to receive an input of a captured image obtained by capturing an image of the object to be recognized;
a matching unit, configured to match the captured image with the two-dimensional template; and
a recognition unit, configured to recognize a three-dimensional position and posture of the object to be recognized by referencing the coordinate relationship information with respect to the object to be recognized in the captured image which is detected according to two-dimensional matching results of the matching unit.

### Claim 2

An information processing method, comprising causing an information processing device to perform:
a process of receiving an input of three-dimensional model information for generating a three-dimensional model of an object to be recognized;
a process of generating a two-dimensional template which shows a shape of an upper edge of the object to be recognized on the basis of three-dimensional model information;
a process of generating coordinate relationship information indicating a relationship between three-dimensional coordinates of the three-dimensional model and two-dimensional coordinates when the three-dimensional model is imaged;
a process of receiving an input of a captured image obtained by capturing an image of the object to be recognized;
a process of matching the captured image with the two-dimensional template; and
a process of recognizing a three-dimensional position and posture of the object to be recognized by referencing the coordinate relationship information with respect to the object to be recognized in the captured image which is detected according to two-dimensional matching results.

### Claim 3

An information processing on-transitory computer readable medium, storing a program, for causing an information processing device to execute:
a process of receiving an input of three-dimensional model information for generating a three-dimensional model of an object to be recognized;
a process of generating a two-dimensional template which shows a shape of an upper edge of the object to be recognized on the basis of three-dimensional model information;
a process of generating coordinate relationship information indicating a relationship between three-dimensional coordinates of the three-dimensional model and two-dimensional coordinates when the three-dimensional model is imaged;
a process of receiving an input of a captured image obtained by capturing an image of the object to be recognized;
a process of matching the captured image with the two-dimensional template; and
a process of recognizing a three-dimensional position and posture of the object to be recognized by referencing the coordinate relationship information with respect to the object to be recognized in the captured image which is detected according to two-dimensional matching results.
