# US 2019/0379836 A1 · Sensor control device, object search system, object search method, and program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0379836 A1
- Document ID: US20190379836A1
- Status: Published application
- Kind: A1
- Application number: 16/395,300
- Filed: 2019-04-26
- Published: 2019-12-12
- Earliest claimed: 2018-06-06
- Assignees: Toyota Jidosha Kabushiki Kaisha
- Applicants: Toyota Jidosha Kabushiki Kaisha
- Inventors: Yuto MORI; Shintaro YOSHIZAWA; Kei YOSHIKAWA
- Web page: https://patentgrep.com/patent/US20190379836A1

## CPC

- H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
- G06T 7/62: of area, perimeter, diameter or volume
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 2207/30244: Camera pose
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

An information acquisition unit 122 acquires three-dimensional environmental information from a three-dimensional environmental information storage unit 114. A placement region detection unit 126 uses the three-dimensional environmental information to detect a placement region where a target object 80 is placed. A calculation unit 130 calculates a size of a placement region in an image including the placement region. A viewpoint position determination unit 134 determines, among a plurality of sensor positions where a sensor 12 photographing the image related to image data can be placed, a sensor position where the placement region is larger than the placement region in the case of using image data of another sensor position, as an optimal viewpoint position. A sensor position control unit 142 performs control so as to move the sensor 12 to the optimal viewpoint position, and then performs control so as to start a search for the target object 80.

## Claims

### Claim 1

A sensor control device configured to control a sensor searching for a target object to be searched for in a three-dimensional environment, comprising:
hardware, including at least one memory configured to store a computer program and at least one processor configured to execute the computer program;
a three-dimensional environmental information acquisition unit, implemented by the hardware, configured to acquire three-dimensional environmental information indicating each point on an object present in the three-dimensional environment;
a region detection unit, implemented by the hardware, configured to detect a placement region where the target object is placed by using the three-dimensional environmental information;
a calculation unit, implemented by the hardware, configured to calculate a size of the placement region in at least one image data representing an image including the placement region;
a position determination unit, implemented by the hardware, configured to determine, among a plurality of sensor positions where the sensor photographing an image related to the image data can be placed, a sensor position where the placement region is larger than the placement region in the case of using image data of another sensor position, as an optimal position; and
a sensor position control unit, implemented by the hardware, configured to perform control, when a search for the target object is made after the optimal position has been determined, so as to move the sensor to the optimal position and then start the search for the target object.

### Claim 2 (depends on 1)

The sensor control device according to claim 1, wherein
the calculation unit calculates a size of the placement region in two or more of the image data indicating an image including the placement region, and
the position determination unit determines, among the two or more of the image data, the sensor position where an image related to the image data having the largest size of the placement region can be photographed as the optimal position.

### Claim 3 (depends on 2)

The sensor control device according to claim 2, further comprising an image extraction unit, implemented by the hardware, configured to extract an image including the placement region, wherein
the calculation unit calculates a size of the placement region in all the image data that can be extracted by the image extraction unit, the image data representing the image including the placement region, and
the position determination unit determines, among all the image data, the sensor position where an image related to the image data having the largest size of the placement region can be photographed as the optimal position.

### Claim 4 (depends on 1)

The sensor control device according to claim 1, wherein the position determination unit determines, as the optimal position, the sensor position where an image related to the image data in which a size of the placement region is a predetermined threshold value or greater can be photographed.

### Claim 5 (depends on 1)

The sensor control device according to claim 1, wherein the placement region corresponds to a plane where the target object is placed.

### Claim 6 (depends on 1)

The sensor control device according to claim 1, wherein the placement region corresponds to a space region where the target object is placed.

### Claim 7

An object search system, comprising:
a sensor configured to search for a target object to be searched for in a three-dimensional environment; and a sensor control device configured to control the sensor, the sensor control device comprising:
hardware, including at least one memory configured to store a computer program and at least one processor configured to execute the computer program;
a three-dimensional environmental information acquisition unit, implemented by the hardware, configured to acquire three-dimensional environmental information indicating each point on an object present in the three-dimensional environment;
a region detection unit, implemented by the hardware, configured to detect a placement region where the target object is placed by using the three-dimensional environmental information;
a calculation unit, implemented by the hardware, configured to calculate a size of the placement region in at least one image data representing an image including the placement region;
a position determination unit, implemented by the hardware, configured to determine, among a plurality of sensor positions where the sensor photographing an image related to the image data can be placed, a sensor position where the placement region is larger than the placement region in the case of using image data of another sensor position, as an optimal position; and
a sensor position control unit, implemented by the hardware, configured to perform control, when a search for the target object is made after the optimal position has been determined, so as to move the sensor to the optimal position and then start the search for the target object.

### Claim 8

An object search method for searching for a target object to be searched for in a three-dimensional environment, comprising:
acquiring three-dimensional environmental information indicating each point on an object present in the three-dimensional environment;
detecting a placement region where the target object is placed by using the three-dimensional environmental information;
calculating a size of the placement region in at least one image data representing an image including the placement region;
determining, among a plurality of sensor positions where the sensor photographing an image related to the image data can be placed, a sensor position where the placement region is larger than the placement region in the case of using image data of another sensor position, as an optimal position; and
controlling the sensor, when a search for the target object is made after the optimal position has been determined, so as to move the sensor to the optimal position and then start the search for the target object.

### Claim 9

A non-transitory computer readable medium storing a program for performing an object search method in which a target object to be searched for is searched for in a three-dimensional environment, the program being adapted to cause a computer to perform:
acquiring three-dimensional environmental information indicating each point on an object present in the three-dimensional environment;
detecting a placement region where the target object is placed by using the three-dimensional environmental information;
calculating a size of the placement region in at least one image data representing an image including the placement region;
determining, among a plurality of sensor positions where the sensor photographing an image related to the image data can be placed, a sensor position where the placement region is larger than the placement region in the case of using image data of another sensor position, as an optimal position; and
controlling the sensor, when a search for the target object is made after the optimal position has been determined, so as to move the sensor to the optimal position and then start the search for the target object.
