Back to results
US 2021/0078166 A1Published application
Display control device, display control method, computer program product, and communication system
AssigneeSony Corporation
InventorsKenta KAWAMOTO, Yoshiaki IWAI, Satoru SHIMIZU
Life of this patent
Estimating the term…
Abstract
A control system, method and computer program product cooperate to assist control for an autonomous robot. An interface receives recognition information from an autonomous robot, said recognition information including candidate target objects to interact with the autonomous robot. A display control unit causes a display image to be displayed on a display of candidate target objects, wherein at least two of the candidate target objects are displayed with an associated indication of a target object score.
Claims20
1:
- A movable apparatus comprising:
- driving circuitry that controllably drives a movement of the movable apparatus;
- a camera configured to capture a captured image and produce image data;
- and
- circuitry configured to
- generate current location information of the movable apparatus to a target position based on GPS,
- transmit to an instructing device the current location information of the movable apparatus, and the image data, the instructing device displaying a map indicating a current location along a planned travel route of the movable apparatus based on the current location information and a visual indicator that indicates a candidate target object in the captured image based on the image data,
- receive from the instructing device feedback information that indicates the candidate target object is correctly identified as the target object in the captured image, the feedback information relating to detected user gesture information that is received to control an operation of the movable apparatus, the visual indicator that indicates the target object being set based on the feedback information, and
- control the movement of the movable apparatus based on the feedback information.
6 more elements
2:
- The movable apparatus according to claim 1, wherein the feedback information is generated based on a command input indicating a selection of the target object.
3:
- The movable apparatus according to claim 1, wherein the driving circuitry controls the movable apparatus to move towards the target object.
4:
- The movable apparatus according to claim 1, wherein the circuitry is further configured to:
- continuously obtain image data of the captured image, the captured image including the target object recognized based on the feedback information.
5:
- The movable apparatus according to claim 4, wherein the circuitry is further configured to:
- continuously transmit the image data of the captured image including the target object.
6:
- The movable apparatus according to claim 1, wherein the circuitry is further configured to:
- control the driving circuitry of the movable apparatus to move the movable apparatus.
7:
- The movable apparatus according to claim 1, wherein the circuitry is further configured to:
- control a moving range of the movable apparatus based on at least the current location information.
8:
- The movable apparatus according to claim 1, wherein the circuitry is further configured to:
- control a moving range of the movable apparatus based on at least the feedback information.
9:
- The movable apparatus according to claim 8, wherein the movable apparatus is controlled based on the moving range of the movable apparatus.
10:
- The movable apparatus according to claim 1, wherein the circuitry is further configured to:
- obtain a distance from the movable apparatus to the target object by a distance sensor of the movable apparatus.
11:
- The movable apparatus according to claim 10, wherein the circuitry is further configured to:
- calculate location information of the target object based on the current location information of the movable apparatus and the distance.
12:
- The movable apparatus according to claim 1, wherein the movable apparatus is controlled based on location information of the target object.
13:
- A method for operating a movable apparatus comprising:
- controllably driving a movement of the movable apparatus;
- capturing with a camera a captured image and producing image data;
- generating GPS information as current location information of the movable apparatus;
- transmitting to an instructing device the current location information of the movable apparatus and the image data, the instructing device displaying a map indicating a current location along a planned travel route of the movable apparatus based on the current location information and a visual indicator that indicates a candidate target object in the display based on the image data;
- receiving from the instructing device feedback information that indicates the candidate target object is correctly identified as the target object in the captured image, the feedback information relating to detected user gesture information that is received to control an operation of the movable apparatus, the visual indicator that indicates the target object based on the feedback information;
- and
- controlling the movement of the movable apparatus and an operation of the camera based on the feedback information.
5 more elements
14:
- The method according to claim 13, wherein the movable apparatus is controlled to move towards the target object.
15:
- The method according to claim 13, further comprising:
- continuously obtaining image data of the captured image, the captured image including the target object recognized based on the feedback information.
16:
- The method according to claim 15, further comprising:
- continuously transmitting the image data of the image including the target object.
17:
- The method according to claim 13, further comprising:
- controlling driving circuitry of the movable apparatus to move the movable apparatus.
18:
- The method according to claim 13, further comprising:
- controlling a moving range of the movable apparatus based on at least the current location information.
19:
- The method according to claim 13, further comprising:
- controlling a moving range of the movable apparatus based on at least the feedback information.
20:
- A non-transitory computer readable medium having a computer program recorded thereon, the computer program configured to perform a method when executed on a computer, the method comprising:
- controllably driving a movement of a movable apparatus;
- capturing with a camera a captured image and producing image data;
- generating GPS information as current location information of the movable apparatus;
- transmitting to an instructing device the current location information of the movable apparatus and the image data, the instructing device displaying a map indicating a current location along a planned travel route of the movable apparatus based on the current location information and a visual indicator that indicates a candidate target object in the based on the image data;
- receiving from the instructing device feedback information that indicates the candidate target object is correctly identified as the target object in the captured image, the feedback information relating to detected user gesture information that is received to control an operation of the movable apparatus, the visual indicator that indicates the target object based on the feedback information;
- and
- controlling the movement of the movable apparatus and an operation of the camera based on the feedback information.
5 more elements