# US 2014/0157156 A1 · Control device, control method, computer program product, and robot control system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2014/0157156 A1 - Document ID: US20140157156A1 - Status: Published application - Kind: A1 - Application number: 14/234,658 - Filed: 2012-07-25 - Published: 2014-06-05 - Earliest claimed: 2011-08-02 - Assignees: Sonny Corporation - Inventors: Kenta Kawamoto; Yoshiaki Iwai - Web page: https://patentgrep.com/patent/US20140157156A1 ## CPC - G05B 15/02: electric - G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)} - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G05B 2219/40099: Graphical user interface for robotics, visual robot user interface - G05B 2219/40123: Indicate, select features on display, remote manipulator will execute - G05B 2219/40409: Robot brings object near operator, operator places object in correct position - G05B 2219/40411: Robot assists human in non-industrial environment like home or office - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A control system, method and computer program product cooperate to assist control for an autonomous robot. The system includes a communications interface that exchanges information with the autonomous robot (22). A user interface displays a scene of location in which the autonomous robot (22) is positioned, and also receives an indication of a user selection of a user selected area within the scene. The communications interface transmits an indication of said user selected area to the autonomous robot (22) for further processing of the area by said autonomous robot (22). ## Claims ### Claim 1 A control system for an autonomous robot, comprising: a communications interface that exchanges information with the autonomous robot; a user interface that displays a scene of a location in which the autonomous robot is positioned, and receives an indication of a user selection of a user selected area within said scene, wherein said communications interface transmits an indication of said user selected area to said autonomous robot for further processing of said area. ### Claim 2 (depends on 1) The control system of claim 1, further comprising: said autonomous robot having a receiver, wherein said autonomous robot moves to another location in response to said receiver receiving said indication of said user selected area. ### Claim 3 (depends on 1) The control system of claim 1, further comprising: a display, wherein said scene is displayed on said display and includes a space diagram of a room and the user selected area is a partial area of said room, said space diagram including objects in said room. ### Claim 4 (depends on 1) The control system of claim 1, further comprising: a receiver that receives images from the autonomous robot; and the autonomous robot, wherein the autonomous robot transmits a second image of the user selected area after transmitting a higher quality image used for selection of the user selected area. ### Claim 5 (depends on 1) The control system of claim 1, further comprising: at least one of a tablet computer and a smartphone that includes said communications interface and a display controller. ### Claim 6 (depends on 1) The control system of claim 1, further comprising: a control unit that generates the indication of said user selected area in response to receiving the indication of the user selection. ### Claim 7 (depends on 2) The control system of claim 2, wherein said autonomous robot is configured to detect a smaller area within said user selected area based on further user input transmitted from the communications interface. ### Claim 8 (depends on 7) The control system of claim 7, wherein said autonomous robot is configured to transmit specified range information that indicates the smaller area that is detected. ### Claim 9 (depends on 1) The control system of claim 1, wherein said scene is a captured image. ### Claim 10 (depends on 1) The control system of claim 1, wherein said scene is a computer generated graphic. ### Claim 11 (depends on 2) The control system of claim 2, wherein the user interface is configured to receive as input a category of a target object within the scene, the communications interface transmits an indication of said category of target object to the autonomous robot, and the autonomous robot attempts to identify within the scene one or more target objects in said category. ### Claim 12 (depends on 11) The control system of claim 11, wherein the autonomous robot assigns degree information for candidate target objects, the degree information being an indication of a likelihood of correct detection of respective target objects in said category. ### Claim 13 (depends on 1) The control system of claim 1, wherein the user interface is configured to receive voice or gesture input commands. ### Claim 14 (depends on 11) The control system of claim 11, further comprising: a display that displays candidate target objects identified by the autonomous robot and user feedback sent via said communications interface to assist in controlling said autonomous robot. ### Claim 15 A display control method for an autonomous robot, comprising: exchanging information with the autonomous robot via a communications interface; displaying on a display a scene of a location in which the autonomous robot is positioned; receiving via a user interface an indication of a user selection of a user selected area within said scene; and forming with a processing circuit and transmitting an indication of said user selected area to said autonomous robot for further processing of said area. ### Claim 16 (depends on 15) The method of claim 15, further comprising: receiving said indication of said user selected area at said autonomous robot; and moving said autonomous robot to another location in response to said receiving. ### Claim 17 (depends on 16) The method of claim 16, wherein detecting with said autonomous robot a smaller area within said user selected area in response to receiving further user input. ### Claim 18 (depends on 15) The control method of claim 15, wherein said scene is a captured image. ### Claim 19 (depends on 15) The control method of claim 15, wherein said scene is a computer generated graphic. ### Claim 20 A non-transitory computer readable storage medium having instructions stored therein that when executed by a processing circuit perform a display control method for an autonomous robot, comprising: exchanging information with the autonomous robot via a communications interface; displaying on a display a scene of a location in which the autonomous robot is positioned; receiving via a user interface an indication of a user selection of a user selected area within said scene; and forming with the processing circuit and transmitting an indication of said user selected area to said autonomous robot for further processing of said area.