# US 12,001,205 B2 · Remote control method and system for robot > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,205 B2 - Document ID: US12001205B2 - Status: Granted - Kind: B2 - Application number: 17/381,800 - Filed: 2021-07-21 - Published: 2022-02-03 - Granted: 2024-06-04 - Earliest claimed: 2020-07-29 - Assignees: Naver Labs Corporation - Applicants: Naver Labs Corporation - Inventors: Kahyeon Kim - Web page: https://patentgrep.com/patent/US12001205B2 ## CPC - G05D 1/0044: {by providing the operator with a computer generated representation of the environment of the vehicle, e.g. virtual reality, maps (maps used for automatic navigation G05D1/0274; flight directors G01C23/005)} - G05D 1/0022: {characterised by the communication link (data switching networks in general H04L12/00)} - G05D 1/0038: {by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation (images analyzed by a computer and used for automatic navigation G05D1/0246)} - G05D 1/224: Output arrangements on the remote controller, e.g. displays, haptics or speakers - G05D 1/225: operated by off-board computers - G05D 1/226: Communication links with the remote-control arrangements - G06F 3/14: Digital output to display device {; Cooperation and interconnection of the display device with other functional units} - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A robot remote control method and system capable of remotely controlling navigation of a robot. The robot remote control method includes outputting both a map image and an ambient image to a display, the map image including location information corresponding to the ambient image, the ambient image being of surroundings of a robot, and the ambient image being received from a camera at the robot, generating a control command for controlling the robot in response to an input to the display during the outputting, and causing the robot to drive according to the control command by transmitting the control command to the robot. ## Claims ### Claim 1 A remote control method for a robot, comprising: outputting, by processing circuitry, both a map image and an ambient image to a display, the map image including location information corresponding to the ambient image, the ambient image being of surroundings of a robot, the ambient image being received from a camera at the robot, and a display area of the display including a main area and a sub-area; receiving, by the processing circuitry, a single input on the display area during the outputting; performing, by the processing circuitry, control operations including both controlling the display area and generating a control command for controlling the robot in response to the single input, the controlling the display including, switching visual information being respectively output to the main area and the sub-area in response to the single input being received on the sub-area, and maintaining the visual information being respectively output to the main area and the sub-area in response to the single input being received on the main area; and causing, by the processing circuitry, the robot to drive according to the control command by transmitting the control command to the robot. ### Claim 2 (depends on 1) The method of claim 1, wherein the sub-area overlaps with a part of the main area; and the outputting includes outputting one of the ambient image or the map image to the main area, and another one of the ambient image or the map image to the sub-area. ### Claim 3 (depends on 2) The method of claim 2, wherein the generating the control command comprises outputting the map image to the main area and the ambient image to the sub-area in response to the input being applied to the sub-area while the map image is being output to the sub-area. ### Claim 4 (depends on 1) The method of claim 1, wherein the map image includes a graphic object indicating the robot; and an output location of the graphic object on the map image corresponds to a current location of the robot. ### Claim 5 (depends on 4) The method of claim 4, further comprising: changing the output location of the graphic object is changed based on a changed position of the robot. ### Claim 6 (depends on 1) The method of claim 1, further comprising: outputting a cursor image on the display, an output location of the cursor image changing in response to the input; and determining navigation attributes of the robot based on a dragging degree of the cursor image and a dragging direction of the cursor image, the dragging degree of the cursor image and the dragging direction of the cursor image corresponding to a change in the output location of the cursor. ### Claim 7 (depends on 6) The method of claim 6, wherein the navigation attributes of the robot include at least one of a moving direction, a moving speed, or a moving distance of the robot; and the determining navigation attributes of the robot determines the moving speed of the robot based on a drag speed of the cursor image, the drag speed of the cursor image corresponding to a change in the output location of the cursor. ### Claim 8 (depends on 1) The method of claim 1, wherein the map image is a first portion of a map image of a space, the map image of the space having a plurality of portions; and the method further comprises determining the first portion from among the plurality of portions based on the ambient image. ### Claim 9 (depends on 8) The method of claim 8, wherein the determining the first portion from among the plurality of portions comprises determining a similarity between the first portion and the ambient image. ### Claim 10 A robot remote control system, comprising: a display having a display area, the display area including a main area and a sub-area; a communication unit configured to receive an ambient image from a camera provided at a robot, the ambient image being of surroundings of the robot; and a controller configured to, control the display to output both a map image and the ambient image, the map image including location information corresponding to the ambient image, receive a single input on the display area during the outputting, perform control operations including both controlling the display area and generating a control command for controlling the robot in response to the single input, the controlling the display including, switching visual information being respectively output to the main area and the sub-area in response to the single input being received on the sub-area, and maintaining the visual information being respectfully output to the main area and the sub-area in response to the single input being received on the main area, and cause the robot to drive according to the control command by controlling the communication unit to transmit the control command to the robot. ### Claim 11 (depends on 10) The robot remote control system of claim 10, wherein the sub-area overlaps with a part of the main area; and the controller is configured to control the display to output one of the ambient image or the map image to the main area, and another one of the ambient image or the map image to the sub-area. ### Claim 12 (depends on 11) The robot remote control system of claim 11, wherein the controller is configured to generate the control command including outputting the map image to the main area and the ambient image to the sub-area in response to the input being applied to the sub-area while the map image is being output to the sub-area. ### Claim 13 (depends on 10) The robot remote control system of claim 10, wherein the map image includes a graphic object indicating the robot; and an output location of the graphic object on the map image corresponds to a current location of the robot. ### Claim 14 (depends on 13) The robot remote control system of claim 13, wherein the controller is configured to control the display to change the output location of the graphic object based on a changed position of the robot. ### Claim 15 (depends on 10) The robot remote control system of claim 10, wherein the controller is configured to: control the display to output a cursor image, an output location of the cursor image changing in response to the input; and determine navigation attributes of the robot based on a dragging degree of the cursor image and a dragging direction of the cursor image, the dragging degree of the cursor image and the dragging direction of the cursor image corresponding to a change in the output location of the cursor. ### Claim 16 (depends on 15) The robot remote control system of claim 15, wherein the navigation attributes of the robot include at least one of a moving direction, a moving speed, or a moving distance of the robot; and the controller is configured to determine the moving speed of the robot based on a drag speed of the cursor image, the drag speed of the cursor image corresponding to a change in the output location of the cursor. ### Claim 17 (depends on 10) The robot remote control system of claim 10, wherein the map image is a first portion of a map image of a space, the map image of the space having a plurality of portions; and the controller is configured to determine the first portion from among the plurality of portions based on the ambient image. ### Claim 18 (depends on 17) The robot remote control system of claim 17, wherein the controller is configured to determine the first portion from among the plurality of portions by determining a similarity between the first portion and the ambient image. ### Claim 19 A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform a remote control method for a robot, the method comprising: outputting, by the at least one processor, both a map image and an ambient image to a display, the map image including location information corresponding to the ambient image, the ambient image being of surroundings of a robot, the ambient image being received from a camera at the robot, and a display area of the display including a main area and a sub-area; receiving, by the at least one processor, a single input on the display area during the outputting; performing, by the at least one processor, control operations including both controlling the display area and generating a control command for controlling the robot in response to the single input, the controlling the display including, switching visual information being respectively output to the main area and the sub-area in response to the single input being received on the sub-area, and maintaining the visual information being respectively output to the main area and the sub-area in response to the single input being received on the main area; and causing, by the at least one processor, the robot to drive according to the control command by transmitting the control command to the robot. ### Claim 20 (depends on 19) The non-transitory computer-readable medium of claim 19, wherein the map image is a first portion of a map image of a space, the map image of the space having a plurality of portions; and the method further comprises determining the first portion from among the plurality of portions based on the ambient image.