# US 12,181,884 B2 · Robot and method for controlling thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,181,884 B2 - Document ID: US12181884B2 - Status: Granted - Kind: B2 - Application number: 17/747,611 - Filed: 2022-05-18 - Published: 2022-09-29 - Granted: 2024-12-31 - Earliest claimed: 2021-03-23 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Aron Baik; Myounggon Kim; Eunsoll Chang; Mideum Choi; Heejun Choi - Web page: https://patentgrep.com/patent/US12181884B2 ## CPC - G05D 1/0214: {in accordance with safety or protection criteria, e.g. avoiding hazardous areas (monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13)} - G05D 1/0248: {in combination with a laser (lasers per se H01S)} - G05D 1/622: Obstacle avoidance (predicting or avoiding probable or impending collision of road vehicles B60W30/08) - G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69) - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A robot includes: a sensor; a driver; a memory storing instructions; and a processor, wherein the processor is configured to execute the instructions to: acquire first scan data including location information of an object around the robot based on a sensing value acquired by the sensor at a first time point, input the first scan data to a neural network model learned to predict scan data to acquire second scan data predicted to be acquired by the robot at a second time point after the first time point, identify a possibility of collision with the object based on location information of the object included in the second scan data; acquire driving data based on the identified possibility of collision; and control the driver based on the acquired driving data. ## Claims ### Claim 1 A robot comprising: a sensor; a driver; a memory storing instructions; and a processor, wherein the processor is configured to execute the instructions to: acquire first scan data including location information of an object around the robot based on a sensing value acquired by the sensor at a first time point, input the first scan data to a neural network model learned to predict scan data to acquire second scan data predicted to be acquired by the robot at a second time point after the first time point, identify a possibility of collision with the object based on location information of the object included in the second scan data; acquire driving data based on the identified possibility of collision; and control the driver based on the acquired driving data, and wherein the processor is further configured to execute the instructions to: group vectors adjacent to each other among a plurality of vectors included in the first scan data in units of a predetermined number, obtain an average value of distance values included in each of the grouped vectors, replace the distance values included in each of the grouped vectors with the obtained average value to perform preprocessing on the first scan data. ### Claim 2 (depends on 1) The robot of claim 1, wherein the first scan data and the second scan data are vector type data including a plurality of angles with respect to a central point of the robot and distance information corresponding to each of the plurality of angles. ### Claim 3 (depends on 1) The robot of claim 1, wherein the sensor comprises a depth sensor, and wherein the processor is further configured to execute the instructions to: acquire a depth image of an area around the robot using the depth sensor, acquire three-dimensional (3D) point clouds based on camera calibration information with respect to the depth image, and identify points corresponding to a predetermined height among the point clouds to acquire the first scan data based on the identified points. ### Claim 4 (depends on 1) The robot of claim 1, wherein the processor is further configured to execute the instructions to, based on a vector including a distance value smaller than a predetermined value being identified among a plurality of vectors included in the second scan data, identify that the possibility of collision with the object at a location corresponding to the identified vector at the second time point is higher than a predetermined possibility. ### Claim 5 (depends on 4) The robot of claim 4, wherein the processor is further configured to acquire driving data for bypassing the location corresponding to the identified vector, and control the driver based on the acquired driving data. ### Claim 6 (depends on 1) The robot of claim 1, wherein the processor is further configured to execute the instructions to: input the preprocessed first scan data into the neural network model to acquire the second scan data. ### Claim 7 (depends on 1) The robot of claim 1, wherein the processor is further configured to execute the instructions to input a plurality of first scan data into the neural network model to acquire the second scan data, and wherein the plurality of first scan data include first-first scan data corresponding to a first-first time point, first-second scan data corresponding to a first-second time point after the first-first time point, and first-third scan data corresponding to a first-third time point after the first-second time point. ### Claim 8 (depends on 1) The robot of claim 1, further comprising: a motion detection sensor, wherein the processor is further configured to execute the instructions to: acquire angular velocity information of the robot based on a sensing value of the motion detection sensor, and input the first scan data and the acquired angular velocity information into the neural network model to acquire the second scan data. ### Claim 9 (depends on 1) The robot of claim 1, wherein the neural network model comprises one of a convolutional neural network (CNN) and a recurrent neural network (RNN). ### Claim 10 (depends on 1) The robot of claim 1, wherein the sensor comprises at least one of a light detection and ranging (LiDAR) sensor, a depth sensor, a camera, and a motion detection sensor. ### Claim 11 (depends on 1) The robot of claim 1, wherein the driver comprises: wheels configured to move the robot; and a motor configured to rotate the wheels. ### Claim 12 (depends on 1) The robot of claim 1, wherein the processor is further configured to acquire the second scan data based on a driving speed of the robot. ### Claim 13 A method for controlling a robot comprising a sensor and a driver, the method comprising: acquiring first scan data including location information of an object around the robot based on a sensing value acquired by the sensor at a first time point; inputting the first scan data to a neural network model learned to predict scan data to acquire second scan data predicted to be acquired by the robot at a second time point after the first time point; identifying a possibility of collision with the object based on location information of the object included in the second scan data; acquiring driving data based on the identified possibility of collision; and controlling the driver based on the acquired driving data, wherein the method further comprises performing preprocessing on the first scan data, and wherein the performing preprocessing comprises: grouping vectors adjacent to each other among a plurality of vectors included in the first scan data in units of a predetermined number; obtaining an average value of distance values included in each of the grouped vectors; and replacing the distance values included in each of the grouped vectors with the obtained average value to perform preprocessing on the first scan data. ### Claim 14 (depends on 13) The method of claim 13, wherein the first scan data and the second scan data are vector type data including a plurality of angles with respect to a central point of the robot and distance information corresponding to each of the plurality of angles. ### Claim 15 (depends on 13) The method of claim 13, wherein the acquiring the first scan data comprises: acquiring a depth image of an area around the robot using a depth sensor; acquiring three-dimensional (3D) point clouds based on camera calibration information with respect to the depth image; and identifying points corresponding to a predetermined height among the point clouds to acquire the first scan data based on the identified points. ### Claim 16 (depends on 13) The method of claim 13, wherein the identifying the possibility of collision with the object comprises, based on a vector including a distance value smaller than a predetermined value being identified among a plurality of vectors included in the second scan data, identifying that the possibility of collision with the object at a location corresponding to the identified vector at the second time point is higher than a predetermined possibility. ### Claim 17 (depends on 16) The method of claim 16, wherein the acquiring the driving data comprises acquiring driving data for bypassing the location corresponding to the identified vector. ### Claim 18 (depends on 13) The method of claim 13, wherein the acquiring the second scan data comprises: inputting the preprocessed first scan data into the neural network model to acquire the second scan data. ### Claim 19 (depends on 13) The method of claim 13, wherein the acquiring the second scan data comprises: inputting a plurality of first scan data into the neural network model to acquire the second scan data, wherein the plurality of first scan data include first-first scan data corresponding to a first-first time point, first-second scan data corresponding to a first-second time point after the first-first time point, and first-third scan data corresponding to a first-third time point after the first-second time point. ### Claim 20 (depends on 13) The method of claim 13, further comprising: acquiring angular velocity information of the robot based on a sensing value of a motion detection sensor, and inputting the first scan data and the acquired angular velocity information into the neural network model to acquire the second scan data.