# US 11,029,700 B2 · Mobile robot and control method thereof > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,029,700 B2 - Document ID: US11029700B2 - Status: Granted - Kind: B2 - Application number: 15/745,679 - Filed: 2016-07-29 - Published: 2018-07-26 - Granted: 2021-06-08 - Earliest claimed: 2015-07-29 - Assignees: LG Electronics Inc.; Seoul National University R&Db Foundation - Applicants: LG Electronics Inc.; Seoul National University R&Db Foundation - Inventors: Yongmin Shin; Donghoon Yi; Dongil Cho; Taejae Lee - Web page: https://patentgrep.com/patent/US11029700B2 ## CPC - G05D 1/0253: {extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow (determining position or orientation from images G06T7/70)} - A47L 9/009: {Carrying-vehicles; Arrangements of trollies or wheels; Means for avoiding mechanical obstacles} - A47L 9/19: Means for monitoring filtering operation - A47L 9/2805: {Parameters or conditions being sensed} - A47L 9/2842: {Suction motors or blowers} - A47L 9/2852: {Elements for displacement of the vacuum cleaner or the accessories therefor, e.g. wheels, casters or nozzles} - G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00) - G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)} - G06T 7/11: Region-based segmentation - G06T 7/174: involving the use of two or more images - G06T 7/579: from motion - G06T 7/74: {involving reference images or patches} - A47L 2201/04: Automatic control of the travelling movement; Automatic obstacle detection - G06T 2207/10024: Color image - G06T 2207/20021: Dividing image into blocks, subimages or windows - G06T 2207/30261: Obstacle - Y10S 901/01: Mobile robot - Y10S 901/47: Optical - G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES - A47L: DOMESTIC WASHING OR CLEANING (brushes A46B; cleaning quantities of bottles or of other hollow articles of one kind B08B9/00; laundry D06F); SUCTION CLEANERS IN GENERAL (cleaning in general B08) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS ## Abstract A cleaner performing autonomous traveling includes a main body having a suction opening, a cleaning unit provided within the main body and sucking a cleaning target through the suction opening, a driving unit moving the main body, a camera sensor attached to the main body and capturing a first image, an operation sensor sensing information related to movement of the main body, and a controller detecting information related to an obstacle on the basis of at least one of the captured image and the information related to movement and controlling the driving unit on the basis of the detected information related to the obstacle. ## Claims ### Claim 1 A cleaner comprising: a main body having a suction opening; a cleaning unit provided within the main body and that sucks a cleaning target through the suction opening; a driving unit that moves the main body; a camera sensor that is attached to the main body and captures a first image; an operation sensor that senses information related to movement of the main body; and a controller that detects information related to an obstacle on a basis of at least one of the captured the image or the information related to movement, and controls the driving unit on a basis of the detected information related to the obstacle, wherein the controller segments the captured first image into a plurality of image regions, and detects first information related to the obstacle from the segmented image regions of the first image, and wherein when a preset time interval has lapsed since the first image was captured: the camera sensor captures a second image, and the controller converts the first image, projects the converted image to the second image to generate a third image, and compares the generated third image with the second image to detect second information related to the obstacle. ### Claim 2 (depends on 1) The cleaner of claim 1, wherein the controller segments the second image into a plurality of image regions, compares the segmented image regions of the first image with the segmented image regions of the second image, and detects the first information related to the obstacle on a basis of comparing the segmented image regions of the first image and the segmented image regions of the second image. ### Claim 3 (depends on 2) The cleaner of claim 2, wherein the controller matches at least one of the segmented image regions of the second image and at least one of the segmented image regions of the first image that correspond to each other. ### Claim 4 (depends on 1) The cleaner of claim 1, wherein the controller detects the second information related to the obstacle on a basis of a difference in color between the generated third image and the second image. ### Claim 5 (depends on 1) The cleaner of claim 1, wherein the controller extracts at least one feature point from the first image, and detects third information related to the obstacle on a basis of the extracted feature point. ### Claim 6 (depends on 5) The cleaner of claim 5, wherein the controller sets information related to respective weight values for the first information, the second information, and the third information, and detects fourth information related to the obstacle on a basis of the set weight value, the first information, the second information, and the third information. ### Claim 7 (depends on 5) The cleaner of claim 5, wherein the controller generates map information related to the obstacle using the first information, the second information, and the third information. ### Claim 8 (depends on 1) The cleaner of claim 1, wherein an image capture angle of the camera sensor is an omnidiectional range with respect to the main body. ### Claim 9 (depends on 1) The cleaner of claim 1, wherein the operation sensor includes at least one of a gyro sensor, an acceleration sensor, or a wheel sensor. ### Claim 10 (depends on 1) The cleaner of claim 1, wherein the camera sensor captures the first image in a movement direction of the main body. ### Claim 11 A device comprising: a main body; a motor that moves the main body; a camera sensor that is attached to the main body and captures a first image; an operation sensor that senses information related to movement of the main body; and a controller that detects information related to an obstacle on a basis of at least one of the captured first image and the information related to movement, and controls the motor on a basis of the detected information related to the obstacle, wherein the controller segments the captured first image into a plurality of image regions, and detects first information related to the obstacle from the segmented image regions of the first image, and wherein after a preset time interval has lapsed since the first image was captured: the camera sensor captures a second image, and the controller converts the first image, projects the converted image to the second image to generate a third image, and compares the generated third image with the second image to detect second information related to the obstacle. ### Claim 12 (depends on 11) The device of claim 11, wherein the controller segments the second image into a plurality of image regions, compares the segmented image regions of the first image with the segmented image regions of the second image, and detects the first information related to the obstacle on a basis of comparing the segmented image regions of the first image with the segmented image regions of the second image. ### Claim 13 (depends on 12) The device of claim 12, wherein the controller matches at least one of the segmented image regions of the second image and at least one of the segmented image regions of the first image that correspond to each other. ### Claim 14 (depends on 11) The device of claim 11, wherein the controller detects the second information related to the obstacle on a basis of a difference in color between the generated third image and the second image. ### Claim 15 (depends on 11) The device of claim 11, wherein the controller extracts at least one feature point from the first image, and detects third information related to the obstacle on a basis of the extracted feature point. ### Claim 16 (depends on 15) The device of claim 15, wherein the controller sets information related to respective weight values for the first information, the second information, and the third information, and detects fourth information related to the obstacle on a basis of the set weight values, the first information, the second information, and the third information. ### Claim 17 (depends on 15) The device of claim 15, wherein the controller generates map information related to the obstacle using the first information, the second information, and the third information. ### Claim 18 (depends on 11) The device of claim 11, wherein an image capture angle of the camera sensor is an omnidiectional range with respect to the main body. ### Claim 19 (depends on 11) The device of claim 11, wherein the operation sensor includes at least one of a gyro sensor, an acceleration sensor, or a wheel sensor. ### Claim 20 (depends on 11) The device of claim 11, wherein the camera sensor captures the first image in a movement direction of the main body.