# US 10,307,912 B2 · Robot cleaner and method for auto-correcting 3D sensor of the robot cleaner

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,307,912 B2
- Document ID: US10307912B2
- Status: Granted
- Kind: B2
- Application number: 14/904,401
- Filed: 2013-07-15
- Published: 2016-05-26
- Granted: 2019-06-04
- Earliest claimed: 2013-07-15
- Assignees: LG Electronics Inc.
- Applicants: LG Electronics Inc.
- Inventors: Yeonsoo Kim
- Web page: https://patentgrep.com/patent/US10307912B2

## CPC

- B25J 11/0085: {Cleaning}
- A47L 9/009: {Carrying-vehicles; Arrangements of trollies or wheels; Means for avoiding mechanical obstacles}
- B25J 9/1674: {characterised by safety, monitoring, diagnostic}
- B25J 9/1694: {characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion}
- B25J 9/1697: {Vision controlled systems}
- G05D 1/024: {in combination with a laser (lasers per se H01S)}
- G05D 1/0248: {in combination with a laser (lasers per se H01S)}
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
- H04N 13/246: Calibration of cameras
- H04N 13/254: in combination with electromagnetic radiation sources for illuminating objects
- H04N 13/271: wherein the generated image signals comprise depth maps or disparity maps
- A47L 2201/02: Docking stations; Docking operations
- A47L 2201/04: Automatic control of the travelling movement; Automatic obstacle detection
- G06T 2207/30252: Vehicle exterior; Vicinity of vehicle
- Y10S 901/01: Mobile robot
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- A47L: DOMESTIC WASHING OR CLEANING (brushes A46B; cleaning quantities of bottles or of other hollow articles of one kind B08B9/00; laundry D06F); SUCTION CLEANERS IN GENERAL (cleaning in general B08)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- Y10S: TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS

## Abstract

A robot cleaner includes a 3D sensor unit installed on a main body to sense nearby objects and output sensing information; a secondary sensor unit configured to sense nearby objects and output sensing information; a storage unit configured to set a diagnostic algorithm according to a diagnostic mode in advance; an input unit configured to input an execution command for the diagnostic mode; a control unit configured to auto-correct the diagnostic mode for the 3D sensor and a parameter of the 3D sensor unit using the diagnostic algorithm in response to the execution command; and an output unit configured to output an execution result of the diagnostic mode and a correction message.

## Claims

### Claim 1

A robot cleaner comprising:
a 3D sensor installed on a main body to sense nearby objects and output sensing information;
a secondary sensor configured to sense nearby objects and output sensing information;
a memory configured to set a diagnostic algorithm according to a diagnostic mode in advance;
an input interface configured to input an execution command for the diagnostic mode;
a controller configured to execute the diagnostic mode for the 3D sensor and auto-correct a parameter of the 3D sensor using the diagnostic algorithm in response to the execution command; and
an output interface configured to output an execution result of the diagnostic mode or a correction message,
wherein the controller is configured to capture a recharging base, position the robot cleaner at a preset point by performing characteristic point matching on the captured recharging base, obtain an image including a laser line at the preset point, and correct the parameter of the 3D sensor based on the laser line displayed on the recharging base.

### Claim 2 (depends on 1)

The robot cleaner according to claim 1, wherein the 3D sensor comprises a laser module that irradiates a target with a laser pattern, and a camera module that obtains an image including the laser pattern.

### Claim 3 (depends on 1)

The robot cleaner according to claim 1, wherein the controller is configured to move the robot cleaner to the preset point to perform correction based on the sensing information from the secondary sensor when as a result of a diagnosis execution of the 3D sensor, there is need to correct the parameter of the 3D sensor.

### Claim 4 (depends on 1)

The robot cleaner according to claim 1, wherein the secondary sensor comprises one or more of an obstacle sensor that signalizes a sensing signal and senses a front obstacle based on a reflected and received signal, a cliff sensor that transmits a sensing signal to a floor surface and senses a cliff based on a reflected and received signal, and a lower camera sensor that takes an image on the floor surface to output image data.

### Claim 5 (depends on 1)

The robot cleaner according to claim 1, further comprising:
a power supply installed at a lower part of the main body and including a rechargeable battery to supply actuating power;
an actuator configured to actuate left and right main wheels provided at opposite sides of a lower part of the main body to move the robot cleaner; and
a cleaner installed at the lower part of the main body and configured to suck up filth or dust from the floor surface or in air.

### Claim 6 (depends on 1)

The robot cleaner according to claim 1, wherein the controller is configured to extract the laser line in the image that is obtained in a state in which a laser beam is projected through the 3D sensor, and diagnose presence or absence of an error to be corrected.

### Claim 7 (depends on 6)

The robot cleaner according to claim 6, wherein it is determined that there is the error to be corrected, when the laser line in the image obtained by the 3D sensor gets out of a designated position.

### Claim 8 (depends on 1)

The robot cleaner according to claim 1, wherein in order to perform the characteristic point matching of the recharging base, the controller obtains an image of the recharging base through a camera module after a laser module is powered off, extracts the characteristic point of the recharging base from the image of the recharging base, and compares the extracted characteristic point of the recharging base with pre-stored image information on the recharging base to perform the characteristic point matching.

### Claim 9 (depends on 1)

The robot cleaner according to claim 1, wherein in order to perform the correction of the parameter of the 3D sensor, the controller extracts pre-stored 3D data on the recharging base according to a position of the laser line obtained from the image, estimates 3D coordinates of the laser line projected to the recharging base based on the extracted 3D data on the recharging base, estimates a laser plane equation from the estimated 3D coordinates of the laser line, and performs correction to the parameter of the 3D sensor by using the estimated laser plane equation.

### Claim 10

A method of automatically correcting a 3D sensor of a robot cleaner, the method comprising:
sensing, by the 3D sensor installed on a main body, nearby objects and outputting sensing information;
sensing, by a secondary sensor, nearby objects and outputting sensing information;
receiving an execution command for a diagnostic mode;
executing the diagnostic mode for the 3D sensor and auto-correcting a parameter of the 3D sensor using a diagnostic algorithm that is preset in order to execute the diagnostic mode, in response to the execution command; and
outputting an execution result of the diagnostic mode or a correction message,
wherein the auto-correcting the parameter of the 3D sensor comprises:
capturing a recharging base;
performing characteristic point matching on the captured recharging base;
positioning the robot cleaner at a preset point using a performance result of the characteristic point matching;
obtaining an image including a laser line at the preset point; and
correcting the parameter of the 3D sensor based on the laser line displayed on the recharging base.

### Claim 11 (depends on 10)

The method according to claim 10, wherein the performing the characteristic point matching on the captured recharging base comprises:
obtaining an image of the recharging base through a camera module after a laser module is powered off;
extracting the characteristic point of the recharging base from the image of the recharging base; and
comparing the extracted characteristic point of the recharging base with pre-stored image information on the recharging base to perform the characteristic point matching.

### Claim 12 (depends on 10)

The method according to claim 10,
wherein the obtaining the image including a laser line at the preset point comprises:
using a camera module to obtain the image including a laser line image after a laser module is powered on, and detecting the laser line existing in the recharging base from the obtained image; and
moving the robot cleaner and then stopping the robot cleaner at a predetermined distance from the recharging base, and obtaining an image taken in a state in which the laser module is powered on and an image taken in a state in which the laser module is powered off to detect the laser line through a difference between two images,
wherein the correcting the parameter of the 3D sensor comprises estimating 3D coordinates of the laser line projected to the recharging base from the detected laser line to perform correction on the parameter of the 3D sensor.

### Claim 13 (depends on 12)

The method according to claim 12, wherein the detecting of the laser line comprises calculating a difference between an image taken in a state in which the laser module is powered on and an image taken in a state in which the laser module is powered off, to detect the laser line.

### Claim 14 (depends on 12)

The method according to claim 12, wherein the performing of the correction on the parameter of the 3D sensor comprises:
extracting pre-stored 3D data on the recharging base according to a position of the laser line obtained from the image;
estimating 3D coordinates of the laser line projected to the recharging base based on the extracted 3D data on the recharging base;
estimating a laser plane equation from the estimated 3D coordinates of the laser line; and
performing correction on the parameter of the 3D sensor by using the estimated laser plane equation.